61 research outputs found
Autonomy Infused Teleoperation with Application to BCI Manipulation
Robot teleoperation systems face a common set of challenges including
latency, low-dimensional user commands, and asymmetric control inputs. User
control with Brain-Computer Interfaces (BCIs) exacerbates these problems
through especially noisy and erratic low-dimensional motion commands due to the
difficulty in decoding neural activity. We introduce a general framework to
address these challenges through a combination of computer vision, user intent
inference, and arbitration between the human input and autonomous control
schemes. Adjustable levels of assistance allow the system to balance the
operator's capabilities and feelings of comfort and control while compensating
for a task's difficulty. We present experimental results demonstrating
significant performance improvement using the shared-control assistance
framework on adapted rehabilitation benchmarks with two subjects implanted with
intracortical brain-computer interfaces controlling a seven degree-of-freedom
robotic manipulator as a prosthetic. Our results further indicate that shared
assistance mitigates perceived user difficulty and even enables successful
performance on previously infeasible tasks. We showcase the extensibility of
our architecture with applications to quality-of-life tasks such as opening a
door, pouring liquids from containers, and manipulation with novel objects in
densely cluttered environments
Shared Autonomy via Hindsight Optimization
In shared autonomy, user input and robot autonomy are combined to control a
robot to achieve a goal. Often, the robot does not know a priori which goal the
user wants to achieve, and must both predict the user's intended goal, and
assist in achieving that goal. We formulate the problem of shared autonomy as a
Partially Observable Markov Decision Process with uncertainty over the user's
goal. We utilize maximum entropy inverse optimal control to estimate a
distribution over the user's goal based on the history of inputs. Ideally, the
robot assists the user by solving for an action which minimizes the expected
cost-to-go for the (unknown) goal. As solving the POMDP to select the optimal
action is intractable, we use hindsight optimization to approximate the
solution. In a user study, we compare our method to a standard
predict-then-blend approach. We find that our method enables users to
accomplish tasks more quickly while utilizing less input. However, when asked
to rate each system, users were mixed in their assessment, citing a tradeoff
between maintaining control authority and accomplishing tasks quickly
Comparative analysis of model-based predictive shared control for delayed operation in object reaching and recognition tasks with tactile sensing
Communication delay represents a fundamental challenge in telerobotics: on one hand, it compromises the stability of teleoperated robots, on the other hand, it decreases the user’s awareness of the designated task. In scientific literature, such a problem has been addressed both with statistical models and neural networks (NN) to perform sensor prediction, while keeping the user in full control of the robot’s motion. We propose shared control as a tool to compensate and mitigate the effects of communication delay. Shared control has been proven to enhance precision and speed in reaching and manipulation tasks, especially in the medical and surgical fields. We analyse the effects of added delay and propose a unilateral teleoperated leader-follower architecture that both implements a predictive system and shared control, in a 1-dimensional reaching and recognition task with haptic sensing. We propose four different control modalities of increasing autonomy: non-predictive human control (HC), predictive human control (PHC), (shared) predictive human-robot control (PHRC), and predictive robot control (PRC). When analyzing how the added delay affects the subjects’ performance, the results show that the HC is very sensitive to the delay: users are not able to stop at the desired position and trajectories exhibit wide oscillations. The degree of autonomy introduced is shown to be effective in decreasing the total time requested to accomplish the task. Furthermore, we provide a deep analysis of environmental interaction forces and performed trajectories. Overall, the shared control modality, PHRC, represents a good trade-off, having peak performance in accuracy and task time, a good reaching speed, and a moderate contact with the object of interest
An optimization-based formalism for shared autonomy in dynamic environments
Teleoperation is an integral component of various industrial processes. For
example, concrete spraying, assisted welding, plastering, inspection, and
maintenance. Often these systems implement direct control that maps interface
signals onto robot motions. Successful completion of tasks typically requires
high levels of manual dexterity and cognitive load. In addition, the operator is
often present nearby dangerous machinery. Consequently, safety is of critical
importance and training is expensive and prolonged -- in some cases taking
several months or even years.
An autonomous robot replacement would be an ideal solution since the human could
be removed from danger and training costs significantly reduced. However, this
is currently not possible due to the complexity and unpredictability of the
environments, and the levels of situational and contextual awareness required to
successfully complete these tasks.
In this thesis, the limitations of direct control are addressed by developing
methods for shared autonomy. A shared autonomous approach combines
human input with autonomy to generate optimal robot motions. The approach taken
in this thesis is to formulate shared autonomy within an optimization framework
that finds optimized states and controls by minimizing a cost function, modeling
task objectives, given a set of (changing) physical and operational constraints.
Online shared autonomy requires the human to be continuously interacting with
the system via an interface (akin to direct control). The key challenges
addressed in this thesis are: 1) ensuring computational feasibility (such a
method should be able to find solutions fast enough to achieve a sampling
frequency bound below by 40Hz), 2) being reactive to changes in the
environment and operator intention, 3) knowing how to appropriately blend
operator input and autonomy, and 4) allowing the operator to supply input in an
intuitive manner that is conducive to high task performance.
Various operator interfaces are investigated with regards to the control space,
called a mode of teleoperation. Extensive evaluations were carried out
to determine for which modes are most intuitive and lead to highest performance
in target acquisition tasks (e.g. spraying/welding/etc). Our performance metrics
quantified task difficulty based on Fitts' law, as well as a measure of how well
constraints affecting the task performance were met. The experimental
evaluations indicate that higher performance is achieved when humans submit
commands in low-dimensional task spaces as opposed to joint space manipulations.
In addition, our multivariate analysis indicated that those with regular
exposure to computer games achieved higher performance.
Shared autonomy aims to relieve human operators of the burden of precise motor
control, tracking, and localization. An optimization-based representation for
shared autonomy in dynamic environments was developed. Real-time tractability is
ensured by modulating the human input with information of the changing
environment within the same task space, instead of adding it to the optimization
cost or constraints. The method was illustrated with two real world
applications: grasping objects in cluttered environments and spraying tasks
requiring sprayed linings with greater homogeneity.
Maintaining motion patterns -- referred to as skills -- is often an
integral part of teleoperation for various industrial processes (e.g. spraying,
welding, plastering). We develop a novel model-based shared autonomous framework
for incorporating the notion of skill assistance to aid operators to sustain
these motion patterns whilst adhering to environment constraints. In order to
achieve computational feasibility, we introduce a novel parameterization for
state and control that combines skill and underlying trajectory models,
leveraging a special type of curve known as Clothoids. This new parameterization
allows for efficient computation of skill-based short term horizon plans,
enabling the use of a model predictive control loop. Our hardware realization
validates the effectiveness of our method to recognize a change of intended
skill, and showing an improved quality of output motion, even under dynamically
changing obstacles.
In addition, extensions of the work to supervisory control are described. An
exploratory study presents an approach that improves computational feasibility
for complex tasks with minimal interactive effort on the part of the human.
Adaptations are theorized which might allow such a method to be applicable and
beneficial to high degree of freedom systems. Finally, a system developed in our
lab is described that implements sliding autonomy and shown to complete
multi-objective tasks in complex environments with minimal interaction from the
human
- …