4,514 research outputs found
DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
Robots are still limited to controlled conditions, that the robot designer
knows with enough details to endow the robot with the appropriate models or
behaviors. Learning algorithms add some flexibility with the ability to
discover the appropriate behavior given either some demonstrations or a reward
to guide its exploration with a reinforcement learning algorithm. Reinforcement
learning algorithms rely on the definition of state and action spaces that
define reachable behaviors. Their adaptation capability critically depends on
the representations of these spaces: small and discrete spaces result in fast
learning while large and continuous spaces are challenging and either require a
long training period or prevent the robot from converging to an appropriate
behavior. Beside the operational cycle of policy execution and the learning
cycle, which works at a slower time scale to acquire new policies, we introduce
the redescription cycle, a third cycle working at an even slower time scale to
generate or adapt the required representations to the robot, its environment
and the task. We introduce the challenges raised by this cycle and we present
DREAM (Deferred Restructuring of Experience in Autonomous Machines), a
developmental cognitive architecture to bootstrap this redescription process
stage by stage, build new state representations with appropriate motivations,
and transfer the acquired knowledge across domains or tasks or even across
robots. We describe results obtained so far with this approach and end up with
a discussion of the questions it raises in Neuroscience
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition
A major goal of artificial intelligence (AI) is to create an agent capable of
acquiring a general understanding of the world. Such an agent would require the
ability to continually accumulate and build upon its knowledge as it encounters
new experiences. Lifelong or continual learning addresses this setting, whereby
an agent faces a continual stream of problems and must strive to capture the
knowledge necessary for solving each new task it encounters. If the agent is
capable of accumulating knowledge in some form of compositional representation,
it could then selectively reuse and combine relevant pieces of knowledge to
construct novel solutions. Despite the intuitive appeal of this simple idea,
the literatures on lifelong learning and compositional learning have proceeded
largely separately. In an effort to promote developments that bridge between
the two fields, this article surveys their respective research landscapes and
discusses existing and future connections between them
A survey of robot manipulation in contact
In this survey, we present the current status on robots performing manipulation tasks that require varying contact with the environment, such that the robot must either implicitly or explicitly control the contact force with the environment to complete the task. Robots can perform more and more manipulation tasks that are still done by humans, and there is a growing number of publications on the topics of (1) performing tasks that always require contact and (2) mitigating uncertainty by leveraging the environment in tasks that, under perfect information, could be performed without contact. The recent trends have seen robots perform tasks earlier left for humans, such as massage, and in the classical tasks, such as peg-in-hole, there is a more efficient generalization to other similar tasks, better error tolerance, and faster planning or learning of the tasks. Thus, in this survey we cover the current stage of robots performing such tasks, starting from surveying all the different in-contact tasks robots can perform, observing how these tasks are controlled and represented, and finally presenting the learning and planning of the skills required to complete these tasks
Simultaneous bilaternal training for improving arm function after stroke
Background Simultaneous bilateral training, the completion of identical activities with both arms simultaneously, is one intervention to improve arm function and reduce impairment. Objectives To determine the effects of simultaneous bilateral training for improving arm function after stroke. Search strategy We searched the Cochrane Stroke Trials Register (last searched August 2009) and 10 electronic bibliographic databases including the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library Issue 3, 2009), MEDLINE, EMBASE, CINAHL and AMED (August 2009). We also searched reference lists and trials registers. Selection criteria Randomised trials in adults after stroke, where the intervention was simultaneous bilateral training compared to placebo or no intervention, usual care or other upper limb (arm) interventions. Primary outcomes were performance in activities of daily living (ADL) and functional movement of the upper limb. Secondary outcomes were performance in extended activities of daily living and motor impairment of the arm. Data collection and analysis Two authors independently screened abstracts, extracted data and appraised trials. Assessment of methodological quality was undertaken for allocation concealment, blinding of outcome assessor, intention-to-treat, baseline similarity and loss to follow up. Main results We included 18 studies involving 549 relevant participants, of which 14 (421 participants) were included in the analysis (one within both comparisons). Four of the 14 studies compared the effects of bilateral training with usual care. Primary outcomes: results were not statistically significant for performance in ADL (standardised mean difference (SMD) 0.25, 95% confidence interval (CI) -0.14 to 0.63); functional movement of the arm (SMD -0.07, 95% CI -0.42 to 0.28) or hand (SMD -0.04, 95% CI -0.50 to 0.42). Secondary outcomes: no statistically significant results. Eleven of the 14 studies compared the effects of bilateral training with other specific upper limb (arm) interventions. Primary outcomes: no statistically significant results for performance of ADL (SMD -0.25, 95% CI -0.57 to 0.08); functional movement of the arm (SMD -0.20, 95% CI -0.49 to 0.09) or hand (SMD -0.21, 95% CI -0.51 to 0.09). Secondary outcomes: one study reported a statistically significant result in favour of another upper limb intervention for performance in extended ADL. No statistically significant differences were found for motor impairment outcomes. Authors' conclusions There is insufficient good quality evidence to make recommendations about the relative effect of simultaneous bilateral training compared to placebo, no intervention or usual care. We identified evidence that suggests that bilateral training may be no more (or less) effective than usual care or other upper limb interventions for performance in ADL, functional movement of the upper limb or motor impairment outcome
Hand eye coordination in surgery
The coordination of the hand in response to visual target selection has always been regarded as an essential quality in a range of professional activities. This quality has thus far been elusive to objective scientific measurements, and is usually engulfed in the overall performance of the individuals. Parallels can be drawn to surgery, especially Minimally Invasive Surgery (MIS), where the physical constraints imposed by the arrangements of the instruments and visualisation methods require certain coordination skills that are unprecedented. With the current paradigm shift towards early specialisation in surgical training and shortened focused training time, selection process should identify trainees with the highest potentials in certain specific skills. Although significant effort has been made in objective assessment of surgical skills, it is only currently possible to measure surgeonsâ abilities at the time of assessment. It has been particularly difficult to quantify specific details of hand-eye coordination and assess innate ability of future skills development. The purpose of this thesis is to examine hand-eye coordination in laboratory-based simulations, with a particular emphasis on details that are important to MIS.
In order to understand the challenges of visuomotor coordination, movement trajectory errors have been used to provide an insight into the innate coordinate mapping of the brain. In MIS, novel spatial transformations, due to a combination of distorted endoscopic image projections and the âfulcrumâ effect of the instruments, accentuate movement generation errors. Obvious differences in the quality of movement trajectories have been observed between novices and experts in MIS, however, this is difficult to measure quantitatively. A Hidden Markov Model (HMM) is used in this thesis to reveal the underlying characteristic movement details of a particular MIS manoeuvre and how such features are exaggerated by the introduction of rotation in the endoscopic camera. The proposed method has demonstrated the feasibility of measuring movement trajectory quality by machine learning techniques without prior arbitrary classification of expertise. Experimental results have highlighted these changes in novice laparoscopic surgeons, even after a short period of training.
The intricate relationship between the hands and the eyes changes when learning a skilled visuomotor task has been previously studied. Reactive eye movement, when visual input is used primarily as a feedback mechanism for error correction, implies difficulties in hand-eye coordination. As the brain learns to adapt to this new coordinate map, eye movements then become predictive of the action generated. The concept of measuring this spatiotemporal relationship is introduced as a measure of hand-eye coordination in MIS, by comparing the Target Distance Function (TDF) between the eye fixation and the instrument tip position on the laparoscopic screen.
Further validation of this concept using high fidelity experimental tasks is presented, where higher cognitive influence and multiple target selection increase the complexity of the data analysis. To this end, Granger-causality is presented as a measure of the predictability of the instrument movement with the eye fixation pattern. Partial Directed Coherence (PDC), a frequency-domain variation of Granger-causality, is used for the first time to measure hand-eye coordination. Experimental results are used to establish the strengths and potential pitfalls of the technique. To further enhance the accuracy of this measurement, a modified Jensen-Shannon Divergence (JSD) measure has been developed for enhancing the signal matching algorithm and trajectory segmentations. The proposed framework incorporates high frequency noise filtering, which represents non-purposeful hand and eye movements. The accuracy of the technique has been demonstrated by quantitative measurement of multiple laparoscopic tasks by expert and novice surgeons.
Experimental results supporting visual search behavioural theory are presented, as this underpins the target selection process immediately prior to visual motor action generation. The effects of specialisation and experience on visual search patterns are also examined. Finally, pilot results from functional brain imaging are presented, where the Posterior Parietal Cortical (PPC) activation is measured using optical spectroscopy techniques. PPC has been demonstrated to involve in the calculation of the coordinate transformations between the visual and motor systems, which establishes the possibilities of exciting future studies in hand-eye coordination
SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation
Learning effective continuous control policies in high-dimensional systems,
including musculoskeletal agents, remains a significant challenge. Over the
course of biological evolution, organisms have developed robust mechanisms for
overcoming this complexity to learn highly sophisticated strategies for motor
control. What accounts for this robust behavioral flexibility? Modular control
via muscle synergies, i.e. coordinated muscle co-contractions, is considered to
be one putative mechanism that enables organisms to learn muscle control in a
simplified and generalizable action space. Drawing inspiration from this
evolved motor control strategy, we use physiologically accurate human hand and
leg models as a testbed for determining the extent to which a Synergistic
Action Representation (SAR) acquired from simpler tasks facilitates learning
more complex tasks. We find in both cases that SAR-exploiting policies
significantly outperform end-to-end reinforcement learning. Policies trained
with SAR were able to achieve robust locomotion on a wide set of terrains with
high sample efficiency, while baseline approaches failed to learn meaningful
behaviors. Additionally, policies trained with SAR on a multiobject
manipulation task significantly outperformed (>70% success) baseline approaches
(<20% success). Both of these SAR-exploiting policies were also found to
generalize zero-shot to out-of-domain environmental conditions, while policies
that did not adopt SAR failed to generalize. Finally, we establish the
generality of SAR on broader high-dimensional control problems using a robotic
manipulation task set and a full-body humanoid locomotion task. To the best of
our knowledge, this investigation is the first of its kind to present an
end-to-end pipeline for discovering synergies and using this representation to
learn high-dimensional continuous control across a wide diversity of tasks.Comment: Accepted to RSS 202
- âŠ