Visuomotor Coordination in Reach-To-Grasp Tasks: From Humans to Humanoids and Vice Versa

Understanding the principles involved in visually-based coordinated motor control is one of the most fundamental and most intriguing research problems across a number of areas, including psychology, neuroscience, computer vision and robotics. Not very much is known regarding computational functions that the central nervous system performs in order to provide a set of requirements for visually-driven reaching and grasping. Additionally, in spite of several decades of advances in the field, the abilities of humanoids to perform similar tasks are by far modest when needed to operate in unstructured and dynamically changing environments. More specifically, our first focus is understanding the principles involved in human visuomotor coordination. Not many behavioral studies considered visuomotor coordination in natural, unrestricted, head-free movements in complex scenarios such as obstacle avoidance. To fill this gap, we provide an assessment of visuomotor coordination when humans perform prehensile tasks with obstacle avoidance, an issue that has received far less attention. Namely, we quantify the relationships between the gaze and arm-hand systems, so as to inform robotic models, and we investigate how the presence of an obstacle modulates this pattern of correlations. Second, to complement these observations, we provide a robotic model of visuomotor coordination, with and without the presence of obstacles in the workspace. The parameters of the controller are solely estimated by using the human motion capture data from our human study. This controller has a number of interesting properties. It provides an efficient way to control the gaze, arm and hand movements in a stable and coordinated manner. When facing perturbations while reaching and grasping, our controller adapts its behavior almost instantly, while preserving coordination between the gaze, arm, and hand. In the third part of the thesis, we study the neuroscientific literature of the primates. We here stress the view that the cerebellum uses the cortical reference frame representation. The cerebellum by taking into account this representation performs closed-loop programming of multi-joint movements and movement synchronization between the eye-head system, arm and hand. Based on this investigation, we propose a functional architecture of the cerebellar-cortical involvement. We derive a number of improvements of our visuomotor controller for obstacle-free reaching and grasping. Because this model is devised by carefully taking into account the neuroscientific evidence, we are able to provide a number of testable predictions about the functions of the central nervous system in visuomotor coordination. Finally, we tackle the flow of the visuomotor coordination in the direction from the arm-hand system to the visual system. We develop two models of motor-primed attention for humanoid robots. Motor-priming of attention is a mechanism that implements prioritizing of visual processing with respect to motor-relevant parts of the visual field. Recent studies in humans and monkeys have shown that visual attention supporting natural behavior is not exclusively defined in terms of visual saliency in color or texture cues, rather the reachable space and motor plans present the predominant source of this attentional modulation. Here, we show that motor-priming of visual attention can be used to efficiently distribute robot's computational resources devoted to visual processing

User Based Development and Test of the EXOTIC Exoskeleton:Empowering Individuals with Tetraplegia Using a Compact, Versatile, 5-DoF Upper Limb Exoskeleton Controlled through Intelligent Semi-Automated Shared Tongue Control

This paper presents the EXOTIC- a novel assistive upper limb exoskeleton for individuals with complete functional tetraplegia that provides an unprecedented level of versatility and control. The current literature on exoskeletons mainly focuses on the basic technical aspects of exoskeleton design and control while the context in which these exoskeletons should function is less or not prioritized even though it poses important technical requirements. We considered all sources of design requirements, from the basic technical functions to the real-world practical application. The EXOTIC features: (1) a compact, safe, wheelchair-mountable, easy to don and doff exoskeleton capable of facilitating multiple highly desired activities of daily living for individuals with tetraplegia; (2) a semi-automated computer vision guidance system that can be enabled by the user when relevant; (3) a tongue control interface allowing for full, volitional, and continuous control over all possible motions of the exoskeleton. The EXOTIC was tested on ten able-bodied individuals and three users with tetraplegia caused by spinal cord injury. During the tests the EXOTIC succeeded in fully assisting tasks such as drinking and picking up snacks, even for users with complete functional tetraplegia and the need for a ventilator. The users confirmed the usability of the EXOTIC

Visual Neuroscience of Robotic Grasping

Supporting Informatio

Visual control of bimanual movements.

Goal directed reaching forms an integral part of human routine movements, and we have a remarkable faculty to perform such actions with both upper limbs and coordinating these to achieve individual and collective outcomes. Everyday actions such as eating using a knife and fork, tying shoelaces, typing on a keyboard distort the complexity involved in the nature of timing synchrony and coordination that occurs between the two limbs. Several factors that can affect the synchrony between limbs during concurrent bimanual movements are; task difficulty, required movement symmetry, competition between limbs for visual resources, hand dominance and impairment to motor or visual system. This thesis explores these factors through a series of experiments in both young and older unimpaired individuals as well as those with limb impairment as a result of stroke. Although observations in relation to movement of the upper limbs and their coordination have been recorded throughout written history, it is during the last few decades where the majority of related empirical research has been undertaken. How the brain controls and coordinates movement remains an important yet inconclusive area in motor control literature thus far, however, it grows as a topic of research due to more advanced technological capabilities and implications for upper limb movement disorder rehabilitation. Studies of the upper limb have considered the spatial and temporal properties of unimanual and bimanual movements; exploring the interaction between the two limbs during bimanual movements. In movements to two separate targets, movement time symmetry (temporal symmetry) has been observed between the two limbs, where the movements are initiated and terminated in similar timing. However, as the relative precision requirements and thus difficulty of the required movement to two separate targets increases, inter-limb coordination may be disrupted. To date, motor control research has failed to establish specific factors that are involved in the integration of the two limbs for bimanual coordination. As well as addressing the interaction between the two limbs, this thesis explores the contribution made by overt and covert visual attention to the control of visual guided upper limb movements with a focus on the coordination between the two limbs. It also explores related performance in stroke survivors with hemiparesis along with an older adults control group; in doing so, this research in the first to explore the important function of visually-guided bimanual movements while examining both eye and limb movements in a clinical population. This thesis is organised into three individual yet interconnected experimental chapters. Following introduction of the key themes motivating the research and related relevant literature (Chapter 1), a general methods section (Chapter 2) describes the development and details of the underlying experimental paradigm and protocol used in all the experimental chapters. Modifications to this basic approach are detailed in the methods sections of individual experimental chapters. Next, the experimental chapters are presented (Chapter 3, 4 and 5). Experiment 1 examines visual control and coordination of the limbs during unimanual and bimanual reaching movements in young left and righthanded adults (Chapter 3). Next, the experimental protocol was changed to restrict the visual control of upper limb movements and the motor coordination between the two limbs was studied (Chapter 4). Unimanual and bimanual movements were examined while participants maintained visual fixation (i.e. without eye movements), and any errant saccades were monitored in addition to the measures gathered in Chapter 3. The third experiment (Chapter 5) examined unimanual and bimanual control and coordination in participants following hemiparetic stroke and compared their performance with a group of age-matched control participants. A general discussion with conclusions and future directions is presented in Chapter 6