De Novo Learning of Motor Skills

From playing the piano to driving a car, humans acquire a wide range of motor skills throughout their lifetimes. How are people capable of learning such a wide repertoire of skills? Studies in motor learning have attempted to address this question by examining "adaptation", a trial-by-trial learning mechanism where movements are updated via the reduction of sensory prediction errors. However, a growing body of literature suggests that adaptation alone cannot account for how people learn many real-world skills. It has instead been hypothesized that the brain acquires many new skills by building a new motor controller "de novo". Currently, little is understood about de novo learning as most prior studies of motor learning have focused on investigating adaptation. In this dissertation, we performed a series of experiments to characterize the nature of de novo learned controllers. First, we devised a novel frequency-domain system identification approach to characterize how people learn to compensate for visuomotor perturbations. We used this approach to demonstrate that people learn skills which require continuous movement output—such as riding a bike or juggling—via de novo learning. We then designed a challenging de novo learning task which involved controlling an on-screen cursor using a bimanual mapping. In contrast to many laboratory-based motor learning tasks which can be learned on the timescale of minutes, participants required multiple days of practice to learn the bimanual mapping. In this task, we found that participants' responses to mid-movement perturbations remained limited after four days of practice, suggesting that limitations in one's ability to select appropriate actions may contribute to performance plateaus during learning. Finally, we used the same bimanual mapping to understand how de novo learned skills become habitual. We found that participants' behavior could continue to become more skillful despite the fact that it had already become habitual, suggesting that the emergence of skill and habit are dissociable during learning. Collectively, our results illustrate the behavioral phenomenology associated with de novo learned controllers and highlight the critical role that de novo learning plays when people learn real-world skills

Reward and punishment: the neural correlates of reinforcement feedback during motor learning

‘By the carrot or the stick’ reward or punishment has been contemplated by instructors to motivate their pupils to learn a new motor skill. The reinforcements of reward and punishment have demonstrated dissociable effects on motor learning with punishment enhancing the learning rate and reward increasing retention of the motor task. However it is still unclear how the brain processes reward and punishment during motor learning. This study sought to investigate the role of reinforcement feedback in cortical neural activity associated with motor learning. A novel visuomotor rotation task was employed with reward punishment or null feedback as the participants adapted their movement to a 30-degree counter-clockwise rotation. We measured movement time and task accuracy throughout the task. Surface electroencephalography was utilized to record cortical neural activity throughout the learning and retention of the motor task. Event-related potentials (ERPs) were calculated to assess how the brain processes the reinforcement feedback and prepares for movement. Repeated measures ANOVAs were utilized to detect differences in the movement parameters and ERP amplitudes. This study found that reward and punishment feedback did not produce different effects on the rate of task learning. However punishment feedback impaired the retention (memory) of the motor task. These behavioral effects were accompanied by changes in the amplitude of ERPs during feedback presentation and movement preparation. These results suggest that punishment feedback alters brain processes involved in memory formation during motor learning

Brain-wave measures of workload in advanced cockpits: The transition of technology from laboratory to cockpit simulator, phase 2

The present Phase 2 small business innovation research study was designed to address issues related to scalp-recorded event-related potential (ERP) indices of mental workload and to transition this technology from the laboratory to cockpit simulator environments for use as a systems engineering tool. The project involved five main tasks: (1) Two laboratory studies confirmed the generality of the ERP indices of workload obtained in the Phase 1 study and revealed two additional ERP components related to workload. (2) A task analysis' of flight scenarios and pilot tasks in the Advanced Concepts Flight Simulator (ACFS) defined cockpit events (i.e., displays, messages, alarms) that would be expected to elicit ERPs related to workload. (3) Software was developed to support ERP data analysis. An existing ARD-proprietary package of ERP data analysis routines was upgraded, new graphics routines were developed to enhance interactive data analysis, and routines were developed to compare alternative single-trial analysis techniques using simulated ERP data. (4) Working in conjunction with NASA Langley research scientists and simulator engineers, preparations were made for an ACFS validation study of ERP measures of workload. (5) A design specification was developed for a general purpose, computerized, workload assessment system that can function in simulators such as the ACFS

Potential of a suite of robot/computer-assisted motivating systems for personalized, home-based, stroke rehabilitation

BACKGROUND: There is a need to improve semi-autonomous stroke therapy in home environments often characterized by low supervision of clinical experts and low extrinsic motivation. Our distributed device approach to this problem consists of an integrated suite of low-cost robotic/computer-assistive technologies driven by a novel universal access software framework called UniTherapy. Our design strategy for personalizing the therapy, providing extrinsic motivation and outcome assessment is presented and evaluated. METHODS: Three studies were conducted to evaluate the potential of the suite. A conventional force-reflecting joystick, a modified joystick therapy platform (TheraJoy), and a steering wheel platform (TheraDrive) were tested separately with the UniTherapy software. Stroke subjects with hemiparesis and able-bodied subjects completed tracking activities with the devices in different positions. We quantify motor performance across subject groups and across device platforms and muscle activation across devices at two positions in the arm workspace. RESULTS: Trends in the assessment metrics were consistent across devices with able-bodied and high functioning strokes subjects being significantly more accurate and quicker in their motor performance than low functioning subjects. Muscle activation patterns were different for shoulder and elbow across different devices and locations. CONCLUSION: The Robot/CAMR suite has potential for stroke rehabilitation. By manipulating hardware and software variables, we can create personalized therapy environments that engage patients, address their therapy need, and track their progress. A larger longitudinal study is still needed to evaluate these systems in under-supervised environments such as the home

Sensorimotor learning under switching dynamics

Humans have a remarkable capacity to learn new motor behaviours without forgetting old ones. This capacity relies on the ability to acquire and express multiple motor memories without interference. Here we combine behavioural experiments and computational modelling to investigate how the sensorimotor system uses contextual information to create, update and recall motor memories. We first examine the role of muscle co-contraction in the learning of novel movement dynamics. We show that muscle co-contraction, as measured by surface electromyography, accelerates motor learning. We then explore the role of control points on objects in the formation of motor memories during object manipulation. We show that opposing dynamic perturbations, which interfere when controlling a single location on an object, can be learned when each is associated with a separate control point. To account for these results, we develop a parametric switching state-space model, in which the association between cues (control points) and contexts (dynamics) is learned from experience rather than fixed. We then extend this model to a Bayesian nonparametric switching state-space model, in which the number of contexts and cues are learned online rather than specified in advance. This model can instantiate new memories when novel perturbations are experienced and exhibits spontaneous recovery of a memory that has been ostensibly overwritten. To test the model, we perform an experiment in which we briefly present a previously experienced perturbation after behaviour has returned to baseline. As predicted, we observe a qualitatively distinct and more pronounced form of recovery, which we refer to as evoked recovery. Finally, we investigate Bayesian context estimation using single-trial learning. We show that people are able to learn novel associations between cues and contexts and that they use both contextual cues and state feedback to infer the current context and partition learning between memories. Taken together, these findings further the understanding of the behaviour and computational principles of sensorimotor learning under switching dynamics.Engineering and Physical Sciences Research Counci

Principles of sensorimotor control and learning in complex motor tasks

The brain coordinates a continuous coupling between perception and action in the presence of uncertainty and incomplete knowledge about the world. This mapping is enabled by control policies and motor learning can be perceived as the update of such policies on the basis of improving performance given some task objectives. Despite substantial progress in computational sensorimotor control and empirical approaches to motor adaptation, to date it remains unclear how the brain learns motor control policies while updating its internal model of the world. In light of this challenge, we propose here a computational framework, which employs error-based learning and exploits the brain’s inherent link between forward models and feedback control to compute dynamically updated policies. The framework merges optimal feedback control (OFC) policy learning with a steady system identification of task dynamics so as to explain behavior in complex object manipulation tasks. Its formalization encompasses our empirical findings that action is learned and generalised both with regard to a body-based and an object-based frame of reference. Importantly, our approach predicts successfully how the brain makes continuous decisions for the generation of complex trajectories in an experimental paradigm of unfamiliar task conditions. A complementary method proposes an expansion of the motor learning perspective at the level of policy optimisation to the level of policy exploration. It employs computational analysis to reverse engineer and subsequently assess the control process in a whole body manipulation paradigm. Another contribution of this thesis is to associate motor psychophysics and computational motor control to their underlying neural foundation; a link which calls for further advancement in motor neuroscience and can inform our theoretical insight to sensorimotor processes in a context of physiological constraints. To this end, we design, build and test an fMRI-compatible haptic object manipulation system to relate closed-loop motor control studies to neurophysiology. The system is clinically adjusted and employed to host a naturalistic object manipulation paradigm on healthy human subjects and Friedreich’s ataxia patients. We present methodology that elicits neuroimaging correlates of sensorimotor control and learning and extracts longitudinal neurobehavioral markers of disease progression (i.e. neurodegeneration). Our findings enhance the understanding of sensorimotor control and learning mechanisms that underlie complex motor tasks. They furthermore provide a unified methodological platform to bridge the divide between behavior, computation and neural implementation with promising clinical and technological implications (e.g. diagnostics, robotics, BMI).Open Acces

Aerospace Medicine and Biology: A continuing bibliography with indexes (supplement 141)

This special bibliography lists 267 reports, articles, and other documents introduced into the NASA scientific and technical information system in April 1975

Mechanisms of motor learning: by humans, for robots

Whenever we perform a movement and interact with objects in our environment, our central nervous system (CNS) adapts and controls the redundant system of muscles actuating our limbs to produce suitable forces and impedance for the interaction. As modern robots are increasingly used to interact with objects, humans and other robots, they too require to continuously adapt the interaction forces and impedance to the situation. This thesis investigated the motor mechanisms in humans through a series of technical developments and experiments, and utilized the result to implement biomimetic motor behaviours on a robot. Original tools were first developed, which enabled two novel motor imaging experiments using functional magnetic resonance imaging (fMRI). The first experiment investigated the neural correlates of force and impedance control to understand the control structure employed by the human brain. The second experiment developed a regressor free technique to detect dynamic changes in brain activations during learning, and applied this technique to investigate changes in neural activity during adaptation to force fields and visuomotor rotations. In parallel, a psychophysical experiment investigated motor optimization in humans in a task characterized by multiple error-effort optima. Finally a computational model derived from some of these results was implemented to exhibit human like control and adaptation of force, impedance and movement trajectory in a robot

Neural State Changes in Primate Motor Cortex During Arm Movements with Distinct Control Requirements

The primary motor cortex (M1) is an important structure of the motor system that contributes to many aspects of movement. Firing patterns of M1 neurons can be surprisingly complex, and there is substantial interest in understanding these patterns and their relation to behavior. Here, we characterize the temporal structure of M1 activity during reaching in several ways. First, we show that single neurons encode movement information in a series of discrete segments. Information is stably encoded during each brief segment, and the firing patterns of most neurons transition between segments at similar times during movement. This pattern may therefore reflect transitions between different neural “states.” Next, we establish that the sequence of states observed during behavior is related to a sequence of distinct drivers, including visuospatial information and visual feedback from a movement. If no feedback is provided, neurons may produce a truncated response sequence. Last, we link the temporal structure of firing patterns to the structure of reaches and demonstrate that the classical two-component model of reaching is reflected in M1 activity. Our findings may help establish a useful framework for interpreting seemingly complex neural activity during behavior