Effects of Human Cerebellar Thalamus Disruption on Adaptive Control of Reaching

Lesion or degeneration of the cerebellum can profoundly impair adaptive control of reaching in humans. Computational models have proposed that internal models that help control movements form in the cerebellum and influence planned motor output through the cerebello-thalamo-cortical pathway. However, lesion studies of the cerebellar thalamus have not consistently found impairment in reaching or adaptation of reaching. To elucidate the role of the cerebellar thalamus in humans, we studied a group of essential tremor (ET) patients with deep brain stimulation (DBS) electrodes placed in the cerebellar thalamus. The stimulation can be turned on or off remotely and is thought to reduce tremor by blocking the spread of the pathological output from the cerebellum. We studied the effect of thalamic DBS on the ability to adapt arm movements to novel force fields. Although thalamic DBS resulted in a dramatic and significant reduction of tremor in ET, it also impaired motor adaptation: the larger the stimulation voltage, the greater the reduction in rates of adaptation. We next examined ET patients that had undergone unilateral thalamotomy in the cerebellar thalamus and found that adaptation with the contralateral arm was impaired compared with the ipsilateral arm. Therefore, although both lesion and electrical stimulation of the cerebellar thalamus are highly effective in reducing tremor, they significantly impair the ability of the brain to form internal models of action. Adaptive control of reaching appears to depend on the integrity of the cerebello-thalamo-cortical pathway.Engineering and Applied Science

A new method for tracking of motor skill learning through practical application of Fitts’ law

This article is made available through the Brunel Open Access Publishing Fund.A novel upper limb motor skill measure, task productivity rate (TPR) was developed integrating speed and spatial error, delivered by a practical motor skill rehabilitation task (MSRT). This prototype task involved placement of 5 short pegs horizontally on a spatially configured rail array. The stability of TPR was tested on 18 healthy right-handed adults (10 women, 8 men, median age 29 years) in a prospective single-session quantitative within-subjects study design. Manipulations of movement rate 10% faster and slower relative to normative states did not significantly affect TPR, F(1.387, 25.009) = 2.465, p = .121. A significant linear association between completion time and error was highest during the normative state condition (Pearson's r = .455, p < .05). Findings provided evidence that improvements in TPR over time reflected motor learning with possible changes in coregulation behavior underlying practice under different conditions. These findings extend Fitts’ law theory to tracking of practical motor skill using a dexterity task, which could have potential clinical applications in rehabilitation

Estimating the Relevance of World Disturbances to Explain Savings, Interference and Long-Term Motor Adaptation Effects

Recent studies suggest that motor adaptation is the result of multiple, perhaps linear processes each with distinct time scales. While these models are consistent with some motor phenomena, they can neither explain the relatively fast re-adaptation after a long washout period, nor savings on a subsequent day. Here we examined if these effects can be explained if we assume that the CNS stores and retrieves movement parameters based on their possible relevance. We formalize this idea with a model that infers not only the sources of potential motor errors, but also their relevance to the current motor circumstances. In our model adaptation is the process of re-estimating parameters that represent the body and the world. The likelihood of a world parameter being relevant is then based on the mismatch between an observed movement and that predicted when not compensating for the estimated world disturbance. As such, adapting to large motor errors in a laboratory setting should alert subjects that disturbances are being imposed on them, even after motor performance has returned to baseline. Estimates of this external disturbance should be relevant both now and in future laboratory settings. Estimated properties of our bodies on the other hand should always be relevant. Our model demonstrates savings, interference, spontaneous rebound and differences between adaptation to sudden and gradual disturbances. We suggest that many issues concerning savings and interference can be understood when adaptation is conditioned on the relevance of parameters

Adaptive Optimal Feedback Control with Learned Internal Dynamics Models

Optimal Feedback Control (OFC) has been proposed as an attractive movement generation strategy in goal reaching tasks for anthropomorphic manipulator systems. Recent developments, such as the Iterative Linear Quadratic Gaussian (ILQG) algorithm, have focused on the case of non-linear, but still analytically available, dynamics. For realistic control systems, however, the dynamics may often be unknown, difficult to estimate, or subject to frequent systematic changes. In this chapter, we combine the ILQG framework with learning the forward dynamics for simulated arms, which exhibit large redundancies, both, in kinematics and in the actuation. We demonstrate how our approach can compensate for complex dynamic perturbations in an online fashion. The specific adaptive framework introduced lends itself to a computationally more efficient implementation of the ILQG optimisation without sacrificing control accuracy – allowing the method to scale to large DoF systems

A biologically inspired neural network controller for ballistic arm movements

<p>Abstract</p> <p>Background</p> <p>In humans, the implementation of multijoint tasks of the arm implies a highly complex integration of sensory information, sensorimotor transformations and motor planning. Computational models can be profitably used to better understand the mechanisms sub-serving motor control, thus providing useful perspectives and investigating different control hypotheses. To this purpose, the use of Artificial Neural Networks has been proposed to represent and interpret the movement of upper limb. In this paper, a neural network approach to the modelling of the motor control of a human arm during planar ballistic movements is presented.</p> <p>Methods</p> <p>The developed system is composed of three main computational blocks: 1) a parallel distributed learning scheme that aims at simulating the internal inverse model in the trajectory formation process; 2) a pulse generator, which is responsible for the creation of muscular synergies; and 3) a limb model based on two joints (two degrees of freedom) and six muscle-like actuators, that can accommodate for the biomechanical parameters of the arm. The learning paradigm of the neural controller is based on a pure exploration of the working space with no feedback signal. Kinematics provided by the system have been compared with those obtained in literature from experimental data of humans.</p> <p>Results</p> <p>The model reproduces kinematics of arm movements, with bell-shaped wrist velocity profiles and approximately straight trajectories, and gives rise to the generation of synergies for the execution of movements. The model allows achieving amplitude and direction errors of respectively 0.52 cm and 0.2 radians.</p> <p>Curvature values are similar to those encountered in experimental measures with humans.</p> <p>The neural controller also manages environmental modifications such as the insertion of different force fields acting on the end-effector.</p> <p>Conclusion</p> <p>The proposed system has been shown to properly simulate the development of internal models and to control the generation and execution of ballistic planar arm movements. Since the neural controller learns to manage movements on the basis of kinematic information and arm characteristics, it could in perspective command a neuroprosthesis instead of a biomechanical model of a human upper limb, and it could thus give rise to novel rehabilitation techniques.</p

Control of position and movement is simplified by combined muscle spindle and Golgi tendon organ feedback

Whereas muscle spindles play a prominent role in current theories of human motor control, Golgi tendon organs (GTO) and their associated tendons are often neglected. This is surprising since there is ample evidence that both tendons and GTOs contribute importantly to neuromusculoskeletal dynamics. Using detailed musculoskeletal models, we provide evidence that simple feedback using muscle spindles alone results in very poor control of joint position and movement since muscle spindles cannot sense changes in tendon length that occur with changes in muscle force. We propose that a combination of spindle and GTO afferents can provide an estimate of muscle-tendon complex length, which can be effectively used for low-level feedback during both postural and movement tasks. The feasibility of the proposed scheme was tested using detailed musculoskeletal models of the human arm. Responses to transient and static perturbations were simulated using a 1-degree-of-freedom (DOF) model of the arm and showed that the combined feedback enabled the system to respond faster, reach steady state faster, and achieve smaller static position errors. Finally, we incorporated the proposed scheme in an optimally controlled 2-DOF model of the arm for fast point-to-point shoulder and elbow movements. Simulations showed that the proposed feedback could be easily incorporated in the optimal control framework without complicating the computation of the optimal control solution, yet greatly enhancing the system's response to perturbations. The theoretical analyses in this study might furthermore provide insight about the strong physiological couplings found between muscle spindle and GTO afferents in the human nervous system. © 2013 the American Physiological Society

Expressions of Multiple Neuronal Dynamics during Sensorimotor Learning in the Motor Cortex of Behaving Monkeys

Previous studies support the notion that sensorimotor learning involves multiple processes. We investigated the neuronal basis of these processes by recording single-unit activity in motor cortex of non-human primates (Macaca fascicularis), during adaptation to force-field perturbations. Perturbed trials (reaching to one direction) were practiced along with unperturbed trials (to other directions). The number of perturbed trials relative to the unperturbed ones was either low or high, in two separate practice schedules. Unsurprisingly, practice under high-rate resulted in faster learning with more pronounced generalization, as compared to the low-rate practice. However, generalization and retention of behavioral and neuronal effects following practice in high-rate were less stable; namely, the faster learning was forgotten faster. We examined two subgroups of cells and showed that, during learning, the changes in firing-rate in one subgroup depended on the number of practiced trials, but not on time. In contrast, changes in the second subgroup depended on time and practice; the changes in firing-rate, following the same number of perturbed trials, were larger under high-rate than low-rate learning. After learning, the neuronal changes gradually decayed. In the first subgroup, the decay pace did not depend on the practice rate, whereas in the second subgroup, the decay pace was greater following high-rate practice. This group shows neuronal representation that mirrors the behavioral performance, evolving faster but also decaying faster at learning under high-rate, as compared to low-rate. The results suggest that the stability of a new learned skill and its neuronal representation are affected by the acquisition schedule.United States-Israel Binational Science FoundationIsrael Science FoundationIda Baruch FundRosetrees Trus

Somatosensory Comparison during Haptic Tracing

Active sensing involves memory retrieval and updating as well as mechanisms that trigger corrections to the ongoing exploratory movement. The present study examined this process in a task where human subjects moved the index fingertip clockwise around the circumference of a virtual sphere created by a robotic device. The fingertip pressed into the sphere during the movement, and the subjects were to report slight differences in sphere size (or surface curvature), which occurred from trial to trial. During each 2- to 3-s trial, subjects gradually adjusted their speed and pressure according to the current surface curvature, achieving a consistent level of contact force in the last half of the exploration. The results demonstrate that subjects were gradually accumulating haptic information about curvature and, at the same time, gradually changing the motor commands for the movement. When subjects encountered an unexpected transition in curvature (from circular to flat), they reacted by abruptly decreasing contact force at a latency of about 50 ms. This short latency indicates that spinally mediated corrections are engaged during this task. The results support the hypothesis that during haptic exploration, the neural comparison between expected and actual somatosensory feedback takes places at multiple levels, including the spinal cord

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

A Single-Rate Context-Dependent Learning Process Underlies Rapid Adaptation to Familiar Object Dynamics

Motor learning has been extensively studied using dynamic (force-field) perturbations. These induce movement errors that result in adaptive changes to the motor commands. Several state-space models have been developed to explain how trial-by-trial errors drive the progressive adaptation observed in such studies. These models have been applied to adaptation involving novel dynamics, which typically occurs over tens to hundreds of trials, and which appears to be mediated by a dual-rate adaptation process. In contrast, when manipulating objects with familiar dynamics, subjects adapt rapidly within a few trials. Here, we apply state-space models to familiar dynamics, asking whether adaptation is mediated by a single-rate or dual-rate process. Previously, we reported a task in which subjects rotate an object with known dynamics. By presenting the object at different visual orientations, adaptation was shown to be context-specific, with limited generalization to novel orientations. Here we show that a multiple-context state-space model, with a generalization function tuned to visual object orientation, can reproduce the time-course of adaptation and de-adaptation as well as the observed context-dependent behavior. In contrast to the dual-rate process associated with novel dynamics, we show that a single-rate process mediates adaptation to familiar object dynamics. The model predicts that during exposure to the object across multiple orientations, there will be a degree of independence for adaptation and de-adaptation within each context, and that the states associated with all contexts will slowly de-adapt during exposure in one particular context. We confirm these predictions in two new experiments. Results of the current study thus highlight similarities and differences in the processes engaged during exposure to novel versus familiar dynamics. In both cases, adaptation is mediated by multiple context-specific representations. In the case of familiar object dynamics, however, the representations can be engaged based on visual context, and are updated by a single-rate process