A computational theory of motor learning

In this paper we present a computational theory of human motor performance and learning. The theory is implemented as a running AI system called MAGGIE. Given a description of a desired movement as input, the system generates simulated motor behavior as output. The theory states that skills are encoded as motor schemas, which specify the positions and velocities of a limb at selected points in time. Moreover, there exist two natural representations for such knowledge: viewer-centered schemas describe visually perceived behavior, and joint-centered schemas are used to generate behavior. When the model acts upon these two representational formats, they exhibit quite different behavioral characteristics. MAGGIE performs the desired movement within a feedback control paradigm, monitoring for errors and correcting them when it detects them. Learning involves improving the joint-centered schema over many practice trials; this reduces the need for monitoring. The model accounts for a number of well-documented motor phenomena, including the speed-accuracy trade-off and the gradual improvement in performance with practice. It also makes several testable predictions. We close with a discussion of the theory's strengths and weaknesses, along with directions for future research

A theory of sensorimotor learning for brain-machine interface control

A remarkable demonstration of the flexibility of mammalian motor systems is primates’ ability to learn to control brain-machine interfaces (BMI’s). This constitutes a completely novel and artificial form of motor behavior, yet primates are capable of learning to control BMI’s under a wide range of conditions. BMI’s with carefully calibrated decoders, for example, can be learned with only minutes to hours of practice. With a few weeks of practice, even BMI’s with random decoders can be learned. What are the biological substrates of this learning process? This thesis proposes a simple theory of the computational principles underlying BMI learning. Through comprehensive numerical and formal analysis, we demonstrate that this theory can provide a unifying explanation for various disparate phenomena observed during BMI learning in three different BMI learning tasks. By explicitly modeling the underlying neural circuitry, the theory reveals an interpretation of these phenomena in terms of the biological non-linear dynamics of neural circuits

Optimal Schedules in Multitask Motor Learning

Although scheduling multiple tasks in motor learning to maximize long-term retention of performance is of great practical importance in sports training and motor rehabilitation after brain injury, it is unclear how to do so. We propose here a novel theoretical approach that uses optimal control theory and computational models of motor adaptation to determine schedules that maximize long-term retention predictively. Using Pontryagin’s maximum principle, we derived a control law that determines the trial-by-trial task choice that maximizes overall delayed retention for all tasks, as predicted by the state-space model. Simulations of a single session of adaptation with two tasks show that when task interference is high, there exists a threshold in relative task difficulty below which the alternating schedule is optimal. Only for large differences in task difficulties do optimal schedules assign more trials to the harder task. However, over the parameter range tested, alternating schedules yield long-term retention performance that is only slightly inferior to performance given by the true optimal schedules. Our results thus predict that in a large number of learning situations wherein tasks interfere, intermixing tasks with an equal number of trials is an effective strategy in enhancing long-term retention

Hierarchical control over effortful behavior by rodent medial frontal cortex : a computational model

The anterior cingulate cortex (ACC) has been the focus of intense research interest in recent years. Although separate theories relate ACC function variously to conflict monitoring, reward processing, action selection, decision making, and more, damage to the ACC mostly spares performance on tasks that exercise these functions, indicating that they are not in fact unique to the ACC. Further, most theories do not address the most salient consequence of ACC damage: impoverished action generation in the presence of normal motor ability. In this study we develop a computational model of the rodent medial prefrontal cortex that accounts for the behavioral sequelae of ACC damage, unifies many of the cognitive functions attributed to it, and provides a solution to an outstanding question in cognitive control research-how the control system determines and motivates what tasks to perform. The theory derives from recent developments in the formal study of hierarchical control and learning that highlight computational efficiencies afforded when collections of actions are represented based on their conjoint goals. According to this position, the ACC utilizes reward information to select tasks that are then accomplished through top-down control over action selection by the striatum. Computational simulations capture animal lesion data that implicate the medial prefrontal cortex in regulating physical and cognitive effort. Overall, this theory provides a unifying theoretical framework for understanding the ACC in terms of the pivotal role it plays in the hierarchical organization of effortful behavior

Parallel Alterations of Functional Connectivity during Execution and Imagination after Motor Imagery Learning

BACKGROUND: Neural substrates underlying motor learning have been widely investigated with neuroimaging technologies. Investigations have illustrated the critical regions of motor learning and further revealed parallel alterations of functional activation during imagination and execution after learning. However, little is known about the functional connectivity associated with motor learning, especially motor imagery learning, although benefits from functional connectivity analysis attract more attention to the related explorations. We explored whether motor imagery (MI) and motor execution (ME) shared parallel alterations of functional connectivity after MI learning. METHODOLOGY/PRINCIPAL FINDINGS: Graph theory analysis, which is widely used in functional connectivity exploration, was performed on the functional magnetic resonance imaging (fMRI) data of MI and ME tasks before and after 14 days of consecutive MI learning. The control group had no learning. Two measures, connectivity degree and interregional connectivity, were calculated and further assessed at a statistical level. Two interesting results were obtained: (1) The connectivity degree of the right posterior parietal lobe decreased in both MI and ME tasks after MI learning in the experimental group; (2) The parallel alterations of interregional connectivity related to the right posterior parietal lobe occurred in the supplementary motor area for both tasks. CONCLUSIONS/SIGNIFICANCE: These computational results may provide the following insights: (1) The establishment of motor schema through MI learning may induce the significant decrease of connectivity degree in the posterior parietal lobe; (2) The decreased interregional connectivity between the supplementary motor area and the right posterior parietal lobe in post-test implicates the dissociation between motor learning and task performing. These findings and explanations further revealed the neural substrates underpinning MI learning and supported that the potential value of MI learning in motor function rehabilitation and motor skill learning deserves more attention and further investigation

Recent data on the cerebellum require new models and theories

The cerebellum has been a popular topic for theoretical studies because its structure was thought to be simple. Since David Marr and James Albus related its function to motor skill learning and proposed the Marr-Albus cerebellar learning model, this theory has guided and inspired cerebellar research. In this review, we summarize the theoretical progress that has been made within this framework of error-based supervised learning. We discuss the experimental progress that demonstrates more complicated molecular and cellular mechanisms in the cerebellum as well as new cell types and recurrent connections. We also cover its involvement in diverse non-motor functions and evidence of other forms of learning. Finally, we highlight the need to explain these new experimental findings into an integrated cerebellar model that can unify its diverse computational functions.journal articl

Adaptive Neural Networks for Control of Movement Trajectories Invariant under Speed and Force Rescaling

This article describes two neural network modules that form part of an emerging theory of how adaptive control of goal-directed sensory-motor skills is achieved by humans and other animals. The Vector-Integration-To-Endpoint (VITE) model suggests how synchronous multi-joint trajectories are generated and performed at variable speeds. The Factorization-of-LEngth-and-TEnsion (FLETE) model suggests how outflow movement commands from a VITE model may be performed at variable force levels without a loss of positional accuracy. The invariance of positional control under speed and force rescaling sheds new light upon a familiar strategy of motor skill development: Skill learning begins with performance at low speed and low limb compliance and proceeds to higher speeds and compliances. The VITE model helps to explain many neural and behavioral data about trajectory formation, including data about neural coding within the posterior parietal cortex, motor cortex, and globus pallidus, and behavioral properties such as Woodworth's Law, Fitts Law, peak acceleration as a function of movement amplitude and duration, isotonic arm movement properties before and after arm-deafferentation, central error correction properties of isometric contractions, motor priming without overt action, velocity amplification during target switching, velocity profile invariance across different movement distances, changes in velocity profile asymmetry across different movement durations, staggered onset times for controlling linear trajectories with synchronous offset times, changes in the ratio of maximum to average velocity during discrete versus serial movements, and shared properties of arm and speech articulator movements. The FLETE model provides new insights into how spina-muscular circuits process variable forces without a loss of positional control. These results explicate the size principle of motor neuron recruitment, descending co-contractive compliance signals, Renshaw cells, Ia interneurons, fast automatic reactive control by ascending feedback from muscle spindles, slow adaptive predictive control via cerebellar learning using muscle spindle error signals to train adaptive movement gains, fractured somatotopy in the opponent organization of cerebellar learning, adaptive compensation for variable moment-arms, and force feedback from Golgi tendon organs. More generally, the models provide a computational rationale for the use of nonspecific control signals in volitional control, or "acts of will", and of efference copies and opponent processing in both reactive and adaptive motor control tasks.National Science Foundation (IRI-87-16960); Air Force Office of Scientific Research (90-0128, 90-0175

fMTP:A Unifying Computational Framework of Temporal Preparation Across Time Scales

In warned reaction time (RT) tasks, a warning stimulus (S1) initiates a process of temporal preparation which promotes a speeded response to the target stimulus (S2). Variations of the S1-S2 interval (foreperiod) have been shown to affect the RT to S2 across a range of time scales: within trials, between consecutive trials, across trials within an experimental block, and across blocks. Theories on temporal preparation thus far have failed to offer a complete account for these effects across all scales. We present a computational framework (fMTP) that formalizes the principles of a previously proposed theory of temporal preparation: Multiple Trace Theory of Temporal Preparation. With fMTP we combine models and theories on time perception, motor planning, and associative learning, and show that by integrating them into a single, computational theory they allow us to capture the range of preparatory phenomena across different scales. fMTP assumes that for each timing experience (trial) a unique trace is formed by means of associative Hebbian learning between a layer of time cells and a motor layer with an inhibition and activation node. On each new trial, traces from past trials are automatically retrieved and collectively determine the temporal preparatory state throughout the trial. Each trace contributes to preparation proportional to its strength, with strength gradually dissipating with time. For experimental setups where the predictions of existing accounts and fMTP differed, we show that the data aligns with our model predictions. In sum, we find that fMTP’s single implicit learning mechanism suffices to explain a range of phenomena that previously have been considered to be the result of distinct processes

Neural Coordination of Distinct Motor Learning Strategies: Latent Neurofunctional Mechanisms Elucidated via Computational Modeling

In this dissertation, a neurofunctional theory of learning is presented as an extension of functional analysis. This new theory clarifies the distinction— via applied quantitative analysis— between functionally intrinsic (essential) mechanistic structures and irrelevant structural details. This thesis is supported by a review of the relevant literature to provide historical context and sufficient scientific background. Further, the scope of this thesis is elucidated by two questions that are posed from a neurofunctional perspective— (1) how can specialized neuromorphology contribute to the functional dynamics of neural learning processes? (2) Can large-scale neurofunctional pathways emerge via inter-network communication between disparate neural circuits? These questions motivate the specific aims of this dissertation. Each aim is addressed by posing a relevant hypothesis, which is then tested via a neurocomputational experiment. In each experiment, computational techniques are leveraged to elucidate specific mechanisms that underlie neurofunctional learning processes. For instance, the role of specialized neuromorphology is investigated via the development of a computational model that replicates the neurophysiological mechanisms that underlie cholinergic interneurons’ regulation of dopamine in the striatum during reinforcement learning. Another research direction focuses on the emergence of large-scale neurofunctional pathways that connect the cerebellum and basal ganglia— this study also involves the construction of a neurocomputational model. The results of each study illustrate the capability of neurocomputational models to replicate functional learning dynamics of human subjects during a variety of motor adaptation tasks. Finally, the significance— and some potential applications— of neurofunctional theory are discussed

Passive Motion Paradigm: An Alternative to Optimal Control

In the last years, optimal control theory (OCT) has emerged as the leading approach for investigating neural control of movement and motor cognition for two complementary research lines: behavioral neuroscience and humanoid robotics. In both cases, there are general problems that need to be addressed, such as the “degrees of freedom (DoFs) problem,” the common core of production, observation, reasoning, and learning of “actions.” OCT, directly derived from engineering design techniques of control systems quantifies task goals as “cost functions” and uses the sophisticated formal tools of optimal control to obtain desired behavior (and predictions). We propose an alternative “softer” approach passive motion paradigm (PMP) that we believe is closer to the biomechanics and cybernetics of action. The basic idea is that actions (overt as well as covert) are the consequences of an internal simulation process that “animates” the body schema with the attractor dynamics of force fields induced by the goal and task-specific constraints. This internal simulation offers the brain a way to dynamically link motor redundancy with task-oriented constraints “at runtime,” hence solving the “DoFs problem” without explicit kinematic inversion and cost function computation. We argue that the function of such computational machinery is not only restricted to shaping motor output during action execution but also to provide the self with information on the feasibility, consequence, understanding and meaning of “potential actions.” In this sense, taking into account recent developments in neuroscience (motor imagery, simulation theory of covert actions, mirror neuron system) and in embodied robotics, PMP offers a novel framework for understanding motor cognition that goes beyond the engineering control paradigm provided by OCT. Therefore, the paper is at the same time a review of the PMP rationale, as a computational theory, and a perspective presentation of how to develop it for designing better cognitive architectures