Emulation as an Integrating Principle for Cognition

Emulations, defined as ongoing internal representations of potential actions and the futures those actions are expected to produce, play a critical role in directing human bodily activities. Studies of gross motor behavior, perception, allocation of attention, response to errors, interoception, and homeostatic activities, and higher cognitive reasoning suggest that the proper execution of all these functions relies on emulations. Further evidence supports the notion that reinforcement learning in humans is aimed at updating emulations, and that action selection occurs via the advancement of preferred emulations toward realization of their action and environmental prediction. Emulations are hypothesized to exist as distributed active networks of neurons in cortical and sub-cortical structures. This manuscript ties together previously unrelated theories of the role of prediction in different aspects of human information processing to create an integrated framework for cognition

Basal ganglia, drug addiction and the neuroscience of maladaptive habits

The mammalian brain has developed memory systems mediating rigid, yet evolutionarily adaptive patterns of responding to invariant environmental stimuli and internal demands. Such memory systems promote the recall of specific response templates and the execution of inflexible actions to liberate buffering capacity for performing conscious, explicit cognitive processing. The dopamine-innervated neostriatum is central to the ability to learn such consistent associations between stimuli and actions implicitly. Controlled by their outcome when initially learned, actions succumb through iteration to the influence of triggering stimuli and progressively detach themselves from the pleasurable results originally produced, thereby becoming pervasive habits. This might be the case for drug-seeking and drug-taking behaviours, actions learned in part through dopamine-dependent drug-induced reinforcement when the drug is first experienced. With extended drug use, however, drugseeking actions might become conditioned to, and triggered by, specific exteroceptive stimuli and/or affective states, gradually becoming irrepressible forms of responding. We will review neuroanatomical, neuropharmacological and behavioural evidence suggesting that the basal ganglia play a prominent role in the shaping of drug addiction, here regarded as a pathological modification of otherwise adaptive habit learning systems mediated by the basal ganglia.peer-reviewe

The gradient of the reinforcement landscape influences sensorimotor learning

© 2019 Cashaback et al. Consideration of previous successes and failures is essential to mastering a motor skill. Much of what we know about how humans and animals learn from such reinforcement feedback comes from experiments that involve sampling from a small number of discrete actions. Yet, it is less understood how we learn through reinforcement feedback when sampling from a continuous set of possible actions. Navigating a continuous set of possible actions likely requires using gradient information to maximize success. Here we addressed how humans adapt the aim of their hand when experiencing reinforcement feedback that was associated with a continuous set of possible actions. Specifically, we manipulated the change in the probability of reward given a change in motor action-the reinforcement gradient-to study its influence on learning. We found that participants learned faster when exposed to a steep gradient compared to a shallow gradient. Further, when initially positioned between a steep and a shallow gradient that rose in opposite directions, participants were more likely to ascend the steep gradient. We introduce a model that captures our results and several features of motor learning. Taken together, our work suggests that the sensorimotor system relies on temporally recent and spatially local gradient information to drive learning

Maximizing the Effects of Passive Training on Visuomotor Adaptation By Incorporating Other Motor Learning Strategies

Passive training has been shown to be an effective rehabilitation approach for stroke survivors, especially for those who suffer from severe control loss or complete paralysis. However, the effectiveness of the treatments that utilize passive assist training is still low. The goal of this dissertation was to develop a training condition that can maximize the effects of passive training on motor learning by combining its effect with other motor learning strategies. To achieve this goal, two specific aims were pursued: one aim was to determine the effects of passive training on learning a visuomotor adaptation task; and the other aim was to determine the effects of passive training in combination with other strategies on learning a visuomotor adaptation task. Experimental results indicated that passive training has a positive effect on visuomotor learning. Furthermore, it was confirmed that a training condition consisting of action observation and passive training leads to significant performance gains beyond what either intervention alone can do. This suggests that passive training could elicit motor representational changes, inducing instance-reliant learning process (use-dependent plasticity) that encodes motor instances associated with specific effectors and task conditions. The findings from this study show great potential for developing specific rehabilitation protocols that utilize passive training and action observation together for severely impaired stroke patients in the future

Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making

Considerable evidence has emerged to implicate ventromedial prefrontal cortex in encoding expectations of future reward during value-based decision making. However, the nature of the learned associations upon which such representations depend is much less clear. Here, we aimed to determine whether expected reward representations in this region could be driven by action–outcome associations, rather than being dependent on the associative value assigned to particular discriminative stimuli. Subjects were scanned with functional magnetic resonance imaging while performing 2 variants of a simple reward-related decision task. In one version, subjects made choices between 2 different physical motor responses in the absence of discriminative stimuli, whereas in the other version, subjects chose between 2 different stimuli that were randomly assigned to different responses on a trial-by-trial basis. Using an extension of a reinforcement learning algorithm, we found activity in ventromedial prefrontal cortex tracked expected future reward during the action-based task as well as during the stimulus-based task, indicating that value representations in this region can be driven by action–outcome associations. These findings suggest that ventromedial prefrontal cortex may play a role in encoding the value of chosen actions irrespective of whether those actions denote physical motor responses or more abstract decision options

Principles of sensorimotor learning.

The exploits of Martina Navratilova and Roger Federer represent the pinnacle of motor learning. However, when considering the range and complexity of the processes that are involved in motor learning, even the mere mortals among us exhibit abilities that are impressive. We exercise these abilities when taking up new activities - whether it is snowboarding or ballroom dancing - but also engage in substantial motor learning on a daily basis as we adapt to changes in our environment, manipulate new objects and refine existing skills. Here we review recent research in human motor learning with an emphasis on the computational mechanisms that are involved

Multiple Motor Learning Processes in Humans: Defining Their Neurophysiological Bases

Learning new motor behaviors or adjusting previously learned actions to account for dynamic changes in our environment requires the operation of multiple distinct motor learning processes, which rely on different neuronal substrates. For instance, humans are capable of acquiring new motor patterns via the formation of internal model representations of the movement dynamics and through positive reinforcement. In this review, we will discuss how changes in human physiological markers, assessed with noninvasive brain stimulation techniques from distinct brain regions, can be utilized to provide insights toward the distinct learning processes underlying motor learning. We will summarize the findings from several behavioral and neurophysiological studies that have made efforts to understand how distinct processes contribute to and interact when learning new motor behaviors. In particular, we will extensively review two types of behavioral processes described in human sensorimotor learning: (1) a recalibration process of a previously learned movement and (2) acquiring an entirely new motor control policy, such as learning to play an instrument. The selected studies will demonstrate in-detail how distinct physiological mechanisms contributions change depending on the time course of learning and the type of behaviors being learned

Does the Sensorimotor System Minimize Prediction Error or Select the Most Likely Prediction During Object Lifting?

This is the author accepted manuscript. The final version is available from American Physiological Society via the DOI in this record.The human sensorimotor system is routinely capable of making accurate predictions about an object's weight, which allows for energetically efficient lifts and prevents objects from being dropped. Often however, poor predictions arise when the weight of an object can vary and sensory cues about object weight are sparse (e.g., picking up an opaque water bottle). The question arises, what strategies does the sensorimotor system use to make weight predictions when dealing with an object whose weight may vary? For example, does the sensorimotor system use a strategy that minimizes prediction error (minimal squared error) or one that selects the weight that is most likely to be correct (maximum a posteriori)? Here we dissociated the predictions of these two strategies by having participants lift an object whose weight varied according to a skewed probability distribution. We found, using a small range of weight uncertainty, that four indexes of sensorimotor prediction (grip force rate, grip force, load force rate, and load force) were consistent with a feedforward strategy that minimizes the square of prediction errors. These findings match research in the visuomotor system, suggesting parallels in underlying processes. We interpret our findings within a Bayesian framework and discuss the potential benefits of using a minimal squared error strategy.This work was supported by Canadian Institutes of Health Research and the Natural Sciences and Engineering Council of Canada

An interoceptive predictive coding model of conscious presence

We describe a theoretical model of the neurocognitive mechanisms underlying conscious presence and its disturbances. The model is based on interoceptive prediction error and is informed by predictive models of agency, general models of hierarchical predictive coding and dopaminergic signaling in cortex, the role of the anterior insular cortex (AIC) in interoception and emotion, and cognitive neuroscience evidence from studies of virtual reality and of psychiatric disorders of presence, specifically depersonalization/derealization disorder. The model associates presence with successful suppression by top-down predictions of informative interoceptive signals evoked by autonomic control signals and, indirectly, by visceral responses to afferent sensory signals. The model connects presence to agency by allowing that predicted interoceptive signals will depend on whether afferent sensory signals are determined, by a parallel predictive-coding mechanism, to be self-generated or externally caused. Anatomically, we identify the AIC as the likely locus of key neural comparator mechanisms. Our model integrates a broad range of previously disparate evidence, makes predictions for conjoint manipulations of agency and presence, offers a new view of emotion as interoceptive inference, and represents a step toward a mechanistic account of a fundamental phenomenological property of consciousness