Is there an integrative center in the vertebrate brain-stem? A robotic evaluation of a model of the reticular formation viewed as an action selection device

Neurobehavioral data from intact, decerebrate, and neonatal rats, suggests that the reticular formation provides a brainstem substrate for action selection in the vertebrate central nervous system. In this article, Kilmer, McCulloch and Blum’s (1969, 1997) landmark reticular formation model is described and re-evaluated, both in simulation and, for the first time, as a mobile robot controller. Particular model configurations are found to provide effective action selection mechanisms in a robot survival task using either simulated or physical robots. The model’s competence is dependent on the organization of afferents from model sensory systems, and a genetic algorithm search identified a class of afferent configurations which have long survival times. The results support our proposal that the reticular formation evolved to provide effective arbitration between innate behaviors and, with the forebrain basal ganglia, may constitute the integrative, ’centrencephalic’ core of vertebrate brain architecture. Additionally, the results demonstrate that the Kilmer et al. model provides an alternative form of robot controller to those usually considered in the adaptive behavior literature

The role of response mechanisms in determining reaction time performance: Piéron’s Law revisited

A response mechanism takes evaluations of the importance of potential actions and selects the most suitable. Response mechanism function is a nontrivial problem that has not received the attention it deserves within cognitive psychology. In this article, we make a case for the importance of considering response mechanism function as a constraint on cognitive processes and emphasized links with the wider problem of behavioral action selection. First, we show that, contrary to previous suggestions, a well–known model of the Stroop task (Cohen, Dunbar, & McClelland, 1990) relies on the response mechanism for a key feature of its results—the interference–facilitation asymmetry. Second, we examine a variety of response mechanisms (including that in the model of Cohen et al., 1990) and show that they all follow a law analogous to Piéron's law in relating their input to reaction time. In particular, this is true of a decision mechanism not designed to explain RT data but based on a proposed solution to the general problem of action selection and grounded in the neurobiology of the vertebrate basal ganglia. Finally, we show that the dynamics of simple artificial neurons also support a Piéron–like law

Remembering Forward: Neural Correlates of Memory and Prediction in Human Motor Adaptation

We used functional MR imaging (FMRI), a robotic manipulandum and systems identification techniques to examine neural correlates of predictive compensation for spring-like loads during goal-directed wrist movements in neurologically-intact humans. Although load changed unpredictably from one trial to the next, subjects nevertheless used sensorimotor memories from recent movements to predict and compensate upcoming loads. Prediction enabled subjects to adapt performance so that the task was accomplished with minimum effort. Population analyses of functional images revealed a distributed, bilateral network of cortical and subcortical activity supporting predictive load compensation during visual target capture. Cortical regions – including prefrontal, parietal and hippocampal cortices – exhibited trial-by-trial fluctuations in BOLD signal consistent with the storage and recall of sensorimotor memories or “states” important for spatial working memory. Bilateral activations in associative regions of the striatum demonstrated temporal correlation with the magnitude of kinematic performance error (a signal that could drive reward-optimizing reinforcement learning and the prospective scaling of previously learned motor programs). BOLD signal correlations with load prediction were observed in the cerebellar cortex and red nuclei (consistent with the idea that these structures generate adaptive fusimotor signals facilitating cancelation of expected proprioceptive feedback, as required for conditional feedback adjustments to ongoing motor commands and feedback error learning). Analysis of single subject images revealed that predictive activity was at least as likely to be observed in more than one of these neural systems as in just one. We conclude therefore that motor adaptation is mediated by predictive compensations supported by multiple, distributed, cortical and subcortical structures

Value encoding in the globus pallidus: fMRI reveals an interaction effect between reward and dopamine drive

The external part of the globus pallidus (GPe) is a core nucleus of the basal ganglia (BG) whose activity is disrupted under conditions of low dopamine release, as in Parkinson's disease. Current models assume decreased dopamine release in the dorsal striatum results in deactivation of dorsal GPe, which in turn affects motor expression via a regulatory effect on other nuclei of the BG. However, recent studies in healthy and pathological animal models have reported neural dynamics that do not match with this view of the GPe as a relay in the BG circuit. Thus, the computational role of the GPe in the BG is still to be determined. We previously proposed a neural model that revisits the functions of the nuclei of the BG, and this model predicts that GPe encodes values which are amplified under a condition of low striatal dopaminergic drive. To test this prediction, we used an fMRI paradigm involving a within-subject placebo-controlled design, using the dopamine antagonist risperidone, wherein healthy volunteers performed a motor selection and maintenance task under low and high reward conditions. ROI-based fMRI analysis revealed an interaction between reward and dopamine drive manipulations, with increased BOLD activity in GPe in a high compared to low reward condition, and under risperidone compared to placebo. These results confirm the core prediction of our computational model, and provide a new perspective on neural dynamics in the BG and their effects on motor selection and cognitive disorders

Increased bradykinesia in Parkinson’s disease with increased movement complexity: elbow flexion-extension movements

The present research investigates factors contributing to bradykinesia in the control of simple and complex voluntary limb movement in Parkinson’s disease (PD) patients. The functional scheme of the basal ganglia (BG)–thalamocortical circuit was described by a mathematical model based on the mean firing rates of BG nuclei. PD was simulated as a reduction in dopamine levels, and a loss of functional segregation between two competing motor modules. In order to compare model simulations with performed movements, flexion and extension at the elbow joint is taken as a test case. Results indicated that loss of segregation contributed to bradykinesia due to interference between competing modules and a reduced ability to suppress unwanted movements. Additionally, excessive neurotransmitter depletion is predicted as a possible mechanism for the increased difficulty in performing complex movements. The simulation results showed that the model is in qualitative agreement with the results from movement experiments on PD patients and healthy subjects. Furthermore, based on changes in the firing rate of BG nuclei, the model demonstrated that the effective mechanism of Deep Brain Stimulation (DBS) in STN may result from stimulation induced inhibition of STN, partial synaptic failure of efferent projections, or excitation of inhibitory afferent axons even though the underlying methods of action may be quite different for the different mechanisms

Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states