Is the short-latency dopamine response too short to signal reward error?

Unexpected stimuli that are behaviourally significant have the capacity to elicit a short-latency, short-duration burst of firing in mesencephalic dopaminergic neurones. An influential interpretation of the experimental data that characterize this response proposes that dopaminergic neurones have a crucial role in reinforcement learning because they signal error in the prediction of future reward. In this article we propose a different functional role for this ‘short-latency dopamine response’ in the mechanisms that underlie associative learning. We suggest that the initial burst of dopaminergic-neurone firing could represent an essential component in the process of switching attentional and behavioural selections to unexpected, behaviourally important stimuli. This switching response could be a crucial prerequisite for associative learning and might be part of a general short-latency response that is mediated by catecholamines and prepares the organism for an appropriate reaction to biologically significant events. Any act which in a given situation produces satisfaction becomes associated with that situation so that when the situation recurs the act is more likely than before to recur also. E.L. Thorndike (1911)

Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

Emergence of visually-evoked reward expectation signals in dopamine neurons via the superior colliculus in V1 lesioned monkeys

Responses of midbrain dopamine (DA) neurons reflecting expected reward from sensory cues are critical for reward-based associative learning. However, critical pathways by which reward-related visual information is relayed to DA neurons remain unclear. To address this question, we investigated Pavlovian conditioning in macaque monkeys with unilateral primary visual cortex (V1) lesions (an animal model of 'blindsight'). Anticipatory licking responses to obtain juice drops were elicited in response to visual conditioned stimuli (CS) in the affected visual field. Subsequent pharmacological inactivation of the superior colliculus (SC) suppressed the anticipatory licking. Concurrent single unit recordings indicated that DA responses reflecting the reward expectation could be recorded in the absence of V1, and that these responses were also suppressed by SC inactivation. These results indicate that the subcortical visual circuit can relay reward-predicting visual information to DA neurons and integrity of the SC is necessary for visually-elicited classically conditioned responses after V1 lesion

The basal ganglia: A vertebrate solution to the selection problem?

A selection problem arises whenever two or more competing systems seek simultaneous access to a restricted resource. Consideration of several selection architectures suggests there are significant advantages for systems which incorporate a central switching mechanism. We propose that the vertebrate basal ganglia have evolved as a centralized selection device, specialized to resolve conflicts over access to limited motor and cognitive resources. Analysis of basal ganglia functional architecture and its position within a wider anatomical framework suggests it can satisfy many of the requirements expected of an efficient selection mechanism

Does the motor system need intermittent control?

Explanation of motor control is dominated by continuous neurophysiological pathways (e.g. trans-cortical, spinal) and the continuous control paradigm. Using new theoretical development, methodology and evidence, we propose intermittent control, which incorporates a serial ballistic process within the main feedback loop, provides a more general and more accurate paradigm necessary to explain attributes highly advantageous for competitive survival and performance

A robot model of the basal ganglia: Behavior and intrinsic processing

The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing

Action Experience and Action Discovery in Medicated Individuals with Parkinson's Disease.

Parkinson's disease (PD) is a neurodegenerative disorder that markedly affects voluntary action. While regular dopamine treatment can help restore motor function, dopamine also influences cognitive portions of the action system. Previous studies have demonstrated that dopamine medication boosts action-effect associations, which are crucial for the discovery of new voluntary actions. In the present study, we investigated whether neural processes involved in the discovery of new actions are altered in PD participants on regular dopamine treatment, compared to healthy age-matched controls. We recorded brain electroencephalography (EEG) activity while PD patients and age-matched controls performed action discovery (AD) and action control tasks. We found that the novelty P3, a component normally present when there is uncertainty about the occurrence of the sensory effect, was enhanced in PD patients. However, AD was maintained in PD patients, and the novelty P3 demonstrated normal learning-related reductions. Crucially, we found that in PD patients the causal association between an action and its resulting sensory outcome did not modulate the amplitude of the feedback correct-related positivity (fCRP), an EEG component sensitive to the association between an action and its resulting effect. Collectively, these preliminary results suggest that the formation of long-term action-outcome representations may be maintained in PD patients on regular dopamine treatment, but the initial experience of action-effect association may be affected

Aerobic exercise interventions reduce blood pressure in patients after stroke or transient ischaemic attack: a systematic review and meta-analysis

Objective: Secondary vascular risk reduction is critical to preventing recurrent stroke. We aimed to evaluate the effect of exercise interventions on vascular risk factors and recurrent ischaemic events after stroke or transient ischaemic attack (TIA). Design: Intervention systematic review and meta-analysis. Data sources: OVID MEDLINE, PubMed, The Cochrane Library, Web of Science, The National Institute for Health and Care Excellence, TRIP Database, CINAHL, PsycINFO, SCOPUS, UK Clinical Trials Gateway and the China National Knowledge Infrastructure were searched from 1966 to October 2017. Eligibility criteria: Randomised controlled trials evaluating aerobic or resistance exercise interventions on vascular risk factors and recurrent ischaemic events among patients with stroke or TIA, compared with control. Results: Twenty studies (n=1031) were included. Exercise interventions resulted in significant reductions in systolic blood pressure (SBP) −4.30 mm Hg (95% CI −6.77 to −1.83) and diastolic blood pressure −2.58 mm Hg (95% CI −4.7 to −0.46) compared with control. Reduction in SBP was most pronounced among studies initiating exercise within 6 months of stroke or TIA (−8.46 mm Hg, 95% CI −12.18 to −4.75 vs −2.33 mm Hg, 95% CI −3.94 to −0.72), and in those incorporating an educational component (−7.81 mm Hg, 95% CI −14.34 to −1.28 vs −2.78 mm Hg, 95% CI −4.33 to −1.23). Exercise was also associated with reductions in total cholesterol (−0.27 mmol/L, 95% CI −0.54 to 0.00), but not fasting glucose or body mass index. One trial reported reductions in secondary vascular events with exercise, but was insufficiently powered. Summary: Exercise interventions can result in clinically meaningful blood pressure reductions, particularly if initiated early and alongside education