Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions

Genetic inhibition of neurotransmission reveals role of glutamatergic input to dopamine neurons in high-effort behavior

Midbrain dopamine neurons are crucial for many behavioral and cognitive functions. As the major excitatory input, glutamatergic afferents are important for control of the activity and plasticity of dopamine neurons. However, the role of glutamatergic input as a whole onto dopamine neurons remains unclear. Here we developed a mouse line in which glutamatergic inputs onto dopamine neurons are specifically impaired, and utilized this genetic model to directly test the role of glutamatergic inputs in dopamine-related functions. We found that while motor coordination and reward learning were largely unchanged, these animals showed prominent deficits in effort-related behavioral tasks. These results provide genetic evidence that glutamatergic transmission onto dopaminergic neurons underlies incentive motivation, a willingness to exert high levels of effort to obtain reinforcers, and have important implications for understanding the normal function of the midbrain dopamine system.Fil: Hutchison, M. A.. National Institutes of Health; Estados UnidosFil: Gu, X.. National Institutes of Health; Estados UnidosFil: Adrover, Martín Federico. National Institutes of Health; Estados Unidos. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Investigaciones en Ingeniería Genética y Biología Molecular "Dr. Héctor N. Torres"; ArgentinaFil: Lee, M. R.. National Institutes of Health; Estados UnidosFil: Hnasko, T. S.. University of California at San Diego; Estados UnidosFil: Alvarez, V. A.. National Institutes of Health; Estados UnidosFil: Lu, W.. National Institutes of Health; Estados Unido

The Cognitive Role of the Globus Pallidus interna; Insights from Disease States.

The motor symptoms of both Parkinson's disease and focal dystonia arise from dysfunction of the basal ganglia, and are improved by pallidotomy or deep brain stimulation of the Globus Pallidus interna (GPi). However, Parkinson's disease is associated with a greater degree of basal ganglia-dependent learning impairment than dystonia. We attempt to understand this observation in terms of a comparison of the electrophysiology of the output of the basal ganglia between the two conditions. We use the natural experiment offered by Deep Brain Stimulation to compare GPi local field potential responses in subjects with Parkinson's disease compared to subjects with dystonia performing a forced-choice decision-making task with sensory feedback. In dystonic subjects, we found that auditory feedback was associated with the presence of high gamma oscillations nestled on a negative deflection, morphologically similar to sharp wave ripple complexes described in human rhinal cortex. These were not present in Parkinson's disease subjects. The temporal properties of the high gamma burst were modified by incorrect trial performance compared to correct trial performance. Both groups exhibited a robust low frequency response to 'incorrect' trial performance in dominant GPi but not non-dominant GPi at theta frequency. Our results suggest that cellular processes associated with striatum-dependent memory function may be selectively impaired in Parkinson's disease even if dopaminergic drugs are administered, but that error detection mechanisms are preserved

Input-specific control of reward and aversion in the ventral tegmental area

Ventral tegmental area (VTA) dopamine neurons have important roles in adaptive and pathological brain functions related to reward and motivation. However, it is unknown whether subpopulations of VTA dopamine neurons participate in distinct circuits that encode different motivational signatures, and whether inputs to the VTA differentially modulate such circuits. Here we show that, because of differences in synaptic connectivity, activation of inputs to the VTA from the laterodorsal tegmentum and the lateral habenula elicit reward and aversion in mice, respectively. Laterodorsal tegmentum neurons preferentially synapse on dopamine neurons projecting to the nucleus accumbens lateral shell, whereas lateral habenula neurons synapse primarily on dopamine neurons projecting to the medial prefrontal cortex as well as on GABAergic (γ-aminobutyric-acid-containing) neurons in the rostromedial tegmental nucleus. These results establish that distinct VTA circuits generate reward and aversion, and thereby provide a new framework for understanding the circuit basis of adaptive and pathological motivated behaviours.National Institutes of Health (U.S.) (Grant NIH NS069375)JPB FoundationNational Institute of Mental Health (U.S.

Similar Neural Activity during Fear and Disgust in the Rat Basolateral Amygdala

Much research has focused on how the amygdala processes individual affects, yet little is known about how multiple types of positive and negative affects are encoded relative to one another at the single-cell level. In particular, it is unclear whether different negative affects, such as fear and disgust, are encoded more similarly than negative and positive affects, such as fear and pleasure. Here we test the hypothesis that the basolateral nucleus of the amygdala (BLA), a region known to be important for learned fear and other affects, encodes affective valence by comparing neuronal activity in the BLA during a conditioned fear stimulus (fear CS) with activity during intraoral delivery of an aversive fluid that induces a disgust response and a rewarding fluid that induces a hedonic response. Consistent with the hypothesis, neuronal activity during the fear CS and aversive fluid infusion, but not during the fear CS and rewarding fluid infusion, was more similar than expected by chance. We also found that the greater similarity in activity during the fear- and disgust-eliciting stimuli was specific to a subpopulation of cells and a limited window of time. Our results suggest that a subpopulation of BLA neurons encodes affective valence during learned fear, and furthermore, within this subpopulation, different negative affects are encoded more similarly than negative and positive affects in a time-specific manner

Disrupted habenula function in major depression.

The habenula is a small, evolutionarily conserved brain structure that plays a central role in aversive processing and is hypothesised to be hyperactive in depression, contributing to the generation of symptoms such as anhedonia. However, habenula responses during aversive processing have yet to be reported in individuals with major depressive disorder (MDD). Unmedicated and currently depressed MDD patients (N=25, aged 18-52 years) and healthy volunteers (N=25, aged 19-52 years) completed a passive (Pavlovian) conditioning task with appetitive (monetary gain) and aversive (monetary loss and electric shock) outcomes during high-resolution functional magnetic resonance imaging; data were analysed using computational modelling. Arterial spin labelling was used to index resting-state perfusion and high-resolution anatomical images were used to assess habenula volume. In healthy volunteers, habenula activation increased as conditioned stimuli (CSs) became more strongly associated with electric shocks. This pattern was significantly different in MDD subjects, for whom habenula activation decreased significantly with increasing association between CSs and electric shocks. Individual differences in habenula volume were negatively associated with symptoms of anhedonia across both groups. MDD subjects exhibited abnormal negative task-related (phasic) habenula responses during primary aversive conditioning. The direction of this effect is opposite to that predicted by contemporary theoretical accounts of depression based on findings in animal models. We speculate that the negative habenula responses we observed may result in the loss of the capacity to actively avoid negative cues in MDD, which could lead to excessive negative focus

Information content and reward processing in the human striatum during performance of a declarative memory task

Negative feedback can signal poor performance, but it also provides information that can help learners reach the goal of task mastery. The primary aim of this study was to test the hypothesis that the amount of information provided by negative feedback during a paired-associate learning task influences feedback-related processing in the caudate nucleus. To do this, we manipulated the number of response options: With two options, positive and negative feedback provide equal amounts of information, whereas with four options, positive feedback provides more information than does negative feedback. We found that positive and negative feedback activated the caudate similarly when there were two response options. With four options, the caudate’s response to negative feedback was reduced. A secondary goal was to investigate the link between brain-based measures of feedback-related processing and behavioral indices of learning. Analysis of the posttest measures showed that trials with positive feedback were associated with higher posttest confidence ratings. Additionally, when positive feedback was delivered, caudate activity was greater for trials with high than with low posttest confidence. This experiment demonstrated the context sensitivity of feedback processing and provided evidence that feedback processing in the striatum can contribute to the strengthening of the representations available within declarative memory

Search for high-mass resonances decaying to dilepton final states in pp collisions at s√=7 TeV with the ATLAS detector

The ATLAS detector at the Large Hadron Collider is used to search for high-mass resonances decaying to an electron-positron pair or a muon-antimuon pair. The search is sensitive to heavy neutral Z′ gauge bosons, Randall-Sundrum gravitons, Z * bosons, techni-mesons, Kaluza-Klein Z/γ bosons, and bosons predicted by Torsion models. Results are presented based on an analysis of pp collisions at a center-of-mass energy of 7 TeV corresponding to an integrated luminosity of 4.9 fb−1 in the e + e − channel and 5.0 fb−1 in the μ + μ −channel. A Z ′ boson with Standard Model-like couplings is excluded at 95 % confidence level for masses below 2.22 TeV. A Randall-Sundrum graviton with coupling k/MPl=0.1 is excluded at 95 % confidence level for masses below 2.16 TeV. Limits on the other models are also presented, including Technicolor and Minimal Z′ Models