Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry.

Midbrain dopamine (DA) neurons are proposed to signal reward prediction error (RPE), a fundamental parameter in associative learning models. This RPE hypothesis provides a compelling theoretical framework for understanding DA function in reward learning and addiction. New studies support a causal role for DA-mediated RPE activity in promoting learning about natural reward; however, this question has not been explicitly tested in the context of drug addiction. In this review, we integrate theoretical models with experimental findings on the activity of DA systems, and on the causal role of specific neuronal projections and cell types, to provide a circuit-based framework for probing DA-RPE function in addiction. By examining error-encoding DA neurons in the neural network in which they are embedded, hypotheses regarding circuit-level adaptations that possibly contribute to pathological error signaling and addiction can be formulated and tested

A Transgenic Rat for Investigating the Anatomy and Function of Corticotrophin Releasing Factor Circuits.

Corticotrophin-releasing factor (CRF) is a 41 amino acid neuropeptide that coordinates adaptive responses to stress. CRF projections from neurons in the central nucleus of the amygdala (CeA) to the brainstem are of particular interest for their role in motivated behavior. To directly examine the anatomy and function of CRF neurons, we generated a BAC transgenic Crh-Cre rat in which bacterial Cre recombinase is expressed from the Crh promoter. Using Cre-dependent reporters, we found that Cre expressing neurons in these rats are immunoreactive for CRF and are clustered in the lateral CeA (CeL) and the oval nucleus of the BNST. We detected major projections from CeA CRF neurons to parabrachial nuclei and the locus coeruleus, dorsal and ventral BNST, and more minor projections to lateral portions of the substantia nigra, ventral tegmental area, and lateral hypothalamus. Optogenetic stimulation of CeA CRF neurons evoked GABA-ergic responses in 11% of non-CRF neurons in the medial CeA (CeM) and 44% of non-CRF neurons in the CeL. Chemogenetic stimulation of CeA CRF neurons induced Fos in a similar proportion of non-CRF CeM neurons but a smaller proportion of non-CRF CeL neurons. The CRF1 receptor antagonist R121919 reduced this Fos induction by two-thirds in these regions. These results indicate that CeL CRF neurons provide both local inhibitory GABA and excitatory CRF signals to other CeA neurons, and demonstrate the value of the Crh-Cre rat as a tool for studying circuit function and physiology of CRF neurons

Heroin Addiction: Anticipating the Reward of Heroin or the Agony of Withdrawal?

International audienc

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement

Reward-seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is non-scalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies

Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions

Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) encode reward prediction errors (RPEs) and are proposed to mediate error-driven learning. However, the learning strategy engaged by DA-RPEs remains controversial. RPEs might imbue predictive cues with pure value, independently of representations of their associated outcome. Alternatively, RPEs might promote learning about the sensory features (the identity) of the rewarding outcome. Here, we show that, although both VTA and SNc DA neuron activation reinforces instrumental responding, only VTA DA neuron activation during consumption of expected sucrose reward restores error-driven learning and promotes formation of a new cue→sucrose association. Critically, expression of VTA DA-dependent Pavlovian associations is abolished following sucrose devaluation, a signature of identity-based learning. These findings reveal that activation of VTA- or SNc-DA neurons engages largely dissociable learning processes with VTA-DA neurons capable of participating in outcome-specific predictive learning, and the role of SNc-DA neurons appears limited to reinforcement of instrumental responses

Partially dissociable roles of the Orbitofrontal cortex and dorsal Hippocampus in context-dependent (hierarchical) reward prediction and contextual inference in learning

Reward cues are often ambiguous; what is good in one context is not necessarily good in another context. To solve this ambiguity, animals form hierarchical associations in which the context acts as a gatekeeper in the retrieval of the appropriate cue-evoked memory, ensuring context-appropriate behavior. These hierarchical associative structures also influence future learning by promoting the formation of new context-dependent associations (leading to the inference of context-dependency for new associations). The orbitofrontal cortex (OFC) and the dorsal hippocampus (DH) are both proposed to encode a “cognitive map” that includes the representation of hierarchical, context-dependent, associations. However the causal role of the OFC and DH in the different functional properties of hierarchical associations remains controversial. Here we used chemogenetic inactivations, in rats, to examine the role of OFC and DH in 1) the contextual regulation of performance, and 2) the contextual learning bias conferred by hierarchical associations. We show that OFC is required for both manifestations of hierarchical associations. In contrast, DH contribution appears limited to the contextual learning bias. This study provides novel insight into the different functional properties of context-dependent hierarchical associations, and establishes the OFC as a critical orchestrator of these different contextual effects

A Transgenic Rat for Investigating the Anatomy and Function of Corticotrophin Releasing Factor Circuits.

Corticotrophin-releasing factor (CRF) is a 41 amino acid neuropeptide that coordinates adaptive responses to stress. CRF projections from neurons in the central nucleus of the amygdala (CeA) to the brainstem are of particular interest for their role in motivated behavior. To directly examine the anatomy and function of CRF neurons, we generated a BAC transgenic Crh-Cre rat in which bacterial Cre recombinase is expressed from the Crh promoter. Using Cre-dependent reporters, we found that Cre expressing neurons in these rats are immunoreactive for CRF and are clustered in the lateral CeA (CeL) and the oval nucleus of the BNST. We detected major projections from CeA CRF neurons to parabrachial nuclei and the locus coeruleus, dorsal and ventral BNST, and more minor projections to lateral portions of the substantia nigra, ventral tegmental area, and lateral hypothalamus. Optogenetic stimulation of CeA CRF neurons evoked GABA-ergic responses in 11% of non-CRF neurons in the medial CeA (CeM) and 44% of non-CRF neurons in the CeL. Chemogenetic stimulation of CeA CRF neurons induced Fos in a similar proportion of non-CRF CeM neurons but a smaller proportion of non-CRF CeL neurons. The CRF1 receptor antagonist R121919 reduced this Fos induction by two-thirds in these regions. These results indicate that CeL CRF neurons provide both local inhibitory GABA and excitatory CRF signals to other CeA neurons, and demonstrate the value of the Crh-Cre rat as a tool for studying circuit function and physiology of CRF neurons

A causal link between prediction errors, dopamine neurons and learning

Situations where rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally-precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks