Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction

Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action–outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action–outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ΔP, as did response rates and explicit judgments of the causal efficacy of the action

Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions

In instrumental conditioning, newly acquired actions are generally goal-directed and are mediated by the relationship between the action and its consequences or outcome. With continued training, however, the performance of such actions can become automatic, reflexive or habitual and under the control of antecedent stimuli rather than their consequences. Recent evidence suggests that habit learning is mediated by plasticity in the dorsolateral striatum (DLS). To date, however, no direct evidence of learning-related plasticity associated with overtraining has been reported in this region, nor is it known whether, or which, specific cell types are involved in this learning process. The striatum is primarily composed of two classes of spiny projection neurons, the striatonigral and striatopallidal spiny projection neurons, which express dopamine D1 and D2 receptors, and control direct and indirect pathways, respectively. Here we found evidence of a post-synaptic depression in DLS striatopallidal projecting neurons in the indirect pathway during habit learning in mice. Moreover, this training-induced depression occluded post-synaptic depression induced by co-activation of D2 receptors and transient receptor potential vanilloid 1 (TRPV1) channels, implying that this pathway is involved in habit learning. This hypothesis was further tested by disrupting this signal pathway by knocking out TRPV1 channels, resulting in compromised habit learning. Our findings suggest that post-synaptic plasticity at D2 neurons in the DLS mediates habit learning and, by implicating an interaction between the D2 receptor and TRPV1 channel activity, provide a potential drug target for influencing habitual action control.NHMRC grants: 1045964 & 108925

A specific role for posterior dorsolateral striatum in human habit learning

Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen–globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans

Alcohol-Paired Contextual Cues Produce an Immediate and Selective Loss of Goal-directed Action in Rats

We assessed whether the presence of contextual cues paired with alcohol would disrupt rats’ capacity to express appropriate goal-directed action control. Rats were first given differential context conditioning such that one set of contextual cues was paired with the injection of ethanol and a second, distinctive set of cues was paired with the injection of saline. All rats were then trained in a third, neutral context to press one lever for grain pellets and another lever for sucrose pellets. They were then given two extinction tests to evaluate their ability to choose between the two actions in response to the devaluation of one of the two food outcomes with one test conducted in the alcohol-paired context and the other conducted in the control (saline-paired) context. In the control context, rats exhibited goal-directed action control; i.e., they were able selectively to withhold the action that previously earned the now devalued outcome. However, these same rats were impaired when tested in the alcohol-paired context, performing both actions at the same rate regardless of the current value of their respective outcomes. Subsequent testing revealed that the rats were capable of overcoming this impairment if they were giving response-contingent feedback about the current value of the food outcomes. These results provide a clear demonstration of the disruptive influence that alcohol-paired cues can exert on decision-making in general and goal-directed action selection and choice in particular

Chronic morphine reduces surface expression of δ-opioid receptors in subregions of rostral striatum

The delta opioid receptor (DOPr), whilst not the primary target of clinically used opioids, is involved in development of opioid tolerance and addiction. There is growing evidence that DOPr trafficking is involved in drug addiction, e.g., a range of studies have shown increased plasma membrane DOPr insertion during chronic treatment with opioids. The present study used a transgenic mouse model in which the C-terminal of the DOPr is tagged with enhanced-green fluorescence protein to examine the effects of chronic morphine treatment on surface membrane expression in striatal cholinergic interneurons that are implicated in motivated learning following both chronic morphine and morphine sensitization treatment schedules in male mice. A sex difference was noted throughout the anterior striatum, which was most prominent in the nucleus accumbens core region. Incontrast with previous studies in other neurons, chronic exposure to a high dose of morphine for 6 days had no effect, or slightly decreased (anterior dorsolateral striatum) surface DOPr expression. A morphine sensitization schedule produced similar results with a significant decrease in surface DOPr expression in nucleus accumbens shell. These results suggest that chronic morphine and morphine sensitisation treatment may have effects on instrumental reward-seeking behaviours and learning processes related to drug addiction, via effects on striatal DOPr function.NHMRC Grants: 1045964 & 108925

Chronic morphine reduces surface expression of δ-opioid receptors in subregions of rostral striatum

The delta opioid receptor (DOPr), whilst not the primary target of clinically used opioids, is involved in development of opioid tolerance and addiction. There is growing evidence that DOPr trafficking is involved in drug addiction, e.g., a range of studies have shown increased plasma membrane DOPr insertion during chronic treatment with opioids. The present study used a transgenic mouse model in which the C-terminal of the DOPr is tagged with enhanced-green fluorescence protein to examine the effects of chronic morphine treatment on surface membrane expression in striatal cholinergic interneurons that are implicated in motivated learning following both chronic morphine and morphine sensitization treatment schedules in male mice. A sex difference was noted throughout the anterior striatum, which was most prominent in the nucleus accumbens core region. Incontrast with previous studies in other neurons, chronic exposure to a high dose of morphine for 6 days had no effect, or slightly decreased (anterior dorsolateral striatum) surface DOPr expression. A morphine sensitization schedule produced similar results with a significant decrease in surface DOPr expression in nucleus accumbens shell. These results suggest that chronic morphine and morphine sensitisation treatment may have effects on instrumental reward-seeking behaviours and learning processes related to drug addiction, via effects on striatal DOPr function.NHMRC Grants: 1045964 & 108925

Learning-related translocation of δ-opioid receptors on ventral striatal cholinergic interneurons mediates choice between goal-directed actions.

The ability of animals to extract predictive information from the environment to inform their future actions is a critical component of decision-making. This phenomenon is studied in the laboratory using the pavlovian-instrumental transfer protocol in which a stimulus predicting a specific pavlovian outcome biases choice toward those actions earning the predicted outcome. It is well established that this transfer effect is mediated by corticolimbic afferents on the nucleus accumbens shell (NAc-S), and recent evidence suggests that delta-opioid receptors (DORs) play an essential role in this effect. In DOR-eGFP knock-in mice, we show a persistent, learning-related plasticity in the translocation of DORs to the somatic plasma membrane of cholinergic interneurons (CINs) in the NAc-S during the encoding of the specific stimulus-outcome associations essential for pavlovian-instrumental transfer. Wefound that increased membrane DOR expression reflected both stimulus-based predictions of reward and the degree to which these stimuli biased choice during the pavlovian-instrumental transfer test. Furthermore, this plasticity altered the firing pattern of CINs increasing the variance of action potential activity, an effect that was exaggerated by DOR stimulation. The relationship between the induction of membrane DOR expression in CINs and both pavlovian conditioning and pavlovian-instrumental transfer provides a highly specific function for DOR-related modulation in the NAc-S, and it is consistent with an emerging role for striatal CIN activity in the processing of predictive information. Therefore, our results reveal evidence of a long-term, experience-dependent plasticity in opioid receptor expression on striatal modulatory interneurons critical for the cognitive control of action

Can understanding reward help illuminate anhedonia?

Purpose of review: The goal of this paper is to examine how reward processing might help us understand the symptom of anhedonia. Recent findings: There are extensive reviews exploring the relationship between responses to rewarding stimuli and depression. These often include a discussion on anhedonia and how this might be underpinned in particular by dysfunctional reward processing. However, there is no specific consensus on whether studies to date have adequately examined the various sub-components of reward processing or how these might relate in turn to various aspects of anhedonia symptoms. Summary: The approach to understanding the symptom of anhedonia should be to examine all the sub-components of reward processing at the subjective and objective behavioural and neural level, with well validated tasks that can be replicated. Investigating real life experiences of anhedonia and how theses might be predicted by objective lab measures is also needed in future research

A Kinetic Model of Dopamine- and Calcium-Dependent Striatal Synaptic Plasticity

Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD), the combination with dopamine switches LTD to long-term potentiation (LTP), which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32), as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA), protein phosphatase 2A (PP2A), and the phosphorylation site at threonine 75 of DARPP-32 (Thr75) served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B)-CK1 (casein kinase 1)-Cdk5 (cyclin-dependent kinase 5)-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP). The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The present model elucidated the mechanisms involved in bidirectional regulation of corticostriatal synapses and will allow for further exploration into causes and therapies for dysfunctions such as drug addiction