Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction

Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action–outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action–outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ΔP, as did response rates and explicit judgments of the causal efficacy of the action

The Neural Mechanisms Underlying the Influence of Pavlovian Cues on Human Decision Making

In outcome-specific transfer, pavlovian cues that are predictive of specific outcomes bias action choice toward actions associated with those outcomes. This transfer occurs despite no explicit training of the instrumental actions in the presence of pavlovian cues. The neural substrates of this effect in humans are unknown. To address this, we scanned 23 human subjects with functional magnetic resonance imaging while they made choices between different liquid food rewards in the presence of pavlovian cues previously associated with one of these outcomes. We found behavioral evidence of outcome-specific transfer effects in our subjects, as well as differential blood oxygenation level-dependent activity in a region of ventrolateral putamen when subjects chose, respectively, actions consistent and inconsistent with the pavlovian-predicted outcome. Our results suggest that choosing an action incompatible with a pavlovian-predicted outcome might require the inhibition of feasible but nonselected action– outcome associations. The results of this study are relevant for understanding how marketing actions can affect consumer choice behavior as well as for how environmental cues can influence drug-seeking behavior in addiction

Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions

In instrumental conditioning, newly acquired actions are generally goal-directed and are mediated by the relationship between the action and its consequences or outcome. With continued training, however, the performance of such actions can become automatic, reflexive or habitual and under the control of antecedent stimuli rather than their consequences. Recent evidence suggests that habit learning is mediated by plasticity in the dorsolateral striatum (DLS). To date, however, no direct evidence of learning-related plasticity associated with overtraining has been reported in this region, nor is it known whether, or which, specific cell types are involved in this learning process. The striatum is primarily composed of two classes of spiny projection neurons, the striatonigral and striatopallidal spiny projection neurons, which express dopamine D1 and D2 receptors, and control direct and indirect pathways, respectively. Here we found evidence of a post-synaptic depression in DLS striatopallidal projecting neurons in the indirect pathway during habit learning in mice. Moreover, this training-induced depression occluded post-synaptic depression induced by co-activation of D2 receptors and transient receptor potential vanilloid 1 (TRPV1) channels, implying that this pathway is involved in habit learning. This hypothesis was further tested by disrupting this signal pathway by knocking out TRPV1 channels, resulting in compromised habit learning. Our findings suggest that post-synaptic plasticity at D2 neurons in the DLS mediates habit learning and, by implicating an interaction between the D2 receptor and TRPV1 channel activity, provide a potential drug target for influencing habitual action control.NHMRC grants: 1045964 & 108925

A specific role for posterior dorsolateral striatum in human habit learning

Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen–globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans

Electrocortical components of anticipation and consumption in a monetary incentive delay task

In order to improve our understanding of the components that reflect functionally important processes during reward anticipation and consumption, we used principle components analyses (PCA) to separate and quantify averaged ERP data obtained from each stage of a modified monetary incentive delay (MID) task. Although a small number of recent ERP studies have reported that reward and loss cues potentiate ERPs during anticipation, action preparation, and consummatory stages of reward processing, these findings are inconsistent due to temporal and spatial overlap between the relevant electrophysiological components. Our results show three components following cue presentation are sensitive to incentive cues (N1, P3a, P3b). In contrast to previous research, reward‐related enhancement occurred only in the P3b, with earlier components more sensitive to break‐even and loss cues. During feedback anticipation, we observed a lateralized centroparietal negativity that was sensitive to response hand but not cue type. We also show that use of PCA on ERPs reflecting reward consumption successfully separates the reward positivity from the independently modulated feedback‐P3. Last, we observe for the first time a new reward consumption component: a late negativity distributed over the left frontal pole. This component appears to be sensitive to response hand, especially in the context of monetary gain. These results illustrate that the time course and sensitivities of electrophysiological activity that follows incentive cues do not follow a simple heuristic in which reward incentive cues produce enhanced activity at all stages and substages

Alcohol-Paired Contextual Cues Produce an Immediate and Selective Loss of Goal-directed Action in Rats

We assessed whether the presence of contextual cues paired with alcohol would disrupt rats’ capacity to express appropriate goal-directed action control. Rats were first given differential context conditioning such that one set of contextual cues was paired with the injection of ethanol and a second, distinctive set of cues was paired with the injection of saline. All rats were then trained in a third, neutral context to press one lever for grain pellets and another lever for sucrose pellets. They were then given two extinction tests to evaluate their ability to choose between the two actions in response to the devaluation of one of the two food outcomes with one test conducted in the alcohol-paired context and the other conducted in the control (saline-paired) context. In the control context, rats exhibited goal-directed action control; i.e., they were able selectively to withhold the action that previously earned the now devalued outcome. However, these same rats were impaired when tested in the alcohol-paired context, performing both actions at the same rate regardless of the current value of their respective outcomes. Subsequent testing revealed that the rats were capable of overcoming this impairment if they were giving response-contingent feedback about the current value of the food outcomes. These results provide a clear demonstration of the disruptive influence that alcohol-paired cues can exert on decision-making in general and goal-directed action selection and choice in particular

Chronic morphine reduces surface expression of δ-opioid receptors in subregions of rostral striatum

The delta opioid receptor (DOPr), whilst not the primary target of clinically used opioids, is involved in development of opioid tolerance and addiction. There is growing evidence that DOPr trafficking is involved in drug addiction, e.g., a range of studies have shown increased plasma membrane DOPr insertion during chronic treatment with opioids. The present study used a transgenic mouse model in which the C-terminal of the DOPr is tagged with enhanced-green fluorescence protein to examine the effects of chronic morphine treatment on surface membrane expression in striatal cholinergic interneurons that are implicated in motivated learning following both chronic morphine and morphine sensitization treatment schedules in male mice. A sex difference was noted throughout the anterior striatum, which was most prominent in the nucleus accumbens core region. Incontrast with previous studies in other neurons, chronic exposure to a high dose of morphine for 6 days had no effect, or slightly decreased (anterior dorsolateral striatum) surface DOPr expression. A morphine sensitization schedule produced similar results with a significant decrease in surface DOPr expression in nucleus accumbens shell. These results suggest that chronic morphine and morphine sensitisation treatment may have effects on instrumental reward-seeking behaviours and learning processes related to drug addiction, via effects on striatal DOPr function.NHMRC Grants: 1045964 & 108925

Chronic morphine reduces surface expression of δ-opioid receptors in subregions of rostral striatum

The delta opioid receptor (DOPr), whilst not the primary target of clinically used opioids, is involved in development of opioid tolerance and addiction. There is growing evidence that DOPr trafficking is involved in drug addiction, e.g., a range of studies have shown increased plasma membrane DOPr insertion during chronic treatment with opioids. The present study used a transgenic mouse model in which the C-terminal of the DOPr is tagged with enhanced-green fluorescence protein to examine the effects of chronic morphine treatment on surface membrane expression in striatal cholinergic interneurons that are implicated in motivated learning following both chronic morphine and morphine sensitization treatment schedules in male mice. A sex difference was noted throughout the anterior striatum, which was most prominent in the nucleus accumbens core region. Incontrast with previous studies in other neurons, chronic exposure to a high dose of morphine for 6 days had no effect, or slightly decreased (anterior dorsolateral striatum) surface DOPr expression. A morphine sensitization schedule produced similar results with a significant decrease in surface DOPr expression in nucleus accumbens shell. These results suggest that chronic morphine and morphine sensitisation treatment may have effects on instrumental reward-seeking behaviours and learning processes related to drug addiction, via effects on striatal DOPr function.NHMRC Grants: 1045964 & 108925

Learning-related translocation of δ-opioid receptors on ventral striatal cholinergic interneurons mediates choice between goal-directed actions.

The ability of animals to extract predictive information from the environment to inform their future actions is a critical component of decision-making. This phenomenon is studied in the laboratory using the pavlovian-instrumental transfer protocol in which a stimulus predicting a specific pavlovian outcome biases choice toward those actions earning the predicted outcome. It is well established that this transfer effect is mediated by corticolimbic afferents on the nucleus accumbens shell (NAc-S), and recent evidence suggests that delta-opioid receptors (DORs) play an essential role in this effect. In DOR-eGFP knock-in mice, we show a persistent, learning-related plasticity in the translocation of DORs to the somatic plasma membrane of cholinergic interneurons (CINs) in the NAc-S during the encoding of the specific stimulus-outcome associations essential for pavlovian-instrumental transfer. Wefound that increased membrane DOR expression reflected both stimulus-based predictions of reward and the degree to which these stimuli biased choice during the pavlovian-instrumental transfer test. Furthermore, this plasticity altered the firing pattern of CINs increasing the variance of action potential activity, an effect that was exaggerated by DOR stimulation. The relationship between the induction of membrane DOR expression in CINs and both pavlovian conditioning and pavlovian-instrumental transfer provides a highly specific function for DOR-related modulation in the NAc-S, and it is consistent with an emerging role for striatal CIN activity in the processing of predictive information. Therefore, our results reveal evidence of a long-term, experience-dependent plasticity in opioid receptor expression on striatal modulatory interneurons critical for the cognitive control of action