A recent study by Saunders, Richard, Margolis, and Janak (2018) shows that there is a great deal left to learn about what different mesotelencephalic dopamine circuits contribute to learning about the motivational significance of reward-related cues. Their findings suggest that nigrostriatal and mesolimbic dopamine pathways support distinct reinforcement processes that independently push and pull animals toward their goals

Ostlund, Sean B

eScholarship - University of California

UC IrvineUC Irvine Previously Published WorksTitleThe push and pull of dopamine in cue-reward learningPermalinkhttps://escholarship.org/uc/item/7tv5f4kfJournalLearning & Behavior, 47(4)ISSN1543-4494AuthorOstlund, Sean BPublication Date2019-12-01DOI10.3758/s13420-018-0370-x Peer reviewedeScholarship.org Powered by the California Digital LibraryUniversity of CaliforniaThe push and pull of dopamine in cue-reward learningSean B. OstlundDepartment of Anesthesiology and Perioperative Care and Irvine Center for Addiction Neuroscience, University of California, Irvine, CA, USAAbstractA recent study by Saunders, Richard, Margolis and Janak (2018) shows that there is a great deal left to learn about what different mesotelencephalic dopamine circuits contribute to learning about the motivational significance of reward-related cues. Their findings suggest that nigrostriatal and mesolimbic dopamine pathways support distinct reinforcement processes that independently push and pull animals toward their goals.Keywordsincentive motivation; general arousal; optogenetics; reinforcement; PavlovianCues that signal reward acquire powerful motivational properties which allow them to demand attention and promote vigorous reward-seeking behavior. Rewards are known to trigger phasic dopamine neuron activity, which may serve as a reinforcement signal for Pavlovian motivational learning (Berridge, 2012). Research on this topic has typically involved manipulating or measuring dopamine system function in animals as they receive pairings between a cue and a natural reward stimulus, like palatable food. While such studies demonstrate the importance of dopamine in cue-reward learning, they do not address whether dopamine is the main driver of this process or whether it plays a supporting role, modulating the efficacy of other reward-elicited neural processes more directly responsible for assigning motivational value to cues.A new study by Saunders, Richard, Margolis and Janak (2018) approached this problem in an innovative way by asking if simply stimulating dopamine neurons – without reward delivery – is sufficient to reinforce Pavlovian learning. Rats were given repeated pairings between an audiovisual cue (7-sec) and phasic, optogenetic stimulation of midbrain dopamine neurons (5-sec, co-terminating with cue). If reward-elicited dopamine neuron firing serves as the crucial reinforcement signal for cue-reward learning, then pairing a neutral cue with artificial dopamine neuron stimulation (without other reward-related neural activity) should be sufficient to assign that cue motivational properties.In line with this prediction, Saunders et al. found that rats developed conditioned locomotor responses to cues paired with dopamine neuron activation, regardless of whether those Sean Ostlund, University of California, Irvine, 837 Health Science Road, 3111GNRF, Irvine, CA 92697, sostlund@uci.edu, Tel: 949-824-1347HHS Public AccessAuthor manuscriptLearn Behav. Author manuscript; available in PMC 2020 December 01.Published in final edited form as:Learn Behav. 2019 December ; 47(4): 273–274. doi:10.3758/s13420-018-0370-x.Author ManuscriptAuthor ManuscriptAuthor ManuscriptAuthor Manuscriptneurons were located in the ventral tegmental area (VTA) or substantia nigra pars compacta (SNc). Moreover, fine-grained analyses revealed that these two dopamine circuits supported different forms of conditioned responding. Rats tended to approach cues associated with VTA dopamine neuron stimulation. This sign-tracking response is also elicited by food-paired cues and is used as a selective assay of Pavlovian incentive motivation (Robinson & Flagel, 2009). Rats also readily pressed a lever to produce cues that had been paired with VTA dopamine neuron stimulation (conditioned reinforcement), another common assay of Pavlovian incentive motivation (Robinson & Flagel, 2009). Therefore, phasic activation of VTA dopamine neurons was sufficient to attribute incentive value to associated cues, at least by these measures. In contrast, cues paired with SNc dopamine neuron stimulation did not support sign-tracking or conditioned reinforcement, but instead caused rats to turn in circles directed away from the stimulated hemisphere. Moreover, the authors found that conditioned responses elicited by cues paired with SNc dopamine neuron stimulation were considerably more vigorous (i.e., faster) that those elicited by cues that signaled VTA dopamine neuron stimulation, providing further evidence that these different dopamine circuits support distinct behavioral functions.Saunders et al. also showed that the learning produced by stimulating dopamine neurons depends on both the location of those neurons and where they send their projections. Cues paired with stimulation of VTA dopamine neurons that innervate the nucleus accumbens core supported sign-tracking and conditioned reinforcement. This was not the case for cues paired with stimulation of a distinct group of VTA dopamine neurons that project to the nucleus accumbens shell or for cues paired with stimulation of SNc dopamine neurons projecting to the dorsal striatum. However, activating this projection-defined nigrostriatal dopamine circuit was sufficient to establish vigorous conditioned turning behavior, replicating the effects of less specific SNc dopamine neuron stimulation described above.Based on such findings, the authors posit that phasic activity within the VTA → core dopamine pathway is responsible for assigning incentive motivational properties to neutral cues – making them attractive and worthy of pursuit, whereas the SNc → dorsal striatum dopamine pathway is said to mediate a separate form of motivational learning, one that promotes vigorous movement without setting a specific direction or target for that behavior. Put another way, these dopamine circuits may motivate behavior by separately pulling (mesolimbic) and pushing (nigrostriatal) animals along toward their goals. This maps on nicely to a long-standing theoretical distinction between incentive motivation, on the one hand, and drive (or the related concept of general motivational arousal), on the other, which can be viewed as serving similar pull and push functions, respectively (Bolles, 1967).The hypothesis that motivational learning is fractionated in this manner within the dopamine system certainly deserves further investigation. One way forward is to probe the ability for dopamine-paired cues to invigorate performance of an independently trained instrumental action. This Pavlovian-to-instrumental transfer effect provides an index of the nonspecific response-invigorating influence of reward-paired cues, which is known to depend on dopamine signaling (Ostlund & Maidment, 2011). The hypothesis stated above predicts that cues paired with SNc dopamine neuron stimulation should elicit more vigorous instrumental responding than cues paired with VTA dopamine neuron stimulation.Ostlund Page 2Learn Behav. Author manuscript; available in PMC 2020 December 01.Author ManuscriptAuthor ManuscriptAuthor ManuscriptAuthor ManuscriptThe notion that the nigrostriatal dopamine pathway modulates response vigor generally fits with the role this system plays in controlling the expression of voluntary movement and how dysfunction within this system produces motor slowing (bradykinesia) in Parkinson’s disease. However, Saunder et al. address a fundamentally different question. Rather than focusing on expression of movement, they show that activating the nigrostriatal pathway can trigger a learning process that allows previously neutral cues to arouse movement. The authors speculate that this motivational process may play an important role in regulating expression of voluntary behaviors, establishing cues that facilitate responding and may therefore help overcome motor deficits. This is a ripe topic for future research.Other findings in this study raise questions about dopamine’s relative contributions to learning versus performance. Saunders et al. focused almost exclusively on learning, showing that phasic dopamine neuron activity is sufficient to establish new conditioned responding. They also applied fiber photometry to reveal that cues paired with dopamine neuron stimulation come to elicit their own conditioned dopamine neuron activity. Although the inference here is that this conditioned dopamine neuron activity may contribute to the expression of cue-elicited behavior, this remains to be established. However, it is notable that the authors found that directly stimulating VTA or SNc dopamine neurons (in the absence of cues) was not sufficient to elicit unconditioned locomotor responses. This suggests that if cue-elicited dopamine neuron activity contributes to the expression of the conditioned responses described above, it likely plays a supporting role, acting in concert with other cue-elicited neural processes.Interestingly, Saunders et al. found that dopamine neuron activity had characteristics resembling a reward-prediction error signal, in that omitting and expected optogenetic stimulation led to a dip in dopamine neuron activity, relative to baseline levels. This fits with the view that phasic dopamine signaling subserves a reinforcement learning process (cf. Berridge, 2012), though the nature of the learning supported by this process is not clear. Saunders et al. provide good evidence that the mesolimbic dopamine system supports Pavlovian incentive learning, but also show that rats will lever press for either VTA or SNc dopamine neuron stimulation. Thus, it is possible that this response-reinforcement process may have also contributed to the development of cue-elicited behavior. While cues were paired with dopamine neuron stimulation in a response-independent (Pavlovian) manner, this would not prevent that stimulation from reinforcing actions that just happened to be performed during cues. This may have helped strengthening or amplify response tendencies established through Pavlovian conditioning, which may otherwise have been relatively weak. Unconditioned orienting responses (approach) directed at the cue light may have also been unintentionally paired with dopamine neuron stimulation, contributing to the development of conditioned sign-tracking in rats receiving VTA dopamine neuron stimulation. However, this would not explain why cues paired with SNc dopamine neuron stimulation elicited conditioned turning behavior. Thus, while the degree to which response-reinforcement processes contributed to these conditioned responses remains unclear, it seems likely that these behaviors were predominantly established through a Pavlovian reinforcement process.Ostlund Page 3Learn Behav. Author manuscript; available in PMC 2020 December 01.Author ManuscriptAuthor ManuscriptAuthor ManuscriptAuthor ManuscriptIn summary, Saunders et al. provide compelling evidence that activating dopamine neurons is sufficient for assigning motivational significance to cues and that the nature of this learning is pathway-specific. This study also demonstrates the unique insights that can be gained by combining cutting-edge neuroscience techniques with sophisticated behavioral analysis. And like with any good study, it raises more questions than it answers.ReferencesBerridge KC (2012). From prediction error to incentive salience: mesolimbic computation of reward motivation. European Journal of Neuroscience, 35(7), 1124–1143. doi:10.1111/j.1460-9568.2012.07990.x [PubMed: 22487042] Bolles RC (1967). Theory of motivation. New York,: Harper & Row.Ostlund SB, & Maidment NT (2011). Dopamine Receptor Blockade Attenuates the General Incentive Motivational Effects of Noncontingently Delivered Rewards and Reward-Paired Cues Without Affecting Their Ability to Bias Action Selection. Neuropsychopharmacology, 37, 508. doi:10.1038/npp.2011.217 [PubMed: 21918507] Robinson TE, & Flagel SB (2009). Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biol Psychiatry, 65(10), 869–873. doi:10.1016/j.biopsych.2008.09.006 [PubMed: 18930184] Saunders BT, Richard JM, Margolis EB, & Janak PH (2018). Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature Neuroscience, 21(8), 1072–1083. doi:10.1038/s41593-018-0191-4 [PubMed: 30038277] Ostlund Page 4Learn Behav. Author manuscript; available in PMC 2020 December 01.Author ManuscriptAuthor ManuscriptAuthor ManuscriptAuthor Manuscript

The push and pull of dopamine in cue-reward learning

Abstract

Similar works

Full text

Available Versions

eScholarship - University of California