The Role of the Striatum in Learning to Orthogonalize CD Action and Valence: A Combined PET and 7 T MRI Aging Study

Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age

From prediction error to incentive salience: mesolimbic computation of reward motivation

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue‐triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue‐triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience. Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/90564/1/j.1460-9568.2012.07990.x.pd

The Nucleus Accumbens Core Dopamine D1 and Glutamate AMPA/NMDA Receptors Play a Transient Role in the Performance of Pavlovian Approach Behavior

The role of the nucleus accumbens core (NAc core) continues to be redefined with newly acquired data on neurochemical mechanisms mediating the learning and performance of behavior. Previous empirical data showed that dopamine transmission at the D1 receptor (D1R) plays a transient role in the expression of learned Pavlovian approach behavior. Here we show that, prior to overtraining, dopamine activity at D1Rs specifically within the NAc core is critical for the performance of approach behavior elicited by the recently-acquired reward-paired cue. Blockade of D1Rs in the NAc core, but not the dorsomedial striatum or NAc shell, disrupted approach responses during early training; however, the dependence of Pavlovian approach on D1R transmission declined throughout training. Upon blockade of NAc core D1Rs during extended training, the expression of Pavlovian approach responses remained intact. Given these findings we next explored whether a) neuronal activity within the core of accumbens still mediates cued approach during the late training stages in the absence of D1R transmission by relying on glutamatergic transmission or b) whether mediation of the cued approach becomes independent of the NAc core itself, i.e., shifts to another substrate. We blocked AMPA/NMDA receptors in the NAc core during early versus extended training and showed that loss of neuronal activation in the NAc core only disrupted expression of conditioned stimulus-elicited responses during early training. Our results indicate that NAc core activity is not necessary for the expression of well-acquired approach

A neurocomputational account of self-other distinction: from cell to society

Human social systems are unique in the animal kingdom. Social norms, constructed at a higher level of organisation, influence individuals across vast spatiotemporal scales. Characterising the neurocomputational processes that enable the emergence of these social systems could inform holistic models of human cognition and mental illness. Social neuroscience has shown that the processing of ‘social’ information demands many of the same computations as those involved in reasoning about inanimate objects in ‘non-social’ contexts. However, for people to reason about each other’s mental states, the brain must be able to distinguish between one mind and another. This ability, to attribute a mental state to a specific agent, has long been studied by philosophers under the guise of ‘meta-representation’. Empathy research has taken strides in describing the neural correlates of representing another person’s affective or bodily state, as distinct from one’s own. However, Self-Other distinction in beliefs, and hence meta-representation, has not figured in formal models of cognitive neuroscience. Here, I introduce a novel behavioural paradigm, which acts as a computational assay for Self-Other distinction in a cognitive domain. The experiments in this thesis combine computational modelling with magnetoencephalography and functional magnetic resonance imaging to explore how basic units of computation, predictions and prediction errors, are selectively attributed to Self and Other, when subjects have to simulate another agent’s learning process. I find that these low-level learning signals encode information about agent identity. Furthermore, the fidelity of this encoding is susceptible to experience-dependent plasticity, and predicts the presence of subclinical psychopathological traits. The results suggest that the neural signals generating an internal model of the world contain information, not only about ‘what’ is out there, but also about ‘who’ the model belongs to. That this agent-specificity is learnable highlights potential computational failure modes in mental illnesses with an altered sense of Self

Neuroeconomics: How Neuroscience Can Inform Economics

Neuroeconomics uses knowledge about brain mechanisms to inform economic analysis, and roots economics in biology. It opens up the "black box" of the brain, much as organizational economics adds detail to the theory of the firm. Neuroscientists use many tools— including brain imaging, behavior of patients with localized brain lesions, animal behavior, and recording single neuron activity. The key insight for economics is that the brain is composed of multiple systems which interact. Controlled systems ("executive function") interrupt automatic ones. Emotions and cognition both guide decisions. Just as prices and allocations emerge from the interaction of two processes—supply and demand— individual decisions can be modeled as the result of two (or more) processes interacting. Indeed, "dual-process" models of this sort are better rooted in neuroscientific fact, and more empirically accurate, than single-process models (such as utility-maximization). We discuss how brain evidence complicates standard assumptions about basic preference, to include homeostasis and other kinds of state-dependence. We also discuss applications to intertemporal choice, risk and decision making, and game theory. Intertemporal choice appears to be domain-specific and heavily influenced by emotion. The simplified ß-d of quasi-hyperbolic discounting is supported by activation in distinct regions of limbic and cortical systems. In risky decision, imaging data tentatively support the idea that gains and losses are coded separately, and that ambiguity is distinct from risk, because it activates fear and discomfort regions. (Ironically, lesion patients who do not receive fear signals in prefrontal cortex are "rationally" neutral toward ambiguity.) Game theory studies show the effect of brain regions implicated in "theory of mind", correlates of strategic skill, and effects of hormones and other biological variables. Finally, economics can contribute to neuroscience because simple rational-choice models are useful for understanding highly-evolved behavior like motor actions that earn rewards, and Bayesian integration of sensorimotor information

Reductions in Mesolimbic Dopamine Signaling and Aversion: Implications for Relapse and Learned Avoidance

The ability to adjust behavior appropriately following an aversive experience is essential for survival, yet variability in this process contributes to a wide range of disorders, including drug addiction. It is clear that proper approach and avoidance is regulated, in part, by the activity of the mesolimbic dopamine system. While the importance of this system as a critical modulator of reward learning has been extensively characterized, its involvement in directing aversion-related behaviors and learning is still poorly understood. Recent studies have revealed that aversive stimuli and their predictors cause rapid reductions in nucleus accumbens (NAc) dopamine concentrations. Furthermore, a normally appetitive stimulus that is made aversive through association with cocaine also decreases dopamine, and the magnitude of the expressed aversion predicts drug-taking. However, whether the presentation of a drug cue that reduces dopamine, and evokes a negative affective state, can motivate relapse is unknown. Here we demonstrate that the presentation of an aversive drug cue both reduces dopamine and causes cocaine-seeking. This finding is provocative because drug seeking in reinstatement designs is typically associated with increased dopamine signaling. Using a combination of fast scan cyclic voltammetry (FSCV) and in vivo electrophysiology we subsequently show that the presence of an aversive drug cue abolishes the dopaminergic encoding of other drug cues and alters NAc neuronal activity patterns. Importantly, a subpopulation of neurons that subsequently encode aspects of drug-seeking behavior increase their baseline firing rates during this aversive experience. We then examine the mechanistic regulation of dopamine signaling by aversive stimuli under more natural conditions. Using FSCV and site-specific behavioral pharmacology we demonstrate that blockade of ventral tegmental area kappa opioid receptors attenuates aversion-induced reductions in dopamine, and prevents proper avoidance learning caused by punishment. By maintaining D2 receptor occupancy within the NAc during punishment, we demonstrate the requirement of aversion-induced reductions in dopamine for aversive learning. Together, these studies inform an evolving model of striatal physiology. Our findings emphasize a role for both increases and decreases in dopamine signaling that modulate behavior by promoting the stimulus-specific activity of distinct striatal output pathways. The continued interrogation of this model may offer novel targets for therapeutic development aimed at treating neurodegenerative disease and drug addiction

Implication du striatum et du pallidum ventral dans le traitement de l'information aversive : approche électrophysiologique et pharmacologique chez le primate non-humain

Striatum and globus pallidus belong to the basal ganglia, which are a group of subcortical structures involved in motor, cognitive and motivational functions. They are also involved in the reward system which enables one’s motivation to initiate approach behaviors in order to get a reward and then consolidate activities that have produced these behaviors. This represents instrumental learning basis. However, in some situations, the behavior to be initiated depends on a motivation to escape or avoid an aversive situation. For a long time, it was thought that aversive information processing system and reward system depended on different networks and cerebral regions. Yet, a growing number of studies tend to show that basal ganglia certainly play an important role in aversive information processing. In this thesis, we recorded neuronal activity and performed local pharmacological perturbations in non-human primates, in two structures of the reward system, anterior striatum and ventral pallidum, while performing a behavioral task requiring them to initiate alternatively approach behaviors toward a reward and avoidance behaviors from an aversive event. We showed aversive information coding for the predictive stimulus, preparation and initiation of the avoidance behavior and anticipation and response to the aversive events. Furthermore, local perturbation experiments demonstrated that a functional impairment of the anterior striatum and the ventral pallidum affects the behaviors usually initiated by the animals in aversive context. Put together, these results clearly show that these two cerebral structures are involved in both appetitive and aversive motivationsLes ganglions de la base, auxquels appartiennent le striatum et le globus pallidus, sont un ensemble de structures sous-corticales impliquées dans des fonctions motrices, cognitives et motivationnelles. Il a également été montré qu’ils font partie du système de récompense, système assurant la motivation d’un organisme à initier des comportements d’approche, de façon à obtenir une récompense, puis à renforcer les activités ayant produit ces comportements pour pouvoir les reproduire par la suite, ce qui constitue la base de l’apprentissage instrumental. Or, dans certaines situations, le comportement à initier va dépendre d’une motivation à fuir ou à éviter un événement ou une situation aversive. Longtemps, on a pensé que le système du traitement des informations aversives était distinct de celui de la récompense, en termes de réseau et de régions cérébrales. Pourtant, de plus en plus d’études montrent à présent que les ganglions de la base ont certainement un rôle à jouer, non négligeable, dans le traitement des situations aversives. Dans ce travail de thèse, nous avons réalisé chez le primate non-humain des enregistrements d’activité neuronale et des perturbations pharmacologiques locales dans deux régions du système de récompense, le striatum antérieur et le pallidum ventral, au cours d’une tâche comportementale nécessitant tour à tour l’initiation de comportements d’approche vers une récompense et de comportements d’évitement d’un événement aversif. Nous avons montré l’existence d’un codage de l’information aversive dans ces régions tant pour un stimulus prédicteur d’un événement aversif, la préparation ou l’initiation d’un comportement d’évitement de cet événement, que pour l’anticipation et la réception de cet événement. Par ailleurs, les expériences de perturbations locales ont démontré qu’une atteinte du fonctionnement normal du striatum antérieur et du pallidum ventral affectait les comportements initiés normalement en contexte aversif. L’ensemble de ces résultats indique clairement que ces deux structures cérébrales, si elles sont impliquées dans la motivation à initier des comportements dirigés vers un but récompensant et l’apprentissage appétitif, le sont aussi dans la motivation aversiv