Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain

The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However,the separate pathways inthe striatumfor go and no-go appearto violatethis independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action- outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects" requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipatedto be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment

Go and No-go Learning in Reward and Punishment: Interactions between Affect and Effect

Decision-making invokes two fundamental axes of control: affect or valence, spanning reward and punishment, and effect or action, spanning invigoration and inhibition. We studied the acquisition of instrumental responding in healthy human volunteers in a task in which we orthogonalized action requirements and outcome valence. Subjects were much more successful in learning active choices in rewarded conditions, and passive choices in punished conditions. Using computational reinforcement-learning models, we teased apart contributions from putatively instrumental and Pavlovian components in the generation of the observed asymmetry during learning. Moreover, using model-based fMRI, we showed that BOLD signals in striatum and substantia nigra/ventral tegmental area (SN/VTA) correlated with instrumentally learnt action values, but with opposite signs for go and no-go choices. Finally, we showed that successful instrumental learning depends on engagement of bilateral inferior frontal gyrus. Our behavioral and computational data showed that instrumental learning is contingent on overcoming inherent and plastic Pavlovian biases, while our neuronal data showed this learning is linked to unique patterns of brain activity in regions implicated in action and inhibition respectively

Corrigendum: Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression

Motivational salience plays an important role in shaping human behavior, but recent studies demonstrate that human performance is not uniformly improved by motivation. Instead, action has been shown to dominate valence in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward, but the neural mechanism behind this behavioral specificity is yet unclear. In all mammals, including humans, the monoamine neurotransmitter dopamine is particularly important in the neural manifestation of appetitively motivated behavior, and the human dopamine system is subject to considerable genetic variability. The well-studied TaqIA restriction fragment length polymorphism (rs1800497) has previously been shown to affect striatal dopamine metabolism. In this study we investigated a potential effect of this genetic variation on motivated action/inhibition learning. Two independent cohorts consisting of 87 and 95 healthy participants, respectively, were tested using the previously described valenced go/no-go learning paradigm in which participants learned the reward-associated no-go condition significantly worse than all other conditions. This effect was modulated by the TaqIA polymorphism, with carriers of the A1 allele showing a diminished learning-related performance enhancement in the rewarded no-go condition compared to the A2 homozygotes. This result highlights a modulatory role for genetic variability of the dopaminergic system in individual learning differences of action-valence interaction

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans

Ansiedad en ratas genéticamente heterogéneas : hacia la identificación de genes para caracteres conductuales cuantitativos

El uso de roedores genéticamente heterogéneos constituye una estrategia única para la identificación y el 'mapeo fino' a alta resolución de locus genéticos (QTL) con influencia sobre fenotipos cuantitativos biológicos y conductuales, permitiendo la identificación de genes individuales (genes cuantitativos) con acción sobre aquéllos. Presentamos el primer estudio de este tipo con ratas genéticamente heterogéneas (N/Nih-HS; derivadas de ocho cepas de ratas consanguíneas), que se evalúan en varias pruebas conductuales que miden ansiedad/miedo no aprendidos ('Caja blanca/negra', 'Laberinto en 'cero' elevado') o aprendidos (conducta de petrificación en contexto condicionado, adquisición de la evitación activa en dos sentidos en la caja de vaivén). Las ratas N/Nih-HS presentan una conducta más parecida a la de la cepa consanguínea RLA-I (ansiosas) que a la de la RHA-I (poco ansiosas). Se hallan correlaciones significativas entre variables incondicionadas de ansiedad y de la adquisición de la evitación activa en dos sentidos, confirmadas parcialmente por análisis de regresión múltiple. Las ratas N/Nih-HS relativamente 'evitadoras' exhiben niveles más bajos de ansiedad no aprendida que las poco 'evitadoras'. Los resultados de esta evaluación del comportamiento de las ratas N/Nih-HS son discutidos en términos de su potencial utilidad para la investigación neurogenética de la ansiedad y el miedoThe use of genetically heterogeneous (outbred) rodents is a unique resource for the identification and fine mapping of genetic loci (QTL) influencing biological and behavioural quantitative phenotypes, allowing the identification of quantitative genes. We present the first study of this kind carried out with genetically heterogeneous rats (N/Nih-HS; derivated from an eight-way cross of inbred strains), whose behaviour is assessed in tests evoking unlearned (Black/white box, Elevated 'zero' maze) or learned (context conditioned freezing, two-way active avoidance acquisition in the shuttlebox) anxious/fearful responses. The behavioural profile of N/Nih-HS rats is more similar to that of RLA-I (anxious) rats rather than to RHA-I (low anxious) rats. Significant correlations are found among unconditioned anxiety variables and two-way active avoidance acquisition in the shuttlebox; these are partially confirmed by multiple regression analysis. 'High avoider' N/Nih-HS rats show lower unlearned anxiety levels than 'low avoiders'. Results of this behavioural assessment of the N/Nih-HS rats are discussed in terms of their potential usefulness for present and future neurobehavioural and genetic studies of fearfulness and anxiet

Divergent sensitivities to drugs of abuse : neurochemical and neuroanatomical characterization of the Roman rats /

Consultable des del TDXTítol obtingut de la portada digitalitzadaNomés un petit percentatge d'entre tots els individus que experimenten amb drogues d'abús esdevenen addictes. La vulnerabilitat per a desenvolupar un trastorn addictiu està relacionada amb trets de personalitat impulsiva o amb apetència per la novetat. Les soques de rata Romanes, genèticament seleccionades per alta (RHA) o baixa (RLA) adquisició de l'evitació activa en dos sentits, són un model de laboratori vàlid de les divergències en apetència per la novetat i substàncies gratificants que s'observen en humans. A més a més, aquestes soques de rata difereixen en la funcionalitat del sistema dopaminèrgic. Se sap que les rates RHA beuen etanol voluntàriament mentre que les rates RLA mostren aversió. En aquesta tesis, les rates Romanes s'empren com a model de vulnerabilitat addictiva. L'objectiu de la tesis ha consistit en entendre els mecanismes neurobiològics que sustenten les diferències de vulnerabilitat addictiva entre les dues soques de rata Romanes. El treball s'ha dividit en 3 fases experimentals. En primer lloc, es va avaluar la resposta conductual de les rates Romanes a la injecció d'una dosi baixa d'etanol. Tal i com s'observa en humans amb elevat risc d'esdevenir alcohòlics, les rates RHA van ser menys sensibles als efectes conductuals d'aquesta dosi. En la segona fase, es varen estudiar els cervells de rates Romanes naïve per tal de caracteritzar vàries dianes moleculars del sistema dopaminèrgic i neuropèptids relacionats: es van quantificar diferents subtipus de receptor de dopamina per mitjà de la tècnica d'autoradiografia de receptors així com els nivells d'expressió d'ARNm per diferents neuropèptids mitjançant la tècnica d'hibridació in situ. Comparades amb les rates RLA, les rates RHA presenten majors nivells de receptors D1 i D3 així com també d'ARNm per la DYN a nivell de l'escorça del nucli accumbens (NAc). En canvi, presenten menors nivells d'expressió del receptor D3 a nivell de les illes de Calleja. A més a més, quan s'administra un agonista D3, les rates RLA mostren major inhibició de l'activitat locomotora i supressió de l'expressió del gen NGFI-A a la Calleja magna (mesurat per mitjà de tècniques d'hibridació in situ) que les rates RHA. Aquests resultats afegeixen noves evidències de les diferències en la funcionalitat del sistema dopaminèrgic i potser representen l'eix central de l'entramat neurobiològic subjacent a les diferències en apetència per la novetat i preferència per a les drogues d'abús com l'etanol entre les dues soques. En la tercera fase, es va estudiar el fenomen de sensibilització conductual induït per amfetamina en les rates Romanes i es van elaborar mapes d'activitat neuronal per mitjà de tècniques d'hibridació in situ amb cinc gens d'expressió immediata. La sensibilització conductual és un model de plasticitat conductual i neuronal induïda per l'ús crònic de drogues. Les rates RHA que van rebre un tractament crònic amb amfetamina van mostrar sensibilització conductual i un augment en l'expressió dels gens de secretogranina i PSD95 al corus del NAc. Aquestes troballes es discuteixen en el context d'altres models de laboratori d'ús crònic de drogues. D'altra banda, les rates RLA no van mostrar sensibilització conductual però en canvi van mostrar adaptacions neuronals que poden estar relacionades amb la manca de sensibilització. A més a més, les rates RLA que van rebre amfetamina per primer cop van mostrar activació del nucli central de l'amígdala (CeA). També es va detectar activació del CeA en ratolins que van rebre un tractament amb etanol i naltrexona, fàrmac que s'empra a la pràctica clínica per tal de prevenir les recaigudes en alcohòlics. Aquesta troballa suggereix que el CeA pot ser una estructura del cervell rellevant per tal de frenar el desenvolupament dels trastorns addictius.Hundreds of millions of people experiment with drugs of abuse, but only a small percentage of them become addicted. Vulnerability to develop addiction has been associated with impulsivity or novelty-seeking. The Roman rats, genetically selected for high (RHA) or low (RLA) active avoidance acquisition in the two-way shuttle box, appear to be a valid laboratory model of divergent novelty and substance-seeking profiles and differ in the functionality of the dopaminergic system. So far, it is known that RHA rats drink ethanol voluntarily whereas RLA rats show aversion to it. In the present thesis, the Roman rats have been used as a model of differences in vulnerability to addiction. The aim of the thesis was to understand the neurobiological mechanisms that underlie such differences in vulnerability between the two Roman rat strains. The work has been divided in 3 experimental blocks. First, we studied the behavioral response to an injection of a low dose of ethanol in the Roman rats. Like those humans that have higher risk to develop alcoholism, RHA rats were less sensitive to the effects of low-dose of ethanol. Second, brains of naïve Roman rats were studied in order to characterize several molecular targets of the dopaminergic system and related neuropeptides: dopamine receptor subtypes were quantified by means of receptor autoradiography and mRNA coding for neuropeptides were quantified using in situ hybridization histochemistry. When compared to RLA rats, RHA showed higher binding of D1 and D3 dopamine receptor subtypes and DYN mRNA expression in the nucleus accumbens (NAc) shell, although they showed lower basal binding of D3 receptors in the Calleja islands. Moreover, a challenge with a D3 agonist resulted in greater inhibition of locomotor activity as well as supression of NGFI-A mRNA as measured with in situ hybridization in the Calleja magna in RLA rats when compared to RHA rats. These results provide further evidences of the differences in dopamine function between the Roman strains and may represent the neurobiological core of the divergences in novelty-seeking and preference for addictive drugs such as ethanol. Third, behavioral sensitization, a model of behavioral and neuronal plasticity secondary to chronic drug use, was also studied in the Roman rats. Neuronal activity maps with 5 different immediate early genes were made by means of in situ hybridization. Amphetamine pre-treated RHA rats showed behavioral sensitization and increased secretogranin and PSD-95 in the NAc core which is suggestive of increased glutamatergic activity at this site. These findings are discussed in the context of the laboratory models of chronic drug use. Pretreatment with amphetamine in RLA rats did not result in behavioral sensitization but induced neuronal adaptations that may be related to the lack of this phenomenon. Moreover, RLA rats that experienced amphetamine for the first time showed activation of the central nucleus of the amygdala (CeA). Activation of the CeA was also seen in mice receiving ethanol and naltrexone, a drug used to prevent relapse in alcoholics. These findings suggest that the CeA may be a relevant brain structure in preventing drug addiction

Neural signatures of arbitration between Pavlovian and instrumental action selection.

Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations "misbehave" when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning.

Unavoidable stress elicits perceived lack of controllability and learned helplessness, which is a risk-factor for depression. Stress, anxiety, and depression can all alter reward learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning of optimal choices. Uncontrollable stress also predicted a greater proportion of probabilistic errors following negative feedback for correct choices, which reflected interpretation of ambiguous negative feedback as a change in task reward contingencies. Moreover, this effect of uncontrollable stress on probabilistic errors was mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression