Inter-Individual Decision-Making Differences in the Effects of Cingulate, Orbitofrontal, and Prelimbic Cortex Lesions in a Rat Gambling Task

Deficits in decision-making is a hallmark of several neuropsychiatric pathologies but is also observed in some healthy individuals that could be at risk to develop these pathologies. Poor decision-making can be revealed experimentally in humans using the Iowa gambling task, through the inability to select options that ensure long term gains over larger immediate gratification. We devised an analogous task in the rat, based on uncertainty and conflicting choices, the rat gambling task (RGT). It similarly reveals good and poor performers within a single session. Using this task, we investigated the role of three prefrontal cortical areas, the orbitofrontal, prelimbic, and cingulate cortices on decision-making, taking into account inter-individual variability in behavioral performances. Here, we show that these three distinct subregions are differentially engaged to solve the RGT. Cingulate cortex lesion mainly delayed good decision-making whereas prelimbic and orbitofrontal cortices induced different patterns of inadapted behaviors in the task, indicating varying degree of functional specialization of these three areas. Their contribution largely depended on the level of adaptability demonstrated by each individual to the constraint of the task. The inter-individual differences in the effect of prefrontal cortex area lesions on decision-making revealed in this study open new perspectives in the search for vulnerability markers to develop disorders related to executive dysfunctioning

Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats

International audienceGoal directed action involves a representation of the consequences of an action. Rats with lesions of the medial prefrontal cortex do not adapt their instrumental response in a Skinner box when food delivery becomes unrelated to lever pressing. This indicates a role for the prefrontal region in adapting to contingency changes, a form of causal learning. We attempted to model this phenomenon in a reinforcement learning framework. Behavioural sequences of normal and lesioned rats were used to feed models based on the SARSA algorithm. One model (factorized-states) focused on temporal factors, representing continuous states as vectors of decaying event traces. The second model (event sequence) emphasized sequences, representing states as n-uplets of events. The values of state-action pairs were incorporated into a softmax policy to derive predicted action probabilities and adjust model parameters. Both models revealed a number of discrepancies between predicted and actual behaviour, emphasising changes in magazine visits rather that lever presses. The models also did not reproduce the differential adaptation of normal and prefrontal lesioned rats to contingency degradation. These data suggest that temporal difference learning models fail to capture causal relationships involved in the adaptation to contingency changes

Coordination des actions et des habitudes (approche neurocomportementale chez le rat)

: Mon travail de thèse a porté sur l étude neurocomportementale des actions et des habitudes chez le Rat. En effet, lors d un apprentissage opérant la réponse peut être acquise selon deux systèmes associatifs. Le premier dépend d une représentation des conséquences de la réponse, le second d une association plus simple entre le stimulus et la réponse. Un premier axe de recherche a consisté à étudier plusieurs paramètres du conditionnement, afin de déterminer leur influence sur le contrôle de la réponse instrumentale par un système plutôt que l autre. Le deuxième axe de recherche a porté sur l étude des substrats neuronaux impliqués dans l acquisition et l expression d une action, par l intermédiaire de techniques d inactivation cérébrale et d étude immuno-histochimique de l expression génique de la protéine Fos.Previous research has established that instrumental conditioning, in both primates and rats, is mediated by two concurrent associative systems. In early stages of training, instrumental response is thought to be mediated by an association between the action and the outcome (A-O). While training proceeds however, as the response becomes less sensible to the outcome value, it is conceived as being mediated by an association between thestimulus and the response (S-R). Recent evidences suggest that the both systems operate in tandem and/or competition from the beginning of training. This work aimed at studying the mechanisms that coordinate the control of the instrumental response by the goal-directedsystem or the habit system. A first batch of results indicates no effect of the amount of training sessions on the goal-directed nature of the conditioned instrumental response. Indeed,the outcome devaluations by CTA or selective satiety reduced the instrumental performances,independently of the training procedure applied. The instrumental responses resulting from our 3 training procedures depend of an actualized representation of their outcomes. A secondbatch of results indicates that information about the context of instrumental conditioning isincluded in the incentive representation of the outcome. Indeed, we observed no sensitivity tooutcome devaluation when devaluation occurred outside the training context. These results offer new original hypotheses about context encoding and the nature of instrumental responding. A third batch of experiments investigates the role of the prelimbic cortex in acquisition vs. expression of goal-directed instrumental behaviour, using reversible neuronal inactivation. The results show that the prelimbic cortex plays a transient but crucial role in theacquisition of goal-directed responding and that the A-O and S-R systems can operate in a competitive fashion early in training. Using ex-vivo imaging, a last batch of experiments aimed to study the temporal cerebral activation throughout instrumental training with a focuson prefrontal and striatal regions. Results show levels of Fos expression that vary with regions. At the beginning of conditioning, the density of Fos positive nuclei is high in the prefrontal regions. It decreases with training. Labelling is denser in the dorsomedial striatumthan in the dorsolateral striatum. The weak activation in the dorsolateral striatum appears consistent with the absence of habit. These data are in accordance with data of the literature concerning dynamics of activation in cortico-striatal circuits. Furthermore, they are in agreement with the suggestion that activity in the prelimbic cortex could promote the acquisition of goal-directed action by the induction of neuronal plasticity in the dorsal striatum.BORDEAUX1-Bib.electronique (335229901) / SudocSudocFranceF

Acquisition of specific response-outcome associations requires NMDA receptor activation in the basolateral amygdala but not in the insular cortex

The basolateral amygdala (BLA) and the gustatory region of the insular cortex (IC) are required for the encoding and retrieval of outcome value. Here, we examined if these regions are also necessary to learn associations between actions and their outcomes. Hungry rats were first trained to press two levers for a common outcome. Next, specific response-outcome (R-O) associations were introduced such that each response now earned a distinct food outcome. Prior to each specific R-O training session, rats received a bilateral infusion of the N-methyl-d-aspartate (NMDA) receptor antagonist, DL-APV, into either the BLA or the IC. One of the two outcomes was then devalued immediately prior to a choice test. Inhibition of NMDA receptor activity in the BLA, but not the IC, during the acquisition of specific R-O associations abolished selective devaluation. These results indicate that the BLA is critical for learning the association between actions and their specific consequences

A reinforcement learning approach to instrumental contingency degradation in rats

International audienceGoal-directed action involves a representation of action consequences. Adapting to changes in action-outcome contingency requires the prefrontal region. Indeed, rats with lesions of the medial prefrontal cortex do not adapt their free operant response when food delivery becomes unrelated to lever-pressing. The present study explores the bases of this deficit through a combined behavioural and computational approach. We show that lesioned rats retain some behavioural flexibility and stop pressing if this action prevents food delivery. We attempt to model this phenomenon in a reinforcement learning framework. The model assumes that distinct action values are learned in an incremental manner in distinct states. The model represents states as n-uplets of events, emphasizing sequences rather than the continuous passage of time. Probabilities of lever-pressing and visits to the food magazine observed in the behavioural experiments are first analyzed as a function of these states, to identify sequences of events that influence action choice. Observed action probabilities appear to be essentially function of the last event that occurred, with reward delivery and waiting significantly facilitating magazine visits and lever-pressing respectively. Behavioural sequences of normal and lesioned rats are then fed into the model, action values are updated at each event transition according to the SARSA algorithm, and predicted action probabilities are derived through a softmax policy. The model captures the time course of learning, as well as the differential adaptation of normal and prefrontal lesioned rats to contingency degradation with the same parameters for both groups. The results suggest that simple temporal difference algorithms with low learning rates can largely account for instrumental learning and performance. Prefrontal lesioned rats appear to mainly differ from control rats in their low rates of visits to the magazine after a lever press, and their inability to initially detect weak contingency changes

Dopamine and noradrenaline modulation of goal-directed behaviour in prefrontal areas: Toward a division of labour?

The prefrontal cortex is considered to be at the core of goal-directed behaviours. Notably, the medial prefrontal cortex (mPFC) is known to play an important role in learning action-outcome associations, as well as in detecting changes in this contingency. Previous studies have also highlighted a specific engagement of the dopaminergic pathway innervating the mPFC in adapting to changes in action causality. While previous research on goal-directed actions has primarily focused on the mPFC region, recent findings have revealed a distinct and specific role of the ventral and lateral orbitofrontal cortex (vlOFC). Indeed, vlOFC is not necessary to learn about action-outcome associations but appears specifically involved when outcome identity is unexpectedly changed. Unlike the mPFC, the vlOFC does not receive a strong dopaminergic innervation. However, it receives a dense noradrenergic innervation which might indicate a crucial role for this neuromodulator. In addition, several lines of evidence highlight a role for noradrenaline in adapting to changes in the environment. We therefore propose that the vlOFC’s function in action control might be under the strong influence of the noradrenergic system. In the present paper, we review anatomical and functional evidence consistent with this proposal and suggest a direction for future studies that aims to shed light on the orbitofrontal mechanisms for flexible action control. Specifically, we suggest that dopaminergic modulation in the mPFC and noradrenergic modulation in the vlOFC may underlie distinct processes related to updating one’s actions

Distinct regional patterns in noradrenergic innervation of the rat prefrontal cortex

International audienc

The Role of the Rat Medial Prefrontal Cortex in Adapting to Changes in Instrumental Contingency

In order to select actions appropriate to current needs, a subject must identify relationships between actions and events. Control over the environment is determined by the degree to which action consequences can be predicted, as described by action-outcome contingencies – i.e. performing an action should affect the probability of the outcome. We evaluated in a first experiment adaptation to contingency changes in rats with neurotoxic lesions of the medial prefrontal cortex. Results indicate that this brain region is not critical to adjust instrumental responding to a negative contingency where the rats must refrain from pressing a lever, as this action prevents reward delivery. By contrast, this brain region is required to reduce responding in a non-contingent situation where the same number of rewards is freely delivered and actions do not affect the outcome any more. In a second experiment, we determined that this effect does not result from a different perception of temporal relationships between actions and outcomes since lesioned rats adapted normally to gradually increasing delays in reward delivery. These data indicate that the medial prefrontal cortex is not directly involved in evaluating the correlation between action-and reward-rates or in the perception of reward delays. The deficit in lesioned rats appears to consist of an abnormal response to the balance between contingent and non-contingent rewards. By highlighting the role of prefrontal regions in adapting to the causal status of actions, these data contribute to our understanding of the neura

Long-lasting deficits in hedonic and nucleus accumbens reactivity to sweet rewards by sugar overconsumption during adolescence

Adolescence is a critical period characterized by major neurobiological changes. Chronic stimulation of the reward system might constitute an important factor in vulnerability to pathological development. In spite of the dramatic increase in the consumption of sweet palatable foods during adolescence in our modern societies, the long-term consequences of such exposure on brain reward processing remain poorly understood. Here, we investigated in rats the long-lasting effects of sugar overconsumption during their adolescence on their adult reactivity to the hedonic properties of sweet rewards. Adolescent rats with continuous access to 5% sucrose solution (from postnatal day 30-46) showed escalating intake. At adulthood (post-natal day 70), using two-bottle free choice tests, sucrose-exposed rats showed lower intake than non-exposed rats suggesting decreased sensitivity to the rewarding properties of sucrose. In Experiment 1, we tested their hedonic-related orofacial reactions to intraoral infusion of tasty solutions. We showed that sucrose-exposed rats presented less hedonic reactions in response to sweet tastes leaving the reactivity to water or quinine unaltered. Hence, in Experiment 2, we observed that this hedonic deficit is associated with lower c-Fos expression levels in the nucleus accumbens, a brain region known to play a central role in hedonic processing. These findings demonstrate that a history of high sucrose intake during the critical period of adolescence induces long-lasting deficits in hedonic treatment that may contribute to reward-related disorders

Excitotoxic lesions of the entorhinal cortex leave gustatory within-event learning intact

The ability of rats with ibotenate lesions of the entorhinal cortex to form memories for events was assessed by using a gustatory within-event learning procedure. Rats first received exposure to 2 events, AX and BY, each composed of a pair of flavors. Following this exposure period, Flavor X alone was paired with the delivery of lithium chloride. Lesioned and control rats showed a greater aversion to A than to B and to AX than to BX. These results challenge theories that suppose that the entorhinal cortex plays a general role in forming representations of patterns of stimulation