Impairments in reinforcement learning do not explain enhanced habit formation in cocaine use disorder.

RATIONALE: Drug addiction has been suggested to develop through drug-induced changes in learning and memory processes. Whilst the initiation of drug use is typically goal-directed and hedonically motivated, over time, drug-taking may develop into a stimulus-driven habit, characterised by persistent use of the drug irrespective of the consequences. Converging lines of evidence suggest that stimulant drugs facilitate the transition of goal-directed into habitual drug-taking, but their contribution to goal-directed learning is less clear. Computational modelling may provide an elegant means for elucidating changes during instrumental learning that may explain enhanced habit formation. OBJECTIVES: We used formal reinforcement learning algorithms to deconstruct the process of appetitive instrumental learning and to explore potential associations between goal-directed and habitual actions in patients with cocaine use disorder (CUD). METHODS: We re-analysed appetitive instrumental learning data in 55 healthy control volunteers and 70 CUD patients by applying a reinforcement learning model within a hierarchical Bayesian framework. We used a regression model to determine the influence of learning parameters and variations in brain structure on subsequent habit formation. RESULTS: Poor instrumental learning performance in CUD patients was largely determined by difficulties with learning from feedback, as reflected by a significantly reduced learning rate. Subsequent formation of habitual response patterns was partly explained by group status and individual variation in reinforcement sensitivity. White matter integrity within goal-directed networks was only associated with performance parameters in controls but not in CUD patients. CONCLUSIONS: Our data indicate that impairments in reinforcement learning are insufficient to account for enhanced habitual responding in CUD.This research was funded by the Medical Research Council (MR/J012084/1) and the NIHR Cambridge Biomedical Research Centre and was conducted at the NIHR Cambridge Clinical Research Facility. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. This research was also supported in part by a Medical Research Council (MRC) Clinical Research Infrastructure award (MR/M009041/1). R.N.C. consults for Campden Instruments and receives royalties from Cambridge Enterprise, Routledge, and Cambridge University Press. RNC’s research is supported by the UK Medical Research Council (MC_PC_17213). T.W.R. discloses consultancy with Cambridge Cognition, Lundbeck, Mundipharma and Unilever; he receives royalties for CANTAB from Cambridge Cognition and editorial honoraria from Springer Verlag and Elsevier. T.V.L., G.S. P.S.J., A.A.M. and K.D.E. declare to have no potential conflict of interest

Impairments in reinforcement learning do not explain enhanced habit formation in cocaine use disorder

Rationale Drug addiction has been suggested to develop through drug-induced changes in learning and memory processes. Whilst the initiation of drug use is typically goal-directed and hedonically motivated, over time, drug-taking may develop into a stimulus-driven habit, characterised by persistent use of the drug irrespective of the consequences. Converging lines of evidence suggest that stimulant drugs facilitate the transition of goal-directed into habitual drug-taking, but their contribution to goal-directed learning is less clear. Computational modelling may provide an elegant means for elucidating changes during instrumental learning that may explain enhanced habit formation. Objectives We used formal reinforcement learning algorithms to deconstruct the process of appetitive instrumental learning and to explore potential associations between goal-directed and habitual actions in patients with cocaine use disorder (CUD). Methods We re-analysed appetitive instrumental learning data in 55 healthy control volunteers and 70 CUD patients by applying a reinforcement learning model within a hierarchical Bayesian framework. We used a regression model to determine the influence of learning parameters and variations in brain structure on subsequent habit formation. Results Poor instrumental learning performance in CUD patients was largely determined by difficulties with learning from feedback, as reflected by a significantly reduced learning rate. Subsequent formation of habitual response patterns was partly explained by group status and individual variation in reinforcement sensitivity. White matter integrity within goal-directed networks was only associated with performance parameters in controls but not in CUD patients. Conclusions Our data indicate that impairments in reinforcement learning are insufficient to account for enhanced habitual responding in CUD

Impairments in reinforcement learning do not explain enhanced habit formation in cocaine use disorder

Abstract: Rationale: Drug addiction has been suggested to develop through drug-induced changes in learning and memory processes. Whilst the initiation of drug use is typically goal-directed and hedonically motivated, over time, drug-taking may develop into a stimulus-driven habit, characterised by persistent use of the drug irrespective of the consequences. Converging lines of evidence suggest that stimulant drugs facilitate the transition of goal-directed into habitual drug-taking, but their contribution to goal-directed learning is less clear. Computational modelling may provide an elegant means for elucidating changes during instrumental learning that may explain enhanced habit formation. Objectives: We used formal reinforcement learning algorithms to deconstruct the process of appetitive instrumental learning and to explore potential associations between goal-directed and habitual actions in patients with cocaine use disorder (CUD). Methods: We re-analysed appetitive instrumental learning data in 55 healthy control volunteers and 70 CUD patients by applying a reinforcement learning model within a hierarchical Bayesian framework. We used a regression model to determine the influence of learning parameters and variations in brain structure on subsequent habit formation. Results: Poor instrumental learning performance in CUD patients was largely determined by difficulties with learning from feedback, as reflected by a significantly reduced learning rate. Subsequent formation of habitual response patterns was partly explained by group status and individual variation in reinforcement sensitivity. White matter integrity within goal-directed networks was only associated with performance parameters in controls but not in CUD patients. Conclusions: Our data indicate that impairments in reinforcement learning are insufficient to account for enhanced habitual responding in CUD

Functional Organization of the Human Brain: How We See, Feel, and Decide.

The human brain is responsible for constructing how we perceive, think, and act in the world around us. The organization of these functions is intricately distributed throughout the brain. Here, I discuss how functional magnetic resonance imaging (fMRI) was employed to understand three broad questions: how do we see, feel, and decide? First, high-resolution fMRI was used to measure the polar angle representation of saccadic eye movements in the superior colliculus. We found that eye movements along the superior-inferior visual field are mapped across the medial-lateral anatomy of a subcortical midbrain structure, the superior colliculus (SC). This result is consistent with the topography in monkey SC. Second, we measured the empathic responses of the brain as people watched a hand get painfully stabbed with a needle. We found that if the hand was labeled as belonging to the same religion as the observer, the empathic neural response was heightened, creating a strong ingroup bias that could not be readily manipulated. Third, we measured brain activity in individuals as they made free decisions (i.e., choosing randomly which of two buttons to press) and found the activity within fronto-thalamic networks to be significantly decreased compared to being instructed (forced) to press a particular button. I also summarize findings from several other projects ranging from addiction therapies to decoding visual imagination to how corporations are represented as people. Together, these approaches illustrate how functional neuroimaging can be used to understand the organization of the human brain

Brain MRI correlates of depression and vascular risk: Whitehall Imaging sub-study

This thesis combines neuroimaging and epidemiological techniques to investigate the hypothesis that late-life depressive symptoms are partially caused by vascular risk factors. Magnetic resonance imaging (MRI) was used to study the structural brain changes associated with depressive symptoms, major depressive disorder and long-term exposure to vascular risk factors (hypertension, dyslipidaemia, diabetes, smoking and Framingham stroke risk). This was complemented by an epidemiological approach to investigate whether vascular risk factors are associated with depressive symptoms. A sample of participants from the Whitehall II study were invited to take part in the Whitehall Imaging sub-study at the University of Oxford. Participants recruited between April 2012 and June 2013 (n=229, mean age 69, age range 60-82 years, 83% male) underwent detailed cognitive testing, a clinical interview and a multi-modal 3 Tesla MRI brain scan. Depressive symptoms were measured at previous Whitehall II phases, and again in 2012-2013 using a structured assessment for DSM-IV mood disorder and a self-report questionnaire. Long-term exposure to vascular risk factors was measured at five collection phases between 1985 and 2009. Ten percent of participants (n=23) had current depressive symptoms and 13% (n=29) had late-onset depressive symptoms (depression onset after age 60). Current and late-onset depressive symptoms were associated with reduced white matter integrity in frontal-subcortical areas. Study of the MRI correlates of vascular risk factors also showed an association between long-term exposure to high fasting glucose (mean across five examinations between 1985 and 2009) and reduced white matter integrity in frontal-subcortical areas. However, long-term exposure to other vascular risk factors was not significantly associated with depressive symptoms. In conclusion, while vascular risk factors were not consistently related to late-life depressive symptoms, long-term exposure to high glucose levels and depressive symptoms were both associated with reduced white matter integrity in frontal-subcortical areas

Developmental neurocognitive pathway of psychosis proneness and the impact of cannabis use

Cette thèse fait la promotion d’une nouvelle approche ciblant le risque de psychose qui consiste à identifier les enfants et les jeunes adolescents de la communauté appartenant à différentes trajectoires développementales d’expériences psychotiques. Une identification très précoce du risque de psychose chez des jeunes de la communauté pourrait ainsi diminuer la période où les symptômes cliniques ne sont pas traités, mais aurait également le potentiel de prévenir efficacement l’émergence de symptômes avérés, et ce, si les facteurs de risque sont identifiés. Étant donné que la consommation de cannabis s’avère un important facteur de risque de la psychose et le contexte actuel où les états en sont à réviser leurs politiques de réglementation du cannabis, il s’avère primordial de mieux comprendre comment la consommation peut mener à la psychose chez les individus vulnérables. Tout d’abord, j’ai investigué la séquence temporelle entre la consommation de cannabis et les expériences psychotiques chez une population de 4000 adolescents, suivis pendant 4 ans, au moment de l’adolescence où les deux phénomènes s’initient. Ensuite, j’ai examiné, chez des adolescents suivant une trajectoire de vulnérabilité, le rôle d’un moins bon fonctionnement cognitif ainsi que celui d’une exacerbation des symptômes anxieux et dépressifs comme médiateurs du lien entre cannabis et risque de psychose. Enfin, j’ai investigué la présence de marqueurs neurocognitifs précoces (fonctionnels et structurels) qui seraient associés à l’émergence de symptômes psychotiques chez des adolescents, et exploré si la consommation de cannabis pourrait modérer l’ampleur de ces marqueurs. Les données proviennent de deux cohortes longitudinales suivant des adolescents de la population générale, l’étude Co-Venture (n=4000, âgés de 12 ans, suivis annuellement pendant 4 ans) et l’étude de neuroimagerie IMAGEN (n=2200, âgés de 14 ans, suivis pendant 2 ans), ainsi qu’un sous-échantillon de l’étude Co-Venture ayant complété des mesures de neuroimagerie (n=151, âgés de 12 ans, suivis annuellement pendant 4 ans). Les résultats ont montré que la consommation de cannabis précédait systématiquement l’augmentation des expériences psychotiques, et non l’inverse. Chez les jeunes suivant une trajectoire de vulnérabilité, la relation entre la consommation de cannabis et le risque de psychose était davantage expliquée par une augmentation des symptômes de dépression et d’anxiété. Une réduction du volume de l’hippocampe et de l’amygdale en combinaison avec une hyperactivité de ces mêmes régions en réponse à des expressions neutres étaient tous associés à l’émergence de symptômes psychotiques. Or, la consommation de cannabis n’a pas exacerbé les altérations structurelles observées chez les adolescents rapportant des expériences psychotiques. Ces résultats ont mis en évidence le rôle primordial d’un hyperfonctionnement du système limbique pouvant mener à l’attribution aberrante d’une importance émotionnelle aux stimuli de l’environnement, et ce, chez des adolescents vulnérables. Il semble que le mécanisme par lequel la consommation de cannabis mène à l’émergence de symptômes cliniques passe par son influence sur les symptômes de dépression et d’anxiété ainsi que leurs mécanismes neuronaux sous-jacents d’une hypersensibilité au stress. Enfin, de par ces résultats, cette thèse permet de contribuer au développement de nouvelles interventions visant à réduire la demande de cannabis chez des adolescents vulnérables.Following the worldwide initiative on intervening early in clinical high-risk individuals for psychosis, this thesis promotes a novel approach to identify those at risk for psychosis by studying children and adolescents from the community who report different trajectories of subclinical psychosis symptoms (i.e., psychotic-like experiences) without the confounds of iatrogenic effects such as major social and cognitive impairments. Early identification from this approach may not only reduce harm by shortening the duration of untreated symptoms, but may also have the capacity to prevent the emergence of clinically validated symptoms, particularly if early risk factors can be identified. Considering the long-standing notion that cannabis misuse is an important risk factor for psychosis and that jurisdictions around the world are currently revising their cannabis regulatory policies, there is a need to better understand how cannabis use may lead to psychosis in vulnerable youths. This thesis examined different mechanisms that may explain the complex relationship between cannabis use and psychosis risk. I first explored the temporal sequence between cannabis use and self-reported psychotic-like experiences in a population-based sample of 4000 adolescents, over a 4-year period when both phenomena have their onset. Second, in vulnerable youths, I investigated the role of impaired cognitive functioning as well as increased affective and anxious symptoms as mediators of the cannabis-to-psychosis relationship. And third, I explored the presence of early neurocognitive markers (both functional and structural) associated with the emergence of psychotic symptoms, and how cannabis use moderates these markers. Two longitudinal cohorts from the general population, the Co-Venture Study (n=4000, aged 12 years old, followed annually for 4 years) and the neuroimaging IMAGEN Study (n=2200, aged 14 years old, followed for 2 years), as well as the neuroimaging subsample from the Co-Venture Study (n=151, aged 12 years old, followed annually for 4 years) were used. It was found that an increase in cannabis use always preceded an increase in reported psychotic-like experiences throughout adolescence, but an increase in psychotic-like experiences rarely predicted an increase in cannabis use. Then, in vulnerable adolescents, the cannabis-to-psychosis risk relationship was better explained by increases in depression and anxiety symptoms relative to changes in cognitive functioning. It was demonstrated that reduced hippocampus and amygdala volumes, combined with hyperactivity of the same regions during neutral cues processing were associated with the emergence of psychotic symptoms in young adolescents reporting psychotic-like experiences. However, cannabis use did not exacerbate the structural alterations observed in youths with psychotic-like experiences. These findings have improved our understanding of the relationship between cannabis use and vulnerability to psychosis. They have also highlighted the important role of an impaired limbic network leading to an aberrant emotional salience attribution in vulnerable adolescents. Although cannabis use did not exacerbate brain structural alterations observed in vulnerable youths, it appears that cannabis will more likely interfere with depression and/or anxiety symptoms and their associated brain mechanisms underlying vulnerability to stress in the path towards psychosis risk. This thesis may inform the development of new evidence-based interventions that reduce demand for cannabis among vulnerable youths

Tinnitus and Decreased Subcortical and Cortical Inhibition

Background: The perception of tinnitus may be triggered by a reduction in inhibitory function in the central auditory nervous system. Evidence, primarily from invasive studies of animal models of tinnitus, indicates that these changes occur at both the subcortical and cortical level. Auditory evoked potential (AEP) indices of subcortical inhibition [auditory brainstem response (ABR) 〖 V/I 〗_ amp ratio ] and cortical inhibition [cortical auditory evoked potential (CAEP) sensory gating ratios] may provide an objective index of whether reduced subcortical and/or cortical inhibition is associated with tinnitus perception in humans. The aims of this study were to assess whether ABR and/or CAEP indices of subcortical and cortical inhibition distinguish between a group with constant tinnitus and matched non-tinnitus controls, and whether tinnitus presence and/or other factors [age, noise exposure history, hearing loss, speech perception in noise (SPIN)] predicted ABR and/or CAEP outcomes related to inhibition. Methods: Individuals with tinnitus and control counterparts matched for sex, age, and hearing thresholds completed the study (n = 18 per group). ABRs were recorded with a tiptrode in response to high intensity click ABRs to determine the 〖 V/I 〗_ amp ratio . CAEPs were recorded in response to two successive high intensity 10 ms clicks. A ratio of the amplitude or area of the first (conditioning CAEP) and second (test CAEP) click response was determined ( test CAEP / conditioning CAEP ) as the primary measure of sensory gating. The latency ratio was also determined as a secondary outcome which may relate to sensory gating. For both the ABR 〖 V/I 〗_ amp ratio and CAEP sensory gating ratios, a larger value indicated reduced inhibition. Ratios were compared between the two groups using independent t-tests. The relative predictive value (proportional reduction in error, PRE) of tinnitus, age, noise exposure history, hearing loss, and SPIN on ABR and CAEP outcome variables related to inhibition was analyzed using regression. Results: Individuals with tinnitus, relative to controls, exhibited similar ABR 〖 V/I 〗_ amp ratio , and significantly larger sensory gating 〖 P1 〗_ lat ratio . None of the variables assessed significantly predicted the ABR 〖 V/I 〗_ amp ratio . Tinnitus significantly predicted 〖 P1-N1 〗_ amp ratio , but not when taking into account age, noise exposure history, hearing loss, and SPIN. The 〖 P1 〗_ lat ratio was significantly predicted by both tinnitus and age, however, best predicted by age. Conclusions: Tinnitus-related reduced inhibition was not evident at the subcortical level based on the ABR 〖 V/I 〗_ amp ratio . At the cortical level, the predictive influence of tinnitus on the 〖 P1-N1 〗_ amp ratio supports the association between reduced sensory gating with tinnitus presence in humans. The significantly larger 〖 P1 〗_ lat ratio in the tinnitus group may also support reduced sensory gating and/or a change in the recovery time, or refractoriness, of auditory evoked responses in individuals with tinnitus. The strong predictive influence of age on the 〖 P1 〗_ lat ratio indicates that increasing age reduced sensory gating above and beyond the effects of tinnitus. Potential limitations to the current study, including the non-normally distributed participant characteristics and AEP methodologies, as well as considerations for future research aiming to improve the reliability and validity of tinnitus AEP assessments are discussed

Functional organisation of behavioural inhibitory control mechanisms in cortico-basal ganglia circuitry: implications for stimulant use disorder.

The neural and psychological mechanisms of inhibitory control processes were investigated, focusing on the cortico-basal ganglia circuits in rats and humans. These included behavioural flexibility, ‘waiting’ and ‘stopping’ impulsivity and involved serial spatial reversal learning task in rodents, and in humans, premature responses in the Monetary Incentive Delay (MID) task and the stop-signal reaction time task. Chapter 2 and Chapter 3 focus on individual differences in behavioural flexibility in rats while Chapter 4, Chapter 5 and Chapter 6 consider how inhibitory control mechanisms are affected by the psychostimulant drug cocaine in both rats and humans. As reported in Chapter 2, systemic modulation of monoaminergic transmission by monoamine oxidase A (MAO-A) inhibitors enhanced reversal learning performance, selectively by decreasing the lose-shift probability, thereby implicating a role for dopamine, serotonin and noradrenaline in facilitating learning from negative feedback. Resting state functional magnetic resonance imaging (fMRI) revealed enhanced functional connectivity of the orbitofrontal and motor cortices as a correlate of flexible reversal learning performance, consistent with elevated levels of monoamines in these region (Chapter 3). Having clarified the mechanisms underlying behavioural flexibility in rats, Chapter 4 reports that escalation of intravenous cocaine self-administration induces behavioural inflexibility in rats even after a relatively short period of cocaine intake. Computational models, including a reinforced and Bayesian learner, revealed a lack of exploitation of the learned response-outcome relationships in cocaine-exposed rats. Chapter 5 focused on impulse control in human volunteers, identifying the striatal and cingulo-opercular networks as substrates of impulsive, premature responding in healthy 4 volunteers, stimulant-dependent individuals and their unaffected siblings. Loss of impulse control was elicited by different incentives for drug-free participants as opposed to drug users. Drug cues elicited striatal activation and increased premature responses in the stimulant-dependent group compared with the control group. In contrast, the ventral striatum was linked to incentive specific activation to reward anticipation. Task-based fMRI demonstrated that interactions between dorsal striatum and cingulo-opercular “cold cognition” networks underlie failures of impulse control in the control, at-risk and stimulant-dependent groups. However, whereas the cingulo-opercular networks were associated with premature responding in all groups, the reward system was activated specifically by the drug incentive cues in the stimulant group, and by monetary incentive cues in the drug-free groups. Chapter 6 presents evidence that corticostriatal functional and effective connectivity in an overlapping network that includes the anterior cingulate and inferior frontal cortices as well as motor cortex, the subthalamic nucleus and dorsal striatum, is critical to stopping impulse control in both control and cocaine individuals. No stopping efficiency impairments were observed in the cocaine-dependent group. Nevertheless, lower structural corticostriatal connectivity measured using diffusion MRI was associated with response execution impairments in cocaine participants performing a stop-signal reaction time task. Further, response execution was rescued by the selective noradrenaline reuptake inhibitor atomoxetine, which also increased corticostriatal effective connectivity. Finally, increased impulsivity and behavioural inflexibility seen in stimulant use disorder in Chapter 5 and Chapter 4, respectively, were not observed in the endophenotype at risk for developing stimulant abuse but were rather a consequence of stimulant abuse. These results further clarify the monoaminergic substrates of behavioural flexibility and specify the neural and computational impairments in inhibitory control induced by stimulant dependence.Pinsent Darwin Studentship from the Dept of Physiology, Development and Neuroscienc