Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2/3 receptor agents

Abstract: Rationale: Disorders of compulsivity such as stimulant use disorder (SUD) and obsessive-compulsive disorder (OCD) are characterised by deficits in behavioural flexibility, some of which have been captured using probabilistic reversal learning (PRL) paradigms. Objectives: This study used computational modelling to characterise the reinforcement learning processes underlying patterns of PRL behaviour observed in SUD and OCD and to show how the dopamine D2/3 receptor agonist pramipexole and the D2/3 antagonist amisulpride affected these responses. Methods: We applied a hierarchical Bayesian method to PRL data across three groups: individuals with SUD, OCD, and healthy controls. Participants completed three sessions where they received placebo, pramipexole, and amisulpride, in a double-blind placebo-controlled, randomised design. We compared seven models using a bridge sampling estimate of the marginal likelihood. Results: Stimulus-bound perseveration, a measure of the degree to which participants responded to the same stimulus as before irrespective of outcome, was significantly increased in SUD, but decreased in OCD, compared to controls (on placebo). Individuals with SUD also exhibited reduced reward-driven learning, whilst both the SUD and OCD groups showed increased learning from punishment (nonreward). Pramipexole and amisulpride had similar effects on the control and OCD groups; both increased punishment-driven learning. These D2/3-modulating drugs affected the SUD group differently, remediating reward-driven learning and reducing aspects of perseverative behaviour, amongst other effects. Conclusions: We provide a parsimonious computational account of how perseverative tendencies and reward- and punishment-driven learning differentially contribute to PRL in SUD and OCD. D2/3 agents modulated these processes and remediated deficits in SUD in particular, which may inform therapeutic effects

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders.

Background: Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior. Aims: It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity. Method: We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data. Results: We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls. Conclusions: Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification

Perseveration and Shifting in Obsessive-Compulsive Disorder as a Function of Uncertainty, Punishment, and Serotonergic Medication

© 2023 The Author(s). Published by Elsevier Inc on behalf of the Society of Biological Psychiatry. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY), https://creativecommons.org/licenses/by/4.0/Background The nature of cognitive flexibility deficits in obsessive-compulsive disorder (OCD), which historically have been tested with probabilistic reversal learning tasks, remains elusive. Here, a novel deterministic reversal task and inclusion of unmedicated patients in the study sample illuminated the role of fixed versus uncertain rules/contingencies and of serotonergic medication. Additionally, our understanding of probabilistic reversal was enhanced through theoretical computational modeling of cognitive flexibility in OCD. Methods We recruited 49 patients with OCD, 21 of whom were unmedicated, and 43 healthy control participants matched for age, IQ, and gender. Participants were tested on 2 tasks: a novel visuomotor deterministic reversal learning task with 3 reversals (feedback rewarding/punishing/neutral) measuring accuracy/perseveration and a 2-choice visual probabilistic reversal learning task with uncertain feedback and a single reversal measuring win-stay and lose-shift. Bayesian computational modeling provided measures of learning rate, reinforcement sensitivity, and stimulus stickiness. Results Unmedicated patients with OCD were impaired on the deterministic reversal task under punishment only at the first and third reversals compared with both control participants and medicated patients with OCD, who had no deficit. Perseverative errors were correlated with OCD severity. On the probabilistic reversal task, unmedicated patients were only impaired at reversal, whereas medicated patients were impaired at both the learning and reversal stages. Computational modeling showed that the overall change was reduced feedback sensitivity in both OCD groups. Conclusions Both perseveration and increased shifting can be observed in OCD, depending on test conditions including the predictability of reinforcement. Perseveration was related to clinical severity and remediated by serotonergic medication.Peer reviewe

Serotonin depletion amplifies distinct human social emotions as a function of individual differences in personality

Funder: Gates Cambridge Trust; doi: https://doi.org/10.13039/501100005370Abstract: Serotonin is involved in a wide range of mental capacities essential for navigating the social world, including emotion and impulse control. Much recent work on serotonin and social functioning has focused on decision-making. Here we investigated the influence of serotonin on human emotional reactions to social conflict. We used a novel computerised task that required mentally simulating social situations involving unjust harm and found that depleting the serotonin precursor tryptophan—in a double-blind randomised placebo-controlled design—enhanced emotional responses to the scenarios in a large sample of healthy volunteers (n = 73), and interacted with individual differences in trait personality to produce distinctive human emotions. Whereas guilt was preferentially elevated in highly empathic participants, annoyance was potentiated in those high in trait psychopathy, with medium to large effect sizes. Our findings show how individual differences in personality, when combined with fluctuations of serotonin, may produce diverse emotional phenotypes. This has implications for understanding vulnerability to psychopathology, determining who may be more sensitive to serotonin-modulating treatments, and casts new light on the functions of serotonin in emotional processing

Serotonin depletion impairs both Pavlovian and instrumental reversal learning in healthy humans.

Funder: Gates Cambridge Trust; doi: https://doi.org/10.13039/501100005370Funder: DH | National Institute for Health Research (NIHR); doi: https://doi.org/10.13039/501100000272Serotonin is involved in updating responses to changing environmental circumstances. Optimising behaviour to maximise reward and minimise punishment may require shifting strategies upon encountering new situations. Likewise, autonomic responses to threats are critical for survival yet must be modified as danger shifts from one source to another. Whilst numerous psychiatric disorders are characterised by behavioural and autonomic inflexibility, few studies have examined the contribution of serotonin in humans. We modelled both processes, respectively, in two independent experiments (N = 97). Experiment 1 assessed instrumental (stimulus-response-outcome) reversal learning whereby individuals learned through trial and error which action was most optimal for obtaining reward or avoiding punishment initially, and the contingencies subsequently reversed serially. Experiment 2 examined Pavlovian (stimulus-outcome) reversal learning assessed by the skin conductance response: one innately threatening stimulus predicted receipt of an uncomfortable electric shock and another did not; these contingencies swapped in a reversal phase. Upon depleting the serotonin precursor tryptophan-in a double-blind randomised placebo-controlled design-healthy volunteers showed impairments in updating both actions and autonomic responses to reflect changing contingencies. Reversal deficits in each domain, furthermore, were correlated with the extent of tryptophan depletion. Initial Pavlovian conditioning, moreover, which involved innately threatening stimuli, was potentiated by depletion. These results translate findings in experimental animals to humans and have implications for the neurochemical basis of cognitive inflexibility.NIH

Association between childhood trauma and risk for obesity: a putative neurocognitive developmental pathway

Funder: Zhangjiang LabFunder: the Shanghai AI Platform for Diagnosis and Treatment of Brain DiseasesFunder: the Project of Zhangjiang Hi-Tech District Management CommitteeFunder: Shanghai (grant 2016-17)Funder: BRIDGET (JPND: BRain Imaging, cognition Dementia and next generation GEnomics); Grant(s): MR/N027558/1Funder: Associazione Emma e Ernesto Rulfo per la Genetica Medica (IT)Funder: the Swedish Research Council FORMASFunder: the Medical Research CouncilFunder: the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College LondonFunder: the Human Brain Project (HBP SGA 2)Funder: the Fondation de France, the Fondation pour la Recherche MédicaleFunder: the Assistance-Publique-Hôpitaux-de-Paris and INSERM (interface grant)Funder: Paris Sud University IDEX 2012Funder: cross-NIH alliance that funds Big Data to Knowledge Centres of ExcellenceAbstract: Background: Childhood trauma increases the risk for adult obesity through multiple complex pathways, and the neural substrates are yet to be determined. Methods: Participants from three population-based neuroimaging cohorts, including the IMAGEN cohort, the UK Biobank (UKB), and the Human Connectome Project (HCP), were recruited. Voxel-based morphometry analysis of both childhood trauma and body mass index (BMI) was performed in the longitudinal IMAGEN cohort; validation of the findings was performed in the UKB. White-matter connectivity analysis was conducted to study the structural connectivity between the identified brain region and subdivisions of the hypothalamus in the HCP. Results: In IMAGEN, a smaller frontopolar cortex (FPC) was associated with both childhood abuse (CA) (β = − .568, 95%CI − .942 to − .194; p = .003) and higher BMI (β = − .086, 95%CI − .128 to − .043; p < .001) in male participants, and these findings were validated in UKB. Across seven data collection sites, a stronger negative CA-FPC association was correlated with a higher positive CA-BMI association (β = − 1.033, 95%CI − 1.762 to − .305; p = .015). Using 7-T diffusion tensor imaging data (n = 156), we found that FPC was the third most connected cortical area with the hypothalamus, especially the lateral hypothalamus. A smaller FPC at age 14 contributed to higher BMI at age 19 in those male participants with a history of CA, and the CA-FPC interaction enabled a model at age 14 to account for some future weight gain during a 5-year follow-up (variance explained 5.8%). Conclusions: The findings highlight that a malfunctioning, top-down cognitive or behavioral control system, independent of genetic predisposition, putatively contributes to excessive weight gain in a particularly vulnerable population, and may inform treatment approaches

Persistence of Amygdala–Hippocampal Connectivity and Multi-Voxel Correlation Structures During Awake Rest After Fear Learning Predicts Long-Term Expression of Fear

After encoding, memories undergo a process of consolidation that determines long-term retention. For conditioned fear, animal models postulate that consolidation involves reactivations of neuronal assemblies supporting fear learning during postlearning "offline" periods. However, no human studies to date have investigated such processes, particularly in relation to long-term expression of fear. We tested 24 participants using functional MRI on 2 consecutive days in a fear conditioning paradigm involving 1 habituation block, 2 acquisition blocks, and 2 extinction blocks on day 1, and 2 re-extinction blocks on day 2. Conditioning blocks were preceded and followed by 4.5-min rest blocks. Strength of spontaneous recovery of fear on day 2 served as a measure of long-term expression of fear. Amygdala connectivity primarily with hippocampus increased progressively during postacquisition and postextinction rest on day 1. Intraregional multi-voxel correlation structures within amygdala and hippocampus sampled during a block of differential fear conditioning furthermore persisted after fear learning. Critically, both these main findings were stronger in participants who exhibited spontaneous recovery 24 h later. Our findings indicate that neural circuits activated during fear conditioning exhibit persistent postlearning activity that may be functionally relevant in promoting consolidation of the fear memory

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans.

BACKGROUND: The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans. METHODS: Healthy volunteers received intravenous LSD (75 μg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history). RESULTS: Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase. CONCLUSIONS: Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.This study was funded by the Walacea.com crowdfunding campaign and the Beckley Foundation, awarded to R.L.C-H. J.W.K. was supported by a Gates Cambridge Scholarship and an Angharad Dodds John Bursary in Mental Health and Neuropsychiatry, T.W.R. by a Wellcome Trust Senior Investigator Grant 104631/Z/14/Z, and H.E.M.d.O. by the Netherlands Organisation for Scientific Research, NWO. R.N.C.’s research is funded by the UK Medical Research Council (MC_PC_17213, MR/W014386/1). Q.L. was partially sup- ported by grants from the National Key Research and Development Program of China (No. 2019YFA0709502), the National Natural Science Foundation of China (No. 81873909), the Science and Technology Commission of Shanghai Municipality (No.s 20ZR1404900 and 20DZ2260300), the Shanghai Municipal Science and Technology Major Project (No.s 2018SHZDZX01 and 2021SHZDZX0103), and the Fundamental Research Funds for the Central Universities. During the prepar- ation of this manuscript, Q.L. was a Visiting Fellow at Clare Hall, University of Cambridge, Cambridge, UK. This research was supported in part by the UK National Health Service (NHS) National Institute for Health Research (NIHR) Cambridge Biomedical Research Centre (BRC-1215-20014); the views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care

Research data supporting "Association of Environmental Uncertainty With Altered Decision-making and Learning Mechanisms in Youths With Obsessive-Compulsive Disorder"

In the 'PRL' folder, there is a 'core' and a 'support' folder. The raw data are saved in 'core' as 'probrev_forstan.csv'. The data in this file are collected from 103 adolescents (50 with OCD and 53 without) who completed a probabilistic reversal learning task (details in paper). The file contains information about subject ID (patient_num), trials, chosen stimulus per trial (chosen_stim), whether stimulus 1 was chosen (stim1_chosen), whether subject chose correct stimulus (chosen_correctly), task feedback (1 - positive, 0 - negative; fdbk), group (1 - Control, 0 - OCD). This data are in a format ready to undergo computational modelling. To run the models, use the Runner.R file in the 'core' folder. The models themselves are in .stan format. Next, in the 'WCST' folder, the raw data (in the wcst_data.txt) file are obtained from 73 adolescents (27 with OCD 46 without) who completed a Wisconsin Card Sorting Task (details in paper). Relevant columns in the wcst_data file are subject ID (subnum), trial (trial), whether the card chosen matches the test card on colour (corr_col), shape (corr_shape) and number (corr_name), group (0 -OCD, 1 - CTL), whether card was chosen correctly (corr). The main script for fitting the computational models to data is WCST_JAGS_2021.R. Models themselves are in .txt files (files that end in _dnormsd_groupdiff).Wellcome Trust grant 104631/Z/14/Z