linking goal-directed and model-based behavior

In experimental psychology different experiments have been developed to assess goal–directed as compared to habitual control over instrumental decisions. Similar to animal studies selective devaluation procedures have been used. More recently sequential decision-making tasks have been designed to assess the degree of goal-directed vs. habitual choice behavior in terms of an influential computational theory of model-based compared to model-free behavioral control. As recently suggested, different measurements are thought to reflect the same construct. Yet, there has been no attempt to directly assess the construct validity of these different measurements. In the present study, we used a devaluation paradigm and a sequential decision-making task to address this question of construct validity in a sample of 18 healthy male human participants. Correlational analysis revealed a positive association between model-based choices during sequential decisions and goal-directed behavior after devaluation suggesting a single framework underlying both operationalizations and speaking in favor of construct validity of both measurement approaches. Up to now, this has been merely assumed but never been directly tested in humans

Opposing roles for amygdala and vmPFC in the return of appetitive conditioned responses in humans

Learning accounts of addiction and obesity emphasize the persistent power of Pavlovian reward cues to trigger craving and increase relapse risk. While extinction can reduce conditioned responding, Pavlovian relapse phenomena-the return of conditioned responding following successful extinction-challenge the long-term success of extinction-based treatments. Translational laboratory models of Pavlovian relapse could therefore represent a valuable tool to investigate the mechanisms mediating relapse, although so far human research has mostly focused on return of fear phenomena. To this end we developed an appetitive conditioning paradigm with liquid food rewards in combination with a 3-day design to investigate the return of appetitive Pavlovian responses and the involved neural structures in healthy subjects. Pavlovian conditioning (day 1) was assessed in 62 participants, and a subsample (n = 33) further completed extinction (day 2) and a reinstatement test (day 3). Conditioned responding was assessed on explicit (pleasantness ratings) and implicit measures (reaction time, skin conductance, heart rate, startle response) and reinstatement effects were further evaluated using fMRI. We observed a return of conditioned responding during the reinstatement test, evident by enhanced skin conductance responses, accompanied by enhanced BOLD responses in the amygdala. On an individual level, psychophysiological reinstatement intensity was significantly anticorrelated with ventromedial prefrontal cortex (vmPFC) activation, and marginally anticorrelated with enhanced amygdala-vmPFC connectivity during late reinstatement. Our results extend evidence from return of fear phenomena to the appetitive domain, and highlight the role of the vmPFC and its functional connection with the amygdala in regulating appetitive Pavlovian relapse

Dorsolateral prefrontal cortex contributes to the impaired behavioral adaptation in alcohol dependence

Substance-dependent individuals often lack the ability to adjust decisions flexibly in response to the changes in reward contingencies. Prediction errors (PEs) are thought to mediate flexible decision-making by updating the reward values associated with available actions. In this study, we explored whether the neurobiological correlates of PEs are altered in alcohol dependence. Behavioral, and functional magnetic resonance imaging (fMRI) data were simultaneously acquired from 34 abstinent alcohol-dependent patients (ADP) and 26 healthy controls (HC) during a probabilistic reward-guided decision-making task with dynamically changing reinforcement contingencies. A hierarchical Bayesian inference method was used to fit and compare learning models with different assumptions about the amount of task-related information subjects may have inferred during the experiment. Here, we observed that the best-fitting model was a modified Rescorla-Wagner type model, the “double-update” model, which assumes that subjects infer the knowledge that reward contingencies are anti-correlated, and integrate both actual and hypothetical outcomes into their decisions. Moreover, comparison of the best-fitting model's parameters showed that ADP were less sensitive to punishments compared to HC. Hence, decisions of ADP after punishments were loosely coupled with the expected reward values assigned to them. A correlation analysis between the model-generated PEs and the fMRI data revealed a reduced association between these PEs and the BOLD activity in the dorsolateral prefrontal cortex (DLPFC) of ADP. A hemispheric asymmetry was observed in the DLPFC when positive and negative PE signals were analyzed separately. The right DLPFC activity in ADP showed a reduced correlation with positive PEs. On the other hand, ADP, particularly the patients with high dependence severity, recruited the left DLPFC to a lesser extent than HC for processing negative PE signals. These results suggest that the DLPFC, which has been linked to adaptive control of action selection, may play an important role in cognitive inflexibility observed in alcohol dependence when reinforcement contingencies change. Particularly, the left DLPFC may contribute to this impaired behavioral adaptation, possibly by impeding the extinction of the actions that no longer lead to a reward

Neuromelanin-Sensitive Magnetic Resonance Imaging in Schizophrenia: A Meta-Analysis of Case-Control Studies

Background: Psychiatry is in urgent need of reliable biomarkers. Novel neuromelanin-sensitive magnetic resonance imaging (NM-MRI) sequences provide a time-efficient and non-invasive way to investigate the human brain in-vivo. This gives insight into the metabolites of dopaminergic signaling and may provide further evidence for potential dopaminergic alterations in patients with schizophrenia (SCZ). The present systematic review provides a meta-analysis of case-control studies using neuromelanin-sensitive sequences in SCZ vs. healthy controls (HC). Methods: According to predefined search terms and inclusion criteria studies were extracted on PubMed. Meta-analyses with a fixed and random-effects model with inverse variance method, DerSimonian-Laird estimator for tau(2), and Cohen's d were calculated. Bias was assessed using funnel plots. The primary study outcome was contrast-to-noise ratio (CNR) in the substantia nigra compared between HC and SCZ. Results: The total sample of k = 6 studies included n = 183 cases and n = 162 controls. Across all studies we found a significant elevation of CNR in the substantia nigra (d = 0.42 [0.187; 0.655], z = 3.521, p < 0.001) in cases compared to controls. We found no significant difference in the control region of locus coeruleus (d = -0.07 [-0.446; 0.302], z = -0.192, p = 0.847), with CNR for the latter only reported in k = 3 studies. Conclusion: CNR in the substantia nigra were significantly elevated in cases compared to controls. Our results support neuromelanin as a candidate biomarker for dopaminergic dysfunction in schizophrenia. Further studies need to assess this candidate marker in large, longitudinal cohorts and address potential effects of disease state, medication and correlations with symptoms

Functional connectivity alterations between default mode network and occipital cortex in patients with obsessive-compulsive disorder (OCD)

Altered brain network connectivity is a potential biomarker for obsessive-compulsive disorder (OCD). A meta-analysis of resting-state MRI studies by Gürsel et al. (2018) described altered functional connectivity in OCD patients within and between the default mode network (DMN), the salience network (SN), and the frontoparietal network (FPN), as well as evidence for aberrant fronto-striatal circuitry. Here, we tested the replicability of these meta-analytic rsfMRI findings by measuring functional connectivity during resting-state fMRI in a new sample of OCD patients (n = 24) and matched controls (n = 33). We performed seed-to-voxel analyses using 30 seed regions from the prior meta-analysis. OCD patients showed reduced functional connectivity between the SN and the DMN compared to controls, replicating previous findings. We did not observe significant group differences of functional connectivity within the DMN, SN, nor FPN. Additionally, we observed reduced connectivity between the visual network to both the DMN and SN in OCD patients, in particular reduced functional connectivity between lateral parietal seeds and the left inferior lateral occipital pole. Furthermore, the right lateral parietal seed (associated with the DMN) was more strongly correlated with a cluster in the right lateral occipital cortex and precuneus (a region partly overlapping with the Dorsal Attentional Network (DAN)) in patients. Importantly, this latter finding was positively correlated to OCD symptom severity. Overall, our study partly replicated prior meta-analytic findings, highlighting hypoconnectivity between SN and DMN as a potential biomarker for OCD. Furthermore, we identified changes between the SN and the DMN with the visual network. This suggests that abnormal connectivity between cortex regions associated with abstract functions (transmodal regions such as the DMN), and cortex regions associated with constrained neural processing (unimodal regions such as the visual cortex), may be important in OCD

Belief Updating in Subclinical and Clinical Delusions

Background and Hypothesis Current frameworks propose that delusions result from aberrant belief updating due to altered prediction error (PE) signaling and misestimation of environmental volatility. We aimed to investigate whether behavioral and neural signatures of belief updating are specifically related to the presence of delusions or generally associated with manifest schizophrenia. Methods Our cross-sectional design includes human participants (n[female/male] = 66[25/41]), stratified into four groups: healthy participants with minimal (n = 22) or strong delusional-like ideation (n = 18), and participants with diagnosed schizophrenia with minimal (n = 13) or strong delusions (n = 13), resulting in a 2 × 2 design, which allows to test for the effects of delusion and diagnosis. Participants performed a reversal learning task with stable and volatile task contingencies during fMRI scanning. We formalized learning with a hierarchical Gaussian filter model and conducted model-based fMRI analysis regarding beliefs of outcome uncertainty and volatility, precision-weighted PEs of the outcome- and the volatility-belief. Results Patients with schizophrenia as compared to healthy controls showed lower accuracy and heightened choice switching, while delusional ideation did not affect these measures. Participants with delusions showed increased precision-weighted PE-related neural activation in fronto-striatal regions. People with diagnosed schizophrenia overestimated environmental volatility and showed an attenuated neural representation of volatility in the anterior insula, medial frontal and angular gyrus. Conclusions Delusional beliefs are associated with altered striatal PE-signals. Juxtaposing, the potentially unsettling belief that the environment is constantly changing and weaker neural encoding of this subjective volatility seems to be associated with manifest schizophrenia, but not with the presence of delusional ideation.Peer Reviewe

Acute stress alters probabilistic reversal learning in healthy male adults

Behavioural adaptation is a fundamental cognitive ability, ensuring survival by allowing for flexible adjustment to changing environments. In laboratory settings, behavioural adaptation can be measured with reversal learning paradigms requiring agents to adjust reward learning to stimulus–action–outcome contingency changes. Stress is found to alter flexibility of reward learning, but effect directionality is mixed across studies. Here, we used model-based functional MRI (fMRI) in a within-subjects design to investigate the effect of acute psychosocial stress on flexible behavioural adaptation. Healthy male volunteers (n = 28) did a reversal learning task during fMRI in two sessions, once after the Trier Social Stress Test (TSST), a validated psychosocial stress induction method, and once after a control condition. Stress effects on choice behaviour were investigated using multilevel generalized linear models and computational models describing different learning processes that potentially generated the data. Computational models were fitted using a hierarchical Bayesian approach, and model-derived reward prediction errors (RPE) were used as fMRI regressors. We found that acute psychosocial stress slightly increased correct response rates. Model comparison revealed that double-update learning with altered choice temperature under stress best explained the observed behaviour. In the brain, model-derived RPEs were correlated with BOLD signals in striatum and ventromedial prefrontal cortex (vmPFC). Striatal RPE signals for win trials were stronger during stress compared with the control condition. Our study suggests that acute psychosocial stress could enhance reversal learning and RPE brain responses in healthy male participants and provides a starting point to explore these effects further in a more diverse population

Computational mechanisms of belief updating in relation to psychotic-like experiences

Introduction Psychotic-like experiences (PLEs) may occur due to changes in weighting prior beliefs and new evidence in the belief updating process. It is still unclear whether the acquisition or integration of stable beliefs is altered, and whether such alteration depends on the level of environmental and belief precision, which reflects the associated uncertainty. This motivated us to investigate uncertainty-related dynamics of belief updating in relation to PLEs using an online study design. Methods We selected a sample (n = 300) of participants who performed a belief updating task with sudden change points and provided self-report questionnaires for PLEs. The task required participants to observe bags dropping from a hidden helicopter, infer its position, and dynamically update their belief about the helicopter's position. Participants could optimize performance by adjusting learning rates according to inferred belief uncertainty (inverse prior precision) and the probability of environmental change points. We used a normative learning model to examine the relationship between adherence to specific model parameters and PLEs. Results PLEs were linked to lower accuracy in tracking the outcome (helicopter location) (β = 0.26 ± 0.11, p = 0.018) and to a smaller increase of belief precision across observations after a change point (β = −0.003 ± 0.0007, p < 0.001). PLEs were related to slower belief updating when participants encountered large prediction errors (β = −0.03 ± 0.009, p = 0.001). Computational modeling suggested that PLEs were associated with reduced overall belief updating in response to prediction errors (βPE = −1.00 ± 0.45, p = 0.028) and reduced modulation of updating at inferred environmental change points (βCPP = −0.84 ± 0.38, p = 0.023). Discussion We conclude that PLEs are associated with altered dynamics of belief updating. These findings support the idea that the process of balancing prior belief and new evidence, as a function of environmental uncertainty, is altered in PLEs, which may contribute to the development of delusions. Specifically, slower learning after large prediction errors in people with high PLEs may result in rigid beliefs. Disregarding environmental change points may limit the flexibility to establish new beliefs in the face of contradictory evidence. The present study fosters a deeper understanding of inferential belief updating mechanisms underlying PLEs.Peer Reviewe

Computational mechanisms of belief updating in relation to psychotic-like experiences

Introduction: Psychotic-like experiences (PLEs) may occur due to changes in weighting prior beliefs and new evidence in the belief updating process. It is still unclear whether the acquisition or integration of stable beliefs is altered, and whether such alteration depends on the level of environmental and belief precision, which reflects the associated uncertainty. This motivated us to investigate uncertainty-related dynamics of belief updating in relation to PLEs using an online study design. Methods: We selected a sample (n = 300) of participants who performed a belief updating task with sudden change points and provided self-report questionnaires for PLEs. The task required participants to observe bags dropping from a hidden helicopter, infer its position, and dynamically update their belief about the helicopter's position. Participants could optimize performance by adjusting learning rates according to inferred belief uncertainty (inverse prior precision) and the probability of environmental change points. We used a normative learning model to examine the relationship between adherence to specific model parameters and PLEs. Results: PLEs were linked to lower accuracy in tracking the outcome (helicopter location) (beta = 0.26 +/- 0.11, p = 0.018) and to a smaller increase of belief precision across observations after a change point (beta = -0.003 +/- 0.0007, p < 0.001). PLEs were related to slower belief updating when participants encountered large prediction errors (beta = -0.03 +/- 0.009, p = 0.001). Computational modeling suggested that PLEs were associated with reduced overall belief updating in response to prediction errors (beta(PE) = -1.00 +/- 0.45, p = 0.028) and reduced modulation of updating at inferred environmental change points (beta(CPP) = -0.84 +/- 0.38, p = 0.023). Discussion: We conclude that PLEs are associated with altered dynamics of belief updating. These findings support the idea that the process of balancing prior belief and new evidence, as a function of environmental uncertainty, is altered in PLEs, which may contribute to the development of delusions. Specifically, slower learning after large prediction errors in people with high PLEs may result in rigid beliefs. Disregarding environmental change points may limit the flexibility to establish new beliefs in the face of contradictory evidence. The present study fosters a deeper understanding of inferential belief updating mechanisms underlying PLEs