Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance

Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

BACKGROUND: Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear. METHODS: Reinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock. RESULTS: We show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (Pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress. CONCLUSIONS: This formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders

Reliability of Fronto-Amygdala Coupling during Emotional Face Processing.

One of the most exciting translational prospects for brain imaging research is the potential use of functional magnetic resonance imaging (fMRI) 'biomarkers' to predict an individual's risk of developing a neuropsychiatric disorder or the likelihood of responding to a particular intervention. This proposal depends critically on reliable measurements at the level of the individual. Several previous studies have reported relatively poor reliability of amygdala activation during emotional face processing, a key putative fMRI 'biomarker'. However, the reliability of amygdala connectivity measures is much less well understood. Here, we assessed the reliability of task-modulated coupling between three seed regions (left and right amygdala and the subgenual anterior cingulate cortex) and the dorsomedial frontal/cingulate cortex (DMFC), measured using a psychophysiological interaction analysis in 29 healthy individuals scanned approximately two weeks apart. We performed two runs on each day of three different emotional face-processing tasks: emotion identification, emotion matching, and gender classification. We tested both between-day reliability and within-day (between-run) reliability. We found good-to-excellent within-subject reliability of amygdala-DMFC coupling, both between days (in two tasks), and within day (in one task). This suggests that disorder-relevant regional coupling may be sufficiently reliable to be used as a predictor of treatment response or clinical risk in future clinical studies

Assessment of cognitive safety in clinical drug development

Cognitive impairment is increasingly recognised as an important potential adverse effect of medication. However, many drug development programmes do not incorporate sensitive cognitive measurements. Here, we review the rationale for cognitive safety assessment, and explain several basic methodological principles for measuring cognition during clinical drug development, including study design and statistical analysis, from Phase I through to postmarketing. The crucial issue of how cognition should be assessed is emphasized, especially the sensitivity of measurement. We also consider how best to interpret the magnitude of any identified effects, including comparison with benchmarks. We conclude by discussing strategies for the effective communication of cognitive risks

Longitudinal alterations in motivational salience processing in ultra-high-risk subjects for psychosis

Impairments in the attribution of salience are thought to be fundamental to the development of psychotic symptoms and the onset of psychotic disorders. The aim of the present study was to explore longitudinal alterations in salience processing in ultra-high-risk subjects for psychosis.; A total of 23 ultra-high-risk subjects and 13 healthy controls underwent functional magnetic resonance imaging at two time points (mean interval of 17 months) while performing the Salience Attribution Test to assess neural responses to task-relevant (adaptive salience) and task-irrelevant (aberrant salience) stimulus features.; At presentation, high-risk subjects were less likely than controls to attribute salience to relevant features, and more likely to attribute salience to irrelevant stimulus features. These behavioural differences were no longer evident at follow-up. When attributing salience to relevant cue features, ultra-high-risk subjects showed less activation than controls in the ventral striatum at both baseline and follow-up. Within the high-risk sample, amelioration of abnormal beliefs over the follow-up period was correlated with an increase in right ventral striatum activation during the attribution of salience to relevant cue features.; These findings confirm that salience processing is perturbed in ultra-high-risk subjects for psychosis, that this is linked to alterations in ventral striatum function, and that clinical outcomes are related to longitudinal changes in ventral striatum function during salience processing

The acute and non-acute effects of cannabis on reward processing:A systematic review

Cannabis use has historically been thought to cause amotivation, but the relationship between cannabis and apathy, anhedonia, and reward processing remains poorly characterised. In this systematic review, we evaluated whether cannabis exposure acutely and/or non-acutely was associated with altered reward processing using questionnaire, behavioural, or functional neuroimaging measures. Questionnaire studies demonstrated greater anhedonia in adolescent cannabis users, and some indication of greater apathy in young adult cannabis users. Behavioural studies yielded some evidence of reduced reward learning in adolescent cannabis users, though there were too few studies in this category for reliable conclusions. Finally, longitudinal and acute functional neuroimaging studies showed an association between cannabis and blunted neural responses to reward, which did not emerge consistently in cross-sectional studies. The current results suggest that cannabis use is associated with specific impairments in reward and motivation. Future large-scale, longitudinal studies which use multiple behavioural and neuroimaging measures of reward processing may further clarify the impact of cannabis use on motivational and reward processes, and neural networks.</p

Cooperative Behavior in the Ultimatum Game and Prisoner's Dilemma Depends on Players' Contributions.

Economic games such as the Ultimatum Game (UG) and Prisoner's Dilemma (PD) are widely used paradigms for studying fairness and cooperation. Monetary versions of these games involve two players splitting an arbitrary sum of money. In real life, however, people's propensity to engage in cooperative behavior depends on their effort and contribution; factors that are well known to affect perceptions of fairness. We therefore sought to explore the impact of relative monetary contributions by players in the UG and PD. Adapted computerized UG and PD games, in which relative contributions from each player were manipulated, were administered to 200 participants aged 18-50 years old (50% female). We found that players' contribution had large effects on cooperative behavior. Specifically, cooperation was greater amongst participants when their opponent had contributed more to joint earnings. This was manifested as higher acceptance rates and higher offers in the UG; and fewer defects in the PD compared to when the participant contributed more. Interestingly, equal contributions elicited the greatest sensitivity to fairness in the UG, and least frequent defection in the PD. Acceptance rates correlated positively with anxiety and sex differences were found in defection behavior. This study highlights the feasibility of computerized games to assess cooperative behavior and the importance of considering cooperation within the context of effortful contribution

Do patients with schizophrenia exhibit aberrant salience?

BACKGROUND: It has been suggested that some psychotic symptoms reflect ‘aberrant salience’, related to dysfunctional reward learning. To test this hypothesis we investigated whether patients with schizophrenia showed impaired learning of task-relevant stimulusreinforcement associations in the presence of distracting task-irrelevant cues. METHODS: We tested 20 medicated patients with schizophrenia and 17 controls on a reaction time game, the Salience Attribution Test. In this game, participants made a speeded response to earn money in the presence of conditioned stimuli (CSs). Each CS comprised two visual dimensions, colour and form. Probability of reinforcement varied over one of these dimensions (task-relevant), but not the other (task-irrelevant). Measures of adaptive and aberrant motivational salience were calculated on the basis of latency and subjective reinforcement probability rating differences over the task-relevant and task-irrelevant dimensions respectively. RESULTS: Participants rated reinforcement significantly more likely and responded significantly faster on high-probability reinforced relative to lowprobability reinforced trials, representing adaptive motivational salience. Patients exhibited reduced adaptive salience relative to controls, but the two groups did not differ in terms of aberrant salience. Patients with delusions exhibited significantly greater aberrant salience than those without delusions, and aberrant salience also correlated with negative symptoms. In the controls, aberrant salience correlated significantly with ‘introvertive anhedonia’ schizotypy. CONCLUSIONS: These data support the hypothesis that aberrant salience is related to the presence of delusions in medicated patients with schizophrenia, but are also suggestive of a link with negative symptoms. The relationship between aberrant salience and psychotic symptoms warrants further investigation in unmedicated patients