Mindfulness meditation modulates reward prediction errors in a passive conditioning task

Reinforcement learning models have demonstrated that phasic activity of dopamine neurons during reward expectation encodes information about the predictability of reward and cues that predict reward. Self-control strategies such as those practiced in mindfulness-based approaches is claimed to reduce negative and positive reactions to stimuli suggesting the hypothesis that such training may influence basic reward processing. Using a passive conditioning task and fMRI in a group of experienced mindfulness meditators and age-matched controls, we tested the hypothesis that mindfulness meditation influence reward and reward prediction error (PE) signals. We found diminished positive and negative PE-related blood-oxygen level-dependent (BOLD) responses in the putamen in meditators compared with controls. In the meditator group this decrease in striatal BOLD responses to reward PE was paralleled by increased activity in posterior insula, a primary interoceptive region. Critically, responses in the putamen during early trials of the conditioning procedure (run 1) were elevated in both meditators and controls. Overall, these results provide evidence that experienced mindfulness meditators are able to attenuate reward prediction signals to valenced stimuli, which may be related to interoceptive processes encoded in the posterior insula

Monte Carlo Planning method estimates planning horizons during interactive social exchange

Reciprocating interactions represent a central feature of all human exchanges. They have been the target of various recent experiments, with healthy participants and psychiatric populations engaging as dyads in multi-round exchanges such as a repeated trust task. Behaviour in such exchanges involves complexities related to each agent's preference for equity with their partner, beliefs about the partner's appetite for equity, beliefs about the partner's model of their partner, and so on. Agents may also plan different numbers of steps into the future. Providing a computationally precise account of the behaviour is an essential step towards understanding what underlies choices. A natural framework for this is that of an interactive partially observable Markov decision process (IPOMDP). However, the various complexities make IPOMDPs inordinately computationally challenging. Here, we show how to approximate the solution for the multi-round trust task using a variant of the Monte-Carlo tree search algorithm. We demonstrate that the algorithm is efficient and effective, and therefore can be used to invert observations of behavioural choices. We use generated behaviour to elucidate the richness and sophistication of interactive inference

Simulating future value in intertemporal choice

The laboratory study of how humans and other animals trade-off value and time has a long and storied history, and is the subject of a vast literature. However, despite a long history of study, there is no agreed upon mechanistic explanation of how intertemporal choice preferences arise. Several theorists have recently proposed model-based reinforcement learning as a candidate framework. This framework describes a suite of algorithms by which a model of the environment, in the form of a state transition function and reward function, can be converted on-line into a decision. The state transition function allows the model-based system to make decisions based on projected future states, while the reward function assigns value to each state, together capturing the necessary components for successful intertemporal choice. Empirical work has also pointed to a possible relationship between increased prospection and reduced discounting. In the current paper, we look for direct evidence of a relationship between temporal discounting and model-based control in a large new data set (n = 168). However, testing the relationship under several different modeling formulations revealed no indication that the two quantities are related

BOLD and its connection to dopamine release in human striatum: a cross-cohort comparison

Activity in midbrain dopamine neurons modulates the release of dopamine in terminal structures including the striatum, and controls reward-dependent valuation and choice. This fluctuating release of dopamine is thought to encode reward prediction error (RPE) signals and other value-related information crucial to decision-making, and such models have been used to track prediction error signals in the striatum as encoded by BOLD signals. However, until recently there have been no comparisons of BOLD responses and dopamine responses except for one clear correlation of these two signals in rodents. No such comparisons have been made in humans. Here, we report on the connection between the RPE-related BOLD signal recorded in one group of subjects carrying out an investment task, and the corresponding dopamine signal recorded directly using fast-scan cyclic voltammetry in a separate group of Parkinson's disease patients undergoing DBS surgery while performing the same task. The data display some correspondence between the signal types; however, there is not a one-to-one relationship. Further work is necessary to quantify the relationship between dopamine release, the BOLD signal and the computational models that have guided our understanding of both at the level of the striatum.This article is part of the themed issue 'Interpreting BOLD: a dialogue between cognitive and cellular neuroscience'

Loss Aversion Correlates With the Propensity to Deploy Model-Based Control

Reward-based decision making is thought to be driven by at least two different types of decision systems: a simple stimulus–response cache-based system which embodies the common-sense notion of “habit,” for which model-free reinforcement learning serves as a computational substrate, and a more deliberate, prospective, model-based planning system. Previous work has shown that loss aversion, a well-studied measure of how much more on average individuals weigh losses relative to gains during decision making, is reduced when participants take all possible decisions and outcomes into account including future ones, relative to when they myopically focus on the current decision. Model-based control offers a putative mechanism for implementing such foresight. Using a well-powered data set (N = 117) in which participants completed two different tasks designed to measure each of the two quantities of interest, and four models of choice data for these tasks, we found consistent evidence of a relationship between loss aversion and model-based control but in the direction opposite to that expected based on previous work: loss aversion had a positive relationship with model-based control. We did not find evidence for a relationship between either decision system and risk aversion, a related aspect of subjective utility

Taming the shrewdness of neural function: Methodological challenges in computational psychiatry

Computational psychiatry involves applying a collection of theoretical notions, including data analysis and mathematical and computational modeling, to the problems of psychiatry. It is a nascent field whose central methods are just in the process of being developed. We consider some of the challenges and opportunities for techniques and approaches that are presenting themselves as it starts to take on a more concrete form

Computational phenotyping of two-person interactions reveals differential neural response to depth-of-thought.

Reciprocating exchange with other humans requires individuals to infer the intentions of their partners. Despite the importance of this ability in healthy cognition and its impact in disease, the dimensions employed and computations involved in such inferences are not clear. We used a computational theory-of-mind model to classify styles of interaction in 195 pairs of subjects playing a multi-round economic exchange game. This classification produces an estimate of a subject's depth-of-thought in the game (low, medium, high), a parameter that governs the richness of the models they build of their partner. Subjects in each category showed distinct neural correlates of learning signals associated with different depths-of-thought. The model also detected differences in depth-of-thought between two groups of healthy subjects: one playing patients with psychiatric disease and the other playing healthy controls. The neural response categories identified by this computational characterization of theory-of-mind may yield objective biomarkers useful in the identification and characterization of pathologies that perturb the capacity to model and interact with other humans

Multidirectional pathways between attachment, mentalizing, and posttraumatic stress symptomatology in the context of childhood trauma

Introduction:Exposure to traumatic stressful events in childhood is an important risk factor for the development of posttraumatic symptomatology. From a mentalization-based developmental perspective, childhood adversity can affect attachment in children and may result in insecure attachment and impaired mentalizing abilities, which increase the lifetime risk for psychopathology. The present cross-sectional study examined the potential mediating role of attachment insecurity and impaired mentalizing on the relationship between childhood trauma and posttraumatic symptomatology. Method: Adults who had experienced childhood neglect and abuse (n = 295, 184 patients with personality disorder and 111 community controls) completed self-report measures of posttraumatic stress disorder (PTSD) symptoms, dissociative experiences, adult attachment insecurity, and mentalizing. Results: Structural equation modelling results revealed that attachment insecurity together with lower mentalizing mediated the link between childhood trauma and PTSD symptoms, and lower mentalizing mediated the link between childhood trauma and dissociative experiences. Conclusion: The findings show that attachment insecurity and lower mentalizing play significant mediating roles in the reporting of posttraumatic symptomatology among survivors of childhood abuse and neglect, with treatment implications for mentalization-based therapy as beneficial for individuals with a history of childhood trauma

Human substantia nigra and ventral tegmental area involvement in computing social error signals during the ultimatum game

Social norms play an essential role in our societies, and since the social environment is changing constantly, our internal models of it also need to change. In humans, there is mounting evidence that neural structures such as the insula and the ventral striatum are involved in detecting norm violation and updating internal models. However, because of methodological challenges, little is known about the possible involvement of midbrain structures in detecting norm violation and updating internal models of our norms. Here we used high-resolution cardiac-gated functional magnetic resonance imaging and a norm adaptation paradigm in healthy adults to investigate the role of the substantia nigra/ventral tegmental area (SN/VTA) complex in tracking signals related to norm violation that can be used to update internal norms. We show that the SN/VTA codes for the norm's variance prediction error (PE) and norm PE with spatially distinct regions coding for negative and positive norm PE. These results point to a common role played by the SN/VTA complex in supporting both simple reward-based and social decision making

The Functional Architecture of the Brain Underlies Strategic Deception in Impression Management

Impression management, as one of the most essential skills of social function, impacts one’s survival and success in human societies. However, the neural architecture underpinning this social skill remains poorly understood. By employing a two-person bargaining game, we exposed three strategies involving distinct cognitive processes for social impression management with different levels of strategic deception. We utilized a novel adaptation of Granger causality accounting for signal-dependent noise (SDN), which captured the directional connectivity underlying the impression management during the bargaining game. We found that the sophisticated strategists engaged stronger directional connectivity from both dorsal anterior cingulate cortex and retrosplenial cortex to rostral prefrontal cortex, and the strengths of these directional influences were associated with higher level of deception during the game. Using the directional connectivity as a neural signature, we identified the strategic deception with 80% accuracy by a machine-learning classifier. These results suggest that different social strategies are supported by distinct patterns of directional connectivity among key brain regions for social cognition