Valence-dependent influence of serotonin depletion on model-based choice strategy.

Human decision-making arises from both reflective and reflexive mechanisms, which underpin goal-directed and habitual behavioural control. Computationally, these two systems of behavioural control have been described by different learning algorithms, model-based and model-free learning, respectively. Here, we investigated the effect of diminished serotonin (5-hydroxytryptamine) neurotransmission using dietary tryptophan depletion (TD) in healthy volunteers on the performance of a two-stage decision-making task, which allows discrimination between model-free and model-based behavioural strategies. A novel version of the task was used, which not only examined choice balance for monetary reward but also for punishment (monetary loss). TD impaired goal-directed (model-based) behaviour in the reward condition, but promoted it under punishment. This effect on appetitive and aversive goal-directed behaviour is likely mediated by alteration of the average reward representation produced by TD, which is consistent with previous studies. Overall, the major implication of this study is that serotonin differentially affects goal-directed learning as a function of affective valence. These findings are relevant for a further understanding of psychiatric disorders associated with breakdown of goal-directed behavioural control such as obsessive-compulsive disorders or addictions.This research was funded by Wellcome Trust Grants awarded to VV (Intermediate WT Fellowship) and Programme Grant (089589/Z/09/Z) awarded to TWR, BJE, ACR, JWD and BJS. It was conducted at the Behavioural and Clinical Neuroscience Institute, which is supported by a joint award from the Medical Research Council and Wellcome Trust (G00001354). YW was supported by the Fyssen Foundation. SP is supported by Marie Curie Intra-European Fellowship (FP7-People-2012-IEF).This is the final version of the article. It first appeared from NPG via http://dx.doi.org/10.1038/mp.2015.4

Cerebral activations related to ballistic, stepwise interrupted and gradually modulated movements in parkinson patients

Patients with Parkinson's disease (PD) experience impaired initiation and inhibition of movements such as difficulty to start/stop walking. At single-joint level this is accompanied by reduced inhibition of antagonist muscle activity. While normal basal ganglia (BG) contributions to motor control include selecting appropriate muscles by inhibiting others, it is unclear how PD-related changes in BG function cause impaired movement initiation and inhibition at single-joint level. To further elucidate these changes we studied 4 right-hand movement tasks with fMRI, by dissociating activations related to abrupt movement initiation, inhibition and gradual movement modulation. Initiation and inhibition were inferred from ballistic and stepwise interrupted movement, respectively, while smooth wrist circumduction enabled the assessment of gradually modulated movement. Task-related activations were compared between PD patients (N = 12) and healthy subjects (N = 18). In healthy subjects, movement initiation was characterized by antero-ventral striatum, substantia nigra (SN) and premotor activations while inhibition was dominated by subthalamic nucleus (STN) and pallidal activations, in line with the known role of these areas in simple movement. Gradual movement mainly involved antero-dorsal putamen and pallidum. Compared to healthy subjects, patients showed reduced striatal/SN and increased pallidal activation for initiation, whereas for inhibition STN activation was reduced and striatal-thalamo-cortical activation increased. For gradual movement patients showed reduced pallidal and increased thalamo-cortical activation. We conclude that PD-related changes during movement initiation fit the (rather static) model of alterations in direct and indirect BG pathways. Reduced STN activation and regional cortical increased activation in PD during inhibition and gradual movement modulation are better explained by a dynamic model that also takes into account enhanced responsiveness to external stimuli in this disease and the effects of hyper-fluctuating cortical inputs to the striatum and STN in particular

Adults with autism overestimate the volatility of the sensory environment.

Insistence on sameness and intolerance of change are among the diagnostic criteria for autism spectrum disorder (ASD), but little research has addressed how people with ASD represent and respond to environmental change. Here, behavioral and pupillometric measurements indicated that adults with ASD are less surprised than neurotypical adults when their expectations are violated, and decreased surprise is predictive of greater symptom severity. A hierarchical Bayesian model of learning suggested that in ASD, a tendency to overlearn about volatility in the face of environmental change drives a corresponding reduction in learning about probabilistically aberrant events, thus putatively rendering these events less surprising. Participant-specific modeled estimates of surprise about environmental conditions were linked to pupil size in the ASD group, thus suggesting heightened noradrenergic responsivity in line with compromised neural gain. This study offers insights into the behavioral, algorithmic and physiological mechanisms underlying responses to environmental volatility in ASD

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning

Surprised at All the Entropy: Hippocampal, Caudate and Midbrain Contributions to Learning from Prediction Errors

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts

Effects of Aversive Stimuli on Prospective Memory. An Event-Related fMRI Study

Prospective memory (PM) describes the ability to execute a previously planned action at the appropriate point in time. Although behavioral studies clearly showed that prospective memory performance is affected by the emotional significance attributed to the intended action, no study so far investigated the brain mechanisms subserving the modulatory effect of emotional salience on PM performance. The general aim of the present study was to explore brain regions involved in prospective memory processes when PM cues are associated with emotional stimuli. In particular, based on the hypothesised critical role of the prefrontal cortex in prospective memory in the presence of emotionally salient stimuli, we expected a stronger involvement of aPFC when the retrieval and execution of the intended action is cued by an aversive stimulus. To this aim BOLD responses of PM trials cued by aversive facial expressions were compared to PM trials cued by neutral facial expressions. Whole brain analysis showed that PM task cued by aversive stimuli is differentially associated with activity in the right lateral prefrontal area (BA 10) and in the left caudate nucleus. Moreover a temporal shift between the response of the caudate nucleus that preceded that of aPFC was observed. These findings suggest that the caudate nucleus might provide an early analysis of the affective properties of the stimuli, whereas the anterior lateral prefrontal cortex (BA10) would be involved in a slower and more deliberative analysis to guide goal-directed behaviour

Pharmacological Fingerprints of Contextual Uncertainty

Successful interaction with the environment requires flexible updating of our beliefs about the world. By estimating the likelihood of future events, it is possible to prepare appropriate actions in advance and execute fast, accurate motor responses. According to theoretical proposals, agents track the variability arising from changing environments by computing various forms of uncertainty. Several neuromodulators have been linked to uncertainty signalling, but comprehensive empirical characterisation of their relative contributions to perceptual belief updating, and to the selection of motor responses, is lacking. Here we assess the roles of noradrenaline, acetylcholine, and dopamine within a single, unified computational framework of uncertainty. Using pharmacological interventions in a sample of 128 healthy human volunteers and a hierarchical Bayesian learning model, we characterise the influences of noradrenergic, cholinergic, and dopaminergic receptor antagonism on individual computations of uncertainty during a probabilistic serial reaction time task. We propose that noradrenaline influences learning of uncertain events arising from unexpected changes in the environment. In contrast, acetylcholine balances attribution of uncertainty to chance fluctuations within an environmental context, defined by a stable set of probabilistic associations, or to gross environmental violations following a contextual switch. Dopamine supports the use of uncertainty representations to engender fast, adaptive responses. \ua9 2016 Marshall et al

Visual mismatch negativity (vMMN): A review and meta-analysis of studies in psychiatric and neurological disorders

The visual mismatch negativity (vMMN) response is an event-related potential (ERP) component, which is automatically elicited by events that violate predictions based on prior events. VMMN experiments use visual stimulus repetition to induce predictions, and vMMN is obtained by subtracting the response to rare unpredicted stimuli from those to frequent stimuli. One increasingly popular interpretation of the mismatch response postulates that vMMN, similar to its auditory counterpart (aMMN), represents a prediction error response generated by cortical mechanisms forming probabilistic representations of sensory signals. Here we discuss the physiological and theoretical basis of vMMN and review thirty-three studies from the emerging field of its clinical applications, presenting a meta-analysis of findings in schizophrenia, mood disorders, substance abuse, neurodegenerative disorders, developmental disorders, deafness, panic disorder and hypertension. Furthermore, we include reports on aging and maturation as they bear upon many clinically relevant conditions. Surveying the literature we found that vMMN is altered in several clinical populations which is in line with aMMN findings. An important potential advantage of vMMN however is that it allows the investigation of deficits in predictive processing in cognitive domains which rely primarily on visual information; a principal sensory modality and thus of vital importance in environmental information processing and response, and a modality which arguably may be more sensitive to some pathological changes. However, due to the relative infancy of research in vMMN compared to aMMN in clinical populations its potential for clinical application is not yet fully appreciated. The aim of this review and meta-analysis therefore is to present, in a detailed systematic manner, the findings from clinically-based vMMN studies, to discuss their potential impact and application, to raise awareness of this measure and to improve our understanding of disease upon fundamental aspects of visual information processing

Neural Network Development in Late Adolescents during Observation of Risk-Taking Action

Emotional maturity and social awareness are important for adolescents, particularly college students beginning to face the challenges and risks of the adult world. However, there has been relatively little research into personality maturation and psychological development during late adolescence and the neural changes underlying this development. We investigated the correlation between psychological properties (neuroticism, extraversion, anxiety, and depression) and age among late adolescents (n = 25, from 18 years and 1 month to 22 years and 8 months). The results revealed that late adolescents became less neurotic, less anxious, less depressive and more extraverted as they aged. Participants then observed video clips depicting hand movements with and without a risk of harm (risk-taking or safe actions) during functional magnetic resonance imaging (fMRI). The results revealed that risk-taking actions elicited significantly stronger activation in the bilateral inferior parietal lobule, temporal visual regions (superior/middle temporal areas), and parieto-occipital visual areas (cuneus, middle occipital gyri, precuneus). We found positive correlations of age and extraversion with neural activation in the insula, middle temporal gyrus, lingual gyrus, and precuneus. We also found a negative correlation of age and anxiety with activation in the angular gyrus, precentral gyrus, and red nucleus/substantia nigra. Moreover, we found that insula activation mediated the relationship between age and extraversion. Overall, our results indicate that late adolescents become less anxious and more extraverted with age, a process involving functional neural changes in brain networks related to social cognition and emotional processing. The possible neural mechanisms of psychological and social maturation during late adolescence are discussed