Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology

Optimal inference with suboptimal models:Addiction and active Bayesian inference

When casting behaviour as active (Bayesian) inference, optimal inference is defined with respect to an agent's beliefs - based on its generative model of the world. This contrasts with normative accounts of choice behaviour, in which optimal actions are considered in relation to the true structure of the environment - as opposed to the agent's beliefs about worldly states (or the task). This distinction shifts an understanding of suboptimal or pathological behaviour away from aberrant inference as such, to understanding the prior beliefs of a subject that cause them to behave less 'optimally' than our prior beliefs suggest they should behave. Put simply, suboptimal or pathological behaviour does not speak against understanding behaviour in terms of (Bayes optimal) inference, but rather calls for a more refined understanding of the subject's generative model upon which their (optimal) Bayesian inference is based. Here, we discuss this fundamental distinction and its implications for understanding optimality, bounded rationality and pathological (choice) behaviour. We illustrate our argument using addictive choice behaviour in a recently described 'limited offer' task. Our simulations of pathological choices and addictive behaviour also generate some clear hypotheses, which we hope to pursue in ongoing empirical work

Logical reasoning in schizotypal personality

It was examined whether psychotic-like personality traits in a sample of 205 college students could predict logical reasoning deficits, akin to those seen in schizophrenia. The participants were tested on their ability to assess the logical validity of premises (Logical Reasoning Task), and completed a multi-dimensional schizotypy inventory (O-LIFE). Low accuracy was associated with increased levels of disorganized schizotypy (‘Cognitive Disorganization’), while elevated errors were associated with increased levels of positive (‘Unusual Experiences’), negative (‘Introvertive Anhedonia’) and impulsive (‘Impulsivity Non-conformity’) schizotypy. Nevertheless, multiple regression analyses revealed that negative schizotypy was retained as the only significant predictor after performance was corrected for random guessing, and the contribution of the average amount of time spent on each premise was controlled. The results suggest that, although most schizotypy dimensions have a detrimental effect on reasoning performance, possibly due to disadvantageous test-taking strategies, negative schizotypy is the most reliable predictor of logical reasoning deficits. It is proposed that social/interpersonal schizotypal traits, like negative symptoms of schizophrenia, are accompanied by deficient executive functions of working memory, which appear to undermine, inter alia, logical reasoning processing

An Exploratory Study on the Influence of Psychopathological Risk and Impulsivity on BMI and Perceived Quality of Life in Obese Patients

The present study aimed to assess the psychological profiles of adult male and female obese patients, as well as to verify the possible influence of their psychopathological risk and impulsivity on their body mass index (BMI) and perceived quality of life. A total of 64 obese subjects accessing a center for care of their obesity were assessed through anthropometric and psychometric measurements. All anthropometric measures in men were higher than in women, while in turn, women showed higher psychopathological symptoms. Furthermore, the symptoms of somatization and psychoticism were predictors for a higher BMI in men, but there was no effect of psychopathological symptoms on the perceived quality of life (QoL) of male subjects. Moreover, in women, somatization and attentional impulsivity were predictors for a higher BMI, whereas no correlation was found between their psychopathological risk and perceived QoL. The results of regression analysis underlined that somatization is a “core” psychopathological symptom in obese subjects regardless of their sex, which is a potential predictor for a higher BMI. The psychological difficulties of the subjects had no effect on their perceived QoL, suggesting that they find it difficult to reflect on the impact that obesity has on their life

Tourette-like behaviors in the normal population are associated with hyperactive/impulsive ADHD-like behaviors but do not relate to deficits in conditioned inhibition or response inhibition

Attention-Deficit Hyperactivity Disorder (ADHD) and Tourette Syndrome (TS) present as distinct conditions clinically; however, comorbidity and inhibitory control deficits have been proposed for both. Whilst such deficits have been studied widely within clinical populations, findings are mixed — partly due to comorbidity and/or medication effects — and studies have rarely distinguished between subtypes of the disorders. Studies in the general population are sparse. Using a continuity approach, the present study examined (i) the relationships between inattentive and hyperactive/impulsive aspects of ADHD and TS-like behaviors in the general population, and (ii) their unique associations with automatic and executive inhibitory control, as well as (iii) yawning (a proposed behavioural model of TS). One hundred and thirty-eight participants completed self-report measures for ADHD and TS-like behaviors as well as yawning, and aconditioned inhibition task to assess automatic inhibition

The value of novelty in schizophrenia

Influential models of schizophrenia suggest that patients experience incoming stimuli as excessively novel and motivating, with important consequences for hallucinatory experience and delusional belief. However, whether schizophrenia patients exhibit excessive novelty value and whether this interferes with adaptive behaviour has not yet been formally tested. Here, we employed a three-armed bandit task to investigate this hypothesis. Schizophrenia patients and healthy controls were first familiarised with a group of images and then asked to repeatedly choose between familiar and unfamiliar images associated with different monetary reward probabilities. By fitting a reinforcement-learning model we were able to estimate the values attributed to familiar and unfamiliar images when first presented in the context of the decision-making task. In line with our hypothesis, we found increased preference for newly introduced images (irrespective of whether these were familiar or unfamiliar) in patients compared to healthy controls and this to correlate with severity of hallucinatory experience. In addition, we found a correlation between value assigned to novel images and task performance, suggesting that excessive novelty value may interfere with optimal learning in patients, putatively through the disruption of the mechanisms regulating exploration versus exploitation. Our results suggest excessive novelty value in patients, whereby even previously seen stimuli acquire higher value as the result of their exposure in a novel context – a form of ‘hyper novelty’ which may explain why patients are often attracted by familiar stimuli experienced as new

Exploring The Neural Correlates of Reading Comprehension and Social Cognition Deficits in College Students with ADHD

Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder characterized by inattention, impulsivity, and hyperactivity. Symptoms of this disorder have been shown to adversely impact academic and social functioning of those with ADHD. College students with ADHD, compared to their non-ADHD peers, are at increased risk for academic and social difficulties. Given the reading-intensive and socially-driven environment of the college campus, empirical literature examining the reading comprehension and social cognition of college students are wanting. The current investigation utilized the Nelson-Denny Reading Test (NDRT) and Faux Pas Recognition test (FPRT) to assess reading comprehension and social cognition, respectively, in college students with (n = 3) and without ADHD (n = 9). The Short Story Task (SST) was administered during functional magnetic resonance imaging (fMRI) to examine neural correlates of narrative comprehension and theory of mind (ToM) while reading short fictional stories of varying prose complexity. The ADHD and control groups did not differ in IQ, GPA, or scores of NDRT, FPRT, or SST, suggesting that they had comparable academic performance, narrative comprehension, and social cognition. The fMRI analysis of SST showed that the ADHD group demonstrated increased activation in the left anterior cingulate (ACC) and parahippocampal gyrus (PHG) while reading the complex story compared to the simple story. This differential activation was not observed in the CTRL group, suggesting that the ADHD group required more neural resources to process the emotional components of the complex story to achieve the comparable performance on the SST. The ADHD group additionally exhibited lower activation in the narrative comprehension and ToM networks (medial prefrontal cortex, Broca’s area, angular gyri). Collectively, these results indicate that while ADHD and CTRL groups did not differ behaviorally, they exhibit differential neural activation patterns in tasks related to narrative comprehension and social cognition. Further investigations may inform the development of educational and psychosocial interventions to improve academic and social functioning in young adults with ADHD

Impulsivity and Compulsivity in Anorexia Nervosa: Cognitive Systems Underlying Variation in Appetite Restraint from an RDoC Perspective

Contemporary nomenclature for anorexia nervosa (AN) describes the eating disorder as transdiagnostic, with overlapping facets of impulsivity and compulsivity contributing to variations in binge-purge, restrictive eating and maladaptive cognitions. It is important to understand how these facets interact, given that those diagnosed with AN often fluctuate and relapse–as opposed to maintaining a stable diagnosis—between Diagnostic and Statistical Manual version 5 (DSM-5) categories, over the life course. The National Institute of Health’s Research Domain Criteria (NIH RDoC) subscribes to the transdiagnostic view of mental disorders and provides progressive guidelines for neuroscience research. As such, using the RDoC guidelines may help to pinpoint how impulsivity and compulsivity contribute to the cognitive mechanisms underlying variations in appetite restraint in eating disorders and common psychiatric comorbidities such as anxiety and obsessive-compulsive disorder. Exploring impulsivity and compulsivity in AN from the perspective of the RDoC cognitive systems domain is aided by measures of genetic, molecular, cellular, neural, physiological, behavioural and cognitive task paradigms. Thus, from the standpoint of the RDoC measures, this chapter will describe some of the ways in which impulsivity and compulsivity contribute to the cognitive systems associated with appetite restraint in AN, with the aim of further clarifying a model of appetite restraint to improve treatment interventions

Computational Modelling of Information Gathering

This thesis describes computational modelling of information gathering behaviour under active inference – a framework for describing Bayes optimal behaviour. Under active inference perception, attention and action all serve for same purpose: minimising variational free energy. Variational free energy is an upper bound on surprise and minimising it maximises an agent’s evidence for its survival. An agent achieves this by acquiring information (resolving uncertainty) about the hidden states of the world and uses the acquired information to act on the outcomes it prefers. In this work I placed special emphasis on the resolution of uncertainty about the states of the world. I first created a visual search task called scene construction task. In this task one needs to accumulate evidence for competing hypotheses (different visual scenes) through sequential sampling of a visual scene and categorising it once there is sufficient evidence. I showed that a computational agent attends to the most salient (epistemically valuable) locations in this task. In the next, this task was performed by healthy humans. Healthy people’s exploration strategies provided evidence for uncertainty driven exploration. I also showed how different exploratory behaviours can be characterised using canonical correlation analysis. In the next study I showed how exploration of a visual scene under different instructions could be explained by appealing to the computational mechanisms that may correspond to attention. This entailed manipulating the precision of task irrelevant cues and their hidden causes as a function of instructions. In the final work, I was interested in characterising impulsive behaviour using a patch leaving paradigm. By varying the parameters of the MDP model, I showed that there could be at least three distinct causes of impulsive behaviour, namely a lower depth of planning, a lower capacity to maintain and process information, and an increased perceived value of immediate rewards

Subjective estimates of uncertainty during gambling and impulsivity after subthalamic deep brain stimulation for Parkinson\u2019s disease

Subthalamic deep brain stimulation (DBS) for Parkinson\u2019s disease (PD) may modulate chronometric and instrumental aspects of choice behaviour, including motor inhibition, decisional slowing, and value sensitivity. However, it is not well known whether subthalamic DBS affects more complex aspects of decision-making, such as the influence of subjective estimates of uncertainty on choices. In this study, 38 participants with PD played a virtual casino prior to subthalamic DBS (whilst \u2018on\u2019 medication) and again, 3-months postoperatively (whilst \u2018on\u2019 stimulation). At the group level, there was a small but statistically significant decrease in impulsivity postoperatively, as quantified by the Barratt Impulsiveness Scale (BIS). The gambling behaviour of participants (bet increases, slot machine switches and double or nothing gambles) was associated with this self-reported measure of impulsivity. However, there was a large variance in outcome amongst participants, and we were interested in whether individual differences in subjective estimates of uncertainty (specifically, volatility) were related to differences in pre- and postoperative impulsivity. To examine these individual differences, we fit a computational model (the Hierarchical Gaussian Filter, HGF), to choices made during slot machine game play as well as a simpler reinforcement learning model based on the Rescorla-Wagner formalism. The HGF was superior in accounting for the behaviour of our participants, suggesting that participants incorporated beliefs about environmental uncertainty when updating their beliefs about gambling outcome and translating these beliefs into action. A specific aspect of subjective uncertainty, the participant\u2019s estimate of the tendency of the slot machine\u2019s winning probability to change (volatility), increased subsequent to DBS. Additionally, the decision temperature of the response model decreased post-operatively, implying greater stochasticity in the belief-to-choice mapping of participants. Model parameter estimates were significantly associated with impulsivity; specifically, increased uncertainty was related to increased postoperative impulsivity. Moreover, changes in these parameter estimates were significantly associated with the maximum post-operative change in impulsivity over a six month follow up period. Our findings suggest that impulsivity in PD patients may be influenced by subjective estimates of uncertainty (environmental volatility) and implicate a role for the subthalamic nucleus in the modulation of outcome certainty. Furthermore, our work outlines a possible approach to characterising those persons who become more impulsive after subthalamic DBS, an intervention in which non-motor outcomes can be highly variable