A Computational Role for Arousal in Optimal Inference

Making accurate predictions is one of the most critical functions of the brain. Whether made by a monkey deciding where to forage, a deer deciding which way to run, or a wall-street broker deciding how to invest, decisions are informed by expectations about possible future outcomes. These expectations are learned over time through experience and are rapidly adjusted when they fail to match observations. Here I propose and support the thesis that learning systems in the brain optimize the accuracy of predictions in a changing world, even though this necessitates becoming insensitive to incoming sensory information under some conditions. Furthermore I propose a biologically inspired model for achieving accurate predictions and suggest a novel role for the arousal system in optimally adjusting the influence of incoming sensory information. I support these theses with a series of experiments that utilize computational modeling, as well as behavioral and pupillometric measurements in humans

Bayesian Online Learning of the Hazard Rate in Change-Point Problems

Change-point models are generative models of time-varying data in which the underlying generative parameters undergo discontinuous changes at different points in time known as change points. Changepoints often represent important events in the underlying processes, like a change in brain state reflected in EEG data or a change in the value of a company reflected in its stock price. However, change-points can be difficult to identify in noisy data streams. Previous attempts to identify change-points online using Bayesian inference relied on specifying in advance the rate at which they occur, called the hazard rate (h). This approach leads to predictions that can depend strongly on the choice of h and is unable to deal optimally with systems in which h is not constant in time. In this letter, we overcome these limitations by developing a hierarchical extension to earlier models. This approach allows h itself to be inferred from the data, which in turn helps to identify when change-points occur. We show that our approach can effectively identify change-points in both toy and real data sets with complex hazard rates and how it can be used as an ideal-observermodel for human and animal behavior when faced with rapidly changing inputs

Functionally Dissociable Influences on Learning Rate in a Dynamic Environment

Maintaining accurate beliefs in a changing environment requires dynamically adapting the rate at which one learns from new experiences. Beliefs should be stable in the face of noisy data but malleable in periods of change or uncertainty. Here we used computational modeling, psychophysics, and fMRI to show that adaptive learning is not a unitary phenomenon in the brain. Rather, it can be decomposed into three computationally and neuroanatomically distinct factors that were evident in human subjects performing a spatial-prediction task: (1) surprise-driven belief updating, related to BOLD activity in visual cortex; (2) uncertainty-driven belief updating, related to anterior prefrontal and parietal activity; and (3) reward-driven belief updating, a context-inappropriate behavioral tendency related to activity in ventral striatum. These distinct factors converged in a core system governing adaptive learning. This system, which included dorsomedial frontal cortex, responded to all three factors and predicted belief updating both across trials and across individuals

Computational neuroscience across the lifespan: Promises and pitfalls

In recent years, the application of computational modeling in studies on age-related changes in decision making and learning has gained in popularity. One advantage of computational models is that they provide access to latent variables that cannot be directly observed from behavior. In combination with experimental manipulations, these latent variables can help to test hypotheses about age-related changes in behavioral and neurobiological measures at a level of specificity that is not achievable with descriptive analysis approaches alone. This level of specificity can in turn be beneficial to establish the identity of the corresponding behavioral and neurobiological mechanisms. In this paper, we will illustrate applications of computational methods using examples of lifespan research on risk taking, strategy selection and reinforcement learning. We will elaborate on problems that can occur when computational neuroscience methods are applied to data of different age groups. Finally, we will discuss potential targets for future applications and outline general shortcomings of computational neuroscience methods for research on human lifespan development

Age differences in learning emerge from an insufficient representation of uncertainty in older adults

Healthy aging can lead to impairments in learning that affect many laboratory and real-life tasks. These tasks often involve the acquisition of dynamic contingencies, which requires adjusting the rate of learning to environmental statistics. For example, learning rate should increase when expectations are uncertain (uncertainty), outcomes are surprising (surprise) or contingencies are more likely to change (hazard rate). In this study, we combine computational modelling with an age-comparative behavioural study to test whether age-related learning deficits emerge from a failure to optimize learning according to the three factors mentioned above. Our results suggest that learning deficits observed in healthy older adults are driven by a diminished capacity to represent and use uncertainty to guide learning. These findings provide insight into age-related cognitive changes and demonstrate how learning deficits can emerge from a failure to accurately assess how much should be learned

Computational mechanisms of belief updating in relation to psychotic-like experiences

Introduction Psychotic-like experiences (PLEs) may occur due to changes in weighting prior beliefs and new evidence in the belief updating process. It is still unclear whether the acquisition or integration of stable beliefs is altered, and whether such alteration depends on the level of environmental and belief precision, which reflects the associated uncertainty. This motivated us to investigate uncertainty-related dynamics of belief updating in relation to PLEs using an online study design. Methods We selected a sample (n = 300) of participants who performed a belief updating task with sudden change points and provided self-report questionnaires for PLEs. The task required participants to observe bags dropping from a hidden helicopter, infer its position, and dynamically update their belief about the helicopter's position. Participants could optimize performance by adjusting learning rates according to inferred belief uncertainty (inverse prior precision) and the probability of environmental change points. We used a normative learning model to examine the relationship between adherence to specific model parameters and PLEs. Results PLEs were linked to lower accuracy in tracking the outcome (helicopter location) (β = 0.26 ± 0.11, p = 0.018) and to a smaller increase of belief precision across observations after a change point (β = −0.003 ± 0.0007, p < 0.001). PLEs were related to slower belief updating when participants encountered large prediction errors (β = −0.03 ± 0.009, p = 0.001). Computational modeling suggested that PLEs were associated with reduced overall belief updating in response to prediction errors (βPE = −1.00 ± 0.45, p = 0.028) and reduced modulation of updating at inferred environmental change points (βCPP = −0.84 ± 0.38, p = 0.023). Discussion We conclude that PLEs are associated with altered dynamics of belief updating. These findings support the idea that the process of balancing prior belief and new evidence, as a function of environmental uncertainty, is altered in PLEs, which may contribute to the development of delusions. Specifically, slower learning after large prediction errors in people with high PLEs may result in rigid beliefs. Disregarding environmental change points may limit the flexibility to establish new beliefs in the face of contradictory evidence. The present study fosters a deeper understanding of inferential belief updating mechanisms underlying PLEs.Peer Reviewe

Computational mechanisms of belief updating in relation to psychotic-like experiences

Introduction: Psychotic-like experiences (PLEs) may occur due to changes in weighting prior beliefs and new evidence in the belief updating process. It is still unclear whether the acquisition or integration of stable beliefs is altered, and whether such alteration depends on the level of environmental and belief precision, which reflects the associated uncertainty. This motivated us to investigate uncertainty-related dynamics of belief updating in relation to PLEs using an online study design. Methods: We selected a sample (n = 300) of participants who performed a belief updating task with sudden change points and provided self-report questionnaires for PLEs. The task required participants to observe bags dropping from a hidden helicopter, infer its position, and dynamically update their belief about the helicopter's position. Participants could optimize performance by adjusting learning rates according to inferred belief uncertainty (inverse prior precision) and the probability of environmental change points. We used a normative learning model to examine the relationship between adherence to specific model parameters and PLEs. Results: PLEs were linked to lower accuracy in tracking the outcome (helicopter location) (beta = 0.26 +/- 0.11, p = 0.018) and to a smaller increase of belief precision across observations after a change point (beta = -0.003 +/- 0.0007, p < 0.001). PLEs were related to slower belief updating when participants encountered large prediction errors (beta = -0.03 +/- 0.009, p = 0.001). Computational modeling suggested that PLEs were associated with reduced overall belief updating in response to prediction errors (beta(PE) = -1.00 +/- 0.45, p = 0.028) and reduced modulation of updating at inferred environmental change points (beta(CPP) = -0.84 +/- 0.38, p = 0.023). Discussion: We conclude that PLEs are associated with altered dynamics of belief updating. These findings support the idea that the process of balancing prior belief and new evidence, as a function of environmental uncertainty, is altered in PLEs, which may contribute to the development of delusions. Specifically, slower learning after large prediction errors in people with high PLEs may result in rigid beliefs. Disregarding environmental change points may limit the flexibility to establish new beliefs in the face of contradictory evidence. The present study fosters a deeper understanding of inferential belief updating mechanisms underlying PLEs

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits