Simulation of cholinergic and noradrenergic modulation of behavior in uncertain environments

Attention is a complex neurobiological process that involves rapidly and flexibly balancing sensory input and goal-directed predictions in response to environmental changes. The cholinergic and noradrenergic systems, which have been proposed to respond to expected and unexpected environmental uncertainty, respectively, play an important role in attention by differentially modulating activity in a multitude of cortical targets. Here we develop a model of an attention task that involves expected and unexpected uncertainty. The cholinergic and noradrenergic systems track this uncertainty and, in turn, influence cortical processing in five different, experimentally verified ways: (1) nicotinic enhancement of thalamocortical input, (2) muscarinic regulation of corticocortical feedback, (3) noradrenergic mediation of a network reset, (4) locus coeruleus (LC) activation of the basal forebrain (BF), and (5) cholinergic and noradrenergic balance between sensory input and frontal cortex predictions. Our results shed light on how the noradrenergic and cholinergic systems interact with each other and a distributed set of neural areas, and how this could lead to behavioral adaptation in the face of uncertainty

The influence of the noradrenergic system on optimal control of neural plasticity

Decision making under uncertainty is challenging for any autonomous agent. The challenge increases when the environment’s stochastic properties change over time, i.e., when the environment is volatile. In order to efficiently adapt to volatile environments, agents must primarily rely on recent outcomes to quickly change their decision strategies; in other words, they need to increase their knowledge plasticity. On the contrary, in stable environments, knowledge stability must be preferred to preserve useful information against noise. Here we propose that in mammalian brain, the locus coeruleus (LC) is one of the nuclei involved in volatility estimation and in the subsequent control of neural plasticity. During a reinforcement learning task, LC activation, measured by means of pupil diameter, coded both for environmental volatility and learning rate. We hypothesize that LC could be responsible, through norepinephrinic modulation, for adaptations to optimize decision making in volatile environments. We also suggest a computational model on the interaction between the anterior cingulate cortex (ACC) and LC for volatility estimation

Posterior parietal cortex dynamically ranks topographic signals via cholinergic influence

The hypothesis to be discussed in this review is that posterior parietal cortex (PPC) is directly involved in selecting relevant stimuli and filtering irrelevant distractors. The PPC receives input from several sensory modalities and integrates them in part to direct the allocation of resources to optimize gains. In conjunction with prefrontal cortex, nucleus accumbens, and basal forebrain cholinergic nuclei, it comprises a network mediating sustained attentional performance. Numerous anatomical, neurophysiological, and lesion studies have substantiated the notion that the basic functions of the PPC are conserved from rodents to humans. One such function is the detection and selection of relevant stimuli necessary for making optimal choices or responses. The issues to be addressed here are how behaviorally relevant targets recruit oscillatory potentials and spiking activity of posterior parietal neurons compared to similar yet irrelevant stimuli. Further, the influence of cortical cholinergic input to PPC in learning and decision-making is also discussed. I propose that these neurophysiological correlates of attention are transmitted to frontal cortical areas contributing to the top-down selection of stimuli in a timely manner

The computational neurology of active vision

In this thesis, we appeal to recent developments in theoretical neurobiology – namely, active inference – to understand the active visual system and its disorders. Chapter 1 reviews the neurobiology of active vision. This introduces some of the key conceptual themes around attention and inference that recur through subsequent chapters. Chapter 2 provides a technical overview of active inference, and its interpretation in terms of message passing between populations of neurons. Chapter 3 applies the material in Chapter 2 to provide a computational characterisation of the oculomotor system. This deals with two key challenges in active vision: deciding where to look, and working out how to look there. The homology between this message passing and the brain networks solving these inference problems provide a basis for in silico lesion experiments, and an account of the aberrant neural computations that give rise to clinical oculomotor signs (including internuclear ophthalmoplegia). Chapter 4 picks up on the role of uncertainty resolution in deciding where to look, and examines the role of beliefs about the quality (or precision) of data in perceptual inference. We illustrate how abnormal prior beliefs influence inferences about uncertainty and give rise to neuromodulatory changes and visual hallucinatory phenomena (of the sort associated with synucleinopathies). We then demonstrate how synthetic pharmacological perturbations that alter these neuromodulatory systems give rise to the oculomotor changes associated with drugs acting upon these systems. Chapter 5 develops a model of visual neglect, using an oculomotor version of a line cancellation task. We then test a prediction of this model using magnetoencephalography and dynamic causal modelling. Chapter 6 concludes by situating the work in this thesis in the context of computational neurology. This illustrates how the variational principles used here to characterise the active visual system may be generalised to other sensorimotor systems and their disorders

Different varieties of uncertainty in human decision-making

The study of uncertainty in decision-making is receiving greater attention in the fields of cognitive and computational neuroscience. Several lines of evidence are beginning to elucidate different variants of uncertainty. Particularly, risk, ambiguity, and expected and unexpected forms of uncertainty are well articulated in the literature. In this article we review both empirical and theoretical evidence arguing for the potential distinction between three forms of uncertainty; expected uncertainty, unexpected uncertainty, and volatility. Particular attention will be devoted to exploring the distinction between unexpected uncertainty and volatility which has been less appreciated in the literature. This includes evidence mainly from neuroimaging, neuromodulation, and electrophysiological studies. We further address the possible differentiation of cognitive control mechanisms used to deal with these forms of uncertainty. Finally, we explore whether the dual modes of control theory provides a theoretical framework for understanding the distinction between unexpected uncertainty and volatility

The Neural Representation of Unexpected Uncertainty during Value-Based Decision Making

Uncertainty is an inherent property of the environment and a central feature of models of decision-making and learning. Theoretical propositions suggest that one form, unexpected uncertainty, may be used to rapidly adapt to changes in the environment, while being influenced by two other forms: risk and estimation uncertainty. While previous studies have reported neural representations of estimation uncertainty and risk, relatively little is known about unexpected uncertainty. Here, participants performed a decision-making task while undergoing functional magnetic resonance imaging (fMRI), which, in combination with a Bayesian model-based analysis, enabled us to separately examine each form of uncertainty examined. We found representations of unexpected uncertainty in multiple cortical areas, as well as the noradrenergic brainstem nucleus locus coeruleus. Other unique cortical regions were found to encode risk, estimation uncertainty, and learning rate. Collectively, these findings support theoretical models in which several formally separable uncertainty computations determine the speed of learning

A model of proactive and reactive cognitive control with anterior cingulate cortex and the neuromodulatory system

Abstract Proactive and reactive cognitive control are often associated with anterior cingulate cortex (ACC). How ACC affects processing in other brain areas, however, is often not explicitly delineated. In this work, we describe a model of how ACC computes measures of conflict and surprise that are in turn relayed to the basal forebrain (BF) and locus coeruleus (LC) in that order. BF and LC signals then respectively sharpen posterior cortical processing and trigger the reframing of prefrontal cortical decision-making frames. We implemented this theory in a large-scale neurocognitive model that performs simulated geospatial intelligence tasks. Experiments demonstrate improved performance while minimizing additional processing. Alternate interpretations of neuromodulatory signals are also discussed.

Balancing New Against Old Information: The Role of Surprise

Surprise is a widely used concept describing a range of phenomena from unexpected events to behavioral responses. We propose a measure of surprise, to arrive at a new framework for surprise-driven learning. There are two components to this framework: (i) a confidence-adjusted surprise measure to capture environmental statistics as well as subjective beliefs, (ii) a surprise-minimization learning rule, or SMiLe-rule, which dynamically adjusts the balance between new and old information without making prior assumptions about the temporal statistics of the environment. We apply our framework to a dynamic decision-making task and a maze exploration task to demonstrate that it is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes. Our proposed surprise-modulated belief update algorithm provides a framework to study the behavior of humans and animals encountering surprising events