Dopamine, affordance and active inference.

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level

Hierarchical Active Inference: A Theory of Motivated Control

Motivated control refers to the coordination of behaviour to achieve affectively valenced outcomes or goals. The study of motivated control traditionally assumes a distinction between control and motivational processes, which map to distinct (dorsolateral versus ventromedial) brain systems. However, the respective roles and interactions between these processes remain controversial. We offer a novel perspective that casts control and motivational processes as complementary aspects − goal propagation and prioritization, respectively − of active inference and hierarchical goal processing under deep generative models. We propose that the control hierarchy propagates prior preferences or goals, but their precision is informed by the motivational context, inferred at different levels of the motivational hierarchy. The ensuing integration of control and motivational processes underwrites action and policy selection and, ultimately, motivated behaviour, by enabling deep inference to prioritize goals in a context-sensitive way

Embodying addiction: a predictive processing account

In this paper we show how addiction can be thought of as the outcome of learning. We look to the increasingly influential predictive processing theory for an account of how learning can go wrong in addiction. Perhaps counter intuitively, it is a consequence of this predictive processing perspective on addiction that while the brain plays a deep and important role in leading a person into addiction, it cannot be the whole story. We’ll argue that predictive processing implies a view of addiction not as a brain disease, but rather as a breakdown in the dynamics of the wider agent-environment system. The environment becomes meaningfully organised around the agent’s drug-seeking and using behaviours. Our account of addiction offers a new perspective on what is harmful about addiction. Philosophers often characterise addiction as a mental illness because addicts irrationally shift in their judgement of how they should act based on cues that predict drug use. We argue that predictive processing leads to a different view of what can go wrong in addiction. We suggest that addiction can prove harmful to the person because as their addiction progressively takes hold, the addict comes to embody a predictive model of the environment that fails to adequately attune them to a volatile, dynamic environment. The use of an addictive substance produces illusory feedback of being well-attuned to the environment when the reality is the opposite. This can be comforting for a person inhabiting a hostile niche, but it can also prove to be harmful to the person as they become skilled at living the life of an addict, to the neglect of all other alternatives. The harm in addiction we’ll argue is not to be found in the brains of addicts, but in their way of life

Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology

How mood tunes prediction: a neurophenomenological account of mood and its disturbance in major depression

In this article, we propose a neurophenomenological account of what moods are, and how they work. We draw upon phenomenology to show how mood attunes a person to a space of significant possibilities. Mood structures a person’s lived experience by fixing the kinds of significance the world can have for them in a given situation. We employ Karl Friston’s free-energy principle to show how this phenomenological concept of mood can be smoothly integrated with cognitive neuroscience. We will argue that mood is a consequence of acting in the world with the aim of minimizing expected free energy—a measure of uncertainty about the future consequences of actions. Moods summarize how the organism is faring overall in its predictive engagements, tuning the organism’s expectations about how it is likely to fare in the future. Agents that act to minimize expected free energy will have a feeling of how well or badly they are doing at maintaining grip on the multiple possibilities that matter to them. They will have what we will call a ‘feeling of grip’ that structures the possibilities they are ready to engage with over long time-scales, just as moods do

The Problem of Mental Action

In mental action there is no motor output to be controlled and no sensory input vector that could be manipulated by bodily movement. It is therefore unclear whether this specific target phenomenon can be accommodated under the predictive processing framework at all, or if the concept of “active inference” can be adapted to this highly relevant explanatory domain. This contribution puts the phenomenon of mental action into explicit focus by introducing a set of novel conceptual instruments and developing a first positive model, concentrating on epistemic mental actions and epistemic self-control. Action initiation is a functionally adequate form of self-deception; mental actions are a specific form of predictive control of effective connectivity, accompanied and possibly even functionally mediated by a conscious “epistemic agent model”. The overall process is aimed at increasing the epistemic value of pre-existing states in the conscious self-model, without causally looping through sensory sheets or using the non-neural body as an instrument for active inference

Bayesian Learning Models of Pain: A Call to Action

Learning is fundamentally about action, enabling the successful navigation of a changing and uncertain environment. The experience of pain is central to this process, indicating the need for a change in action so as to mitigate potential threat to bodily integrity. This review considers the application of Bayesian models of learning in pain that inherently accommodate uncertainty and action, which, we shall propose are essential in understanding learning in both acute and persistent cases of pain

Active interoceptive inference and the emotional brain

We review a recent shift in conceptions of interoception and its relationship to hierarchical inference in the brain. The notion of interoceptive inference means that bodily states are regulated by autonomic reflexes that are enslaved by descending predictions from deep generative models of our internal and external milieu. This re-conceptualization illuminates several issues in cognitive and clinical neuroscience with implications for experiences of selfhood and emotion. We first contextualize interoception in terms of active (Bayesian) inference in the brain, highlighting its enactivist (embodied) aspects. We then consider the key role of uncertainty or precision and how this might translate into neuromodulation. We next examine the implications for understanding the functional anatomy of the emotional brain, surveying recent observations on agranular cortex. Finally, we turn to theoretical issues, namely, the role of interoception in shaping a sense of embodied self and feelings. We will draw links between physiological homoeostasis and allostasis, early cybernetic ideas of predictive control and hierarchical generative models in predictive processing. The explanatory scope of interoceptive inference ranges from explanations for autism and depression, through to consciousness. We offer a brief survey of these exciting developments

An aberrant precision account of autism.

Autism is a neurodevelopmental disorder characterized by problems with social-communication, restricted interests and repetitive behavior. A recent and thought-provoking article presented a normative explanation for the perceptual symptoms of autism in terms of a failure of Bayesian inference (Pellicano and Burr, 2012). In response, we suggested that when Bayesian inference is grounded in its neural instantiation-namely, predictive coding-many features of autistic perception can be attributed to aberrant precision (or beliefs about precision) within the context of hierarchical message passing in the brain (Friston et al., 2013). Here, we unpack the aberrant precision account of autism. Specifically, we consider how empirical findings-that speak directly or indirectly to neurobiological mechanisms-are consistent with the aberrant encoding of precision in autism; in particular, an imbalance of the precision ascribed to sensory evidence relative to prior beliefs

All Thinking is 'Wishful' Thinking

Motivation to engage in any epistemic behavior can be decomposed into two basic types that emerge in various guises across different disciplines and areas of study. The first basic dimension refers to a desire to approach versus avoid nonspecific certainty, which has epistemic value. It describes a need for an unambiguous, precise answer to a question, regardless of that answer’s specific content. Second basic dimension refers to a desire to approach versus avoid specific certainty, which has instrumental value. It concerns a need for the specific content of one’s beliefs and prior preferences. Together, they explain diverse epistemic behaviors, such as seeking, avoiding, and biasing new information and revising and updating, versus protecting, one’s beliefs, when confronted with new evidence. The relative strength of these motivational components determines the form of (Bayes optimal) epistemic behavior that follows