Dopamine, affordance and active inference.

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level

Bayesian Learning Models of Pain: A Call to Action

Learning is fundamentally about action, enabling the successful navigation of a changing and uncertain environment. The experience of pain is central to this process, indicating the need for a change in action so as to mitigate potential threat to bodily integrity. This review considers the application of Bayesian models of learning in pain that inherently accommodate uncertainty and action, which, we shall propose are essential in understanding learning in both acute and persistent cases of pain

Computational psychiatry: a Rosetta Stone linking the brain to mental illness.

PCF is supported by the Bernard Wolfe Health Neuroscience Fund and the Wellcome Trust. This work was carried out within the Wellcome- and MRC-funded Behavioural and Clinical Neuroscience Institute and the Cambridge and Peterborough NHS Foundation Trust.This is the accepted manuscript. The final version is available from the Lancet Psychiatry at: http://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366%2814%2970298-6/fulltex

Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior--in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology

Changing perspective on perception physiology: Can you really see what is happening?

Perception is a complex, neural mechanism that requires organization and interpretation of input meaning and it has been a key topic in medicine, neuroscience and philosophy for centuries. Gestalt psychology proposed that the underlying mechanism is a constructive process that depends on both input of stimuli and the sensory-motor state of the agent. The Bayesian Brain hypothesis reframed it as probabilistic inference of previous beliefs, which are revised to accommodate new information. The Predictive Coding Theory proposes that this process is implemented through a top-down cascade of cortical predictions of lower level input and the concurrent propagation of a bottom-up prediction error aimed at revising higher level expectations. The „Active Inference‟ theory explains both perception and action, generalising the prediction error minimisation process. In this focused-review we provide a historical overview of the topic and an intuitive approach to the new computational models

Social affordances in context: What is it that we are bodily responsive to

We propose to understand social affordances in the broader context of responsiveness to a field of relevant affordances in general. This perspective clarifies our everyday ability to unreflectively switch between social and other affordances. Moreover, based on our experience with Deep Brain Stimulation for treating obsessive-compulsive disorder (OCD) patients, we suggest that psychiatric disorders may affect skilled intentionality, including responsiveness to social affordance

Bayesian Learning Models of Pain:A Call to Action

Learning is fundamentally about action, enabling the successful navigation of a changing and uncertain environment. The experience of pain is central to this process, indicating the need for a change in action so as to mitigate potential threat to bodily integrity. This review considers the application of Bayesian models of learning in pain that inherently accommodate uncertainty and action, which, we shall propose are essential in understanding learning in both acute and persistent cases of pain.</p

Predictive Coding Strategies for Developmental Neurorobotics

In recent years, predictive coding strategies have been proposed as a possible means by which the brain might make sense of the truly overwhelming amount of sensory data available to the brain at any given moment of time. Instead of the raw data, the brain is hypothesized to guide its actions by assigning causal beliefs to the observed error between what it expects to happen and what actually happens. In this paper, we present a variety of developmental neurorobotics experiments in which minimalist prediction error-based encoding strategies are utilize to elucidate the emergence of infant-like behavior in humanoid robotic platforms. Our approaches will be first naively Piagian, then move onto more Vygotskian ideas. More specifically, we will investigate how simple forms of infant learning, such as motor sequence generation, object permanence, and imitation learning may arise if minimizing prediction errors are used as objective functions

Linking unfounded beliefs to genetic dopamine availability

Unfounded convictions involving beliefs in the paranormal, grandiosity ideas or suspicious thoughts are endorsed at varying degrees among the general population. Here, we investigated the neurobiopsychological basis of the observed inter-individual variability in the propensity toward unfounded beliefs. One hundred two healthy individuals were genotyped for four polymorphisms in the COMT gene (rs6269, rs4633, rs4818, and rs4680, also known as val158met) that define common functional haplotypes with substantial impact on synaptic dopamine degradation, completed a questionnaire measuring unfounded beliefs, and took part in a behavioral experiment assessing perceptual inference. We found that greater dopamine availability was associated with a stronger propensity toward unfounded beliefs, and that this effect was statistically mediated by an enhanced influence of expectations on perceptual inference. Our results indicate that genetic differences in dopaminergic neurotransmission account for inter-individual differences in perceptual inference linked to the formation and maintenance of unfounded beliefs. Thus, dopamine might be critically involved in the processes underlying one's interpretation of the relationship between the self and the world

Sensory attenuation in Parkinson’s disease is related to disease severity and dopamine dose

Abnormal initiation and control of voluntary movements are among the principal manifestations of Parkinson’s disease (PD). However, the processes underlying these abnormalities and their potential remediation by dopamine treatment remain poorly understood. Normally, movements depend on the integration of sensory information with the predicted consequences of action. This integration leads to a suppression in the intensity of predicted sensations, reflected in a ‘sensory attenuation’. We examined this integration process and its relation to dopamine in PD, by measuring sensory attenuation. Patients with idiopathic PD (n = 18) and population-derived controls (n = 175) matched a set of target forces applied to their left index finger by a torque motor. To match the force, participants either pressed with their right index finger (‘Direct’ condition) or moved a knob that controlled a motor through a linear potentiometer (‘Slider’ condition). We found that despite changes in sensitivity to different forces, overall sensory attenuation did not differ between medicated PD patients and controls. Importantly, the degree of attenuation was negatively related to PD motor severity but positively related to individual patient dopamine dose, as measured by levodopa dose equivalent. The results suggest that dopamine could regulate the integration of sensorimotor prediction with sensory information to facilitate the control of voluntary movements