A Neurocomputational Model of the Functional Role of Dopamine in Stimulus-Response Task Learning and Performance

Thesis (Ph.D.) - Indiana University, Psychology, 2009The neuromodulatory neurotransmitter dopamine (DA) plays a complex, but central role in the learning and performance of stimulus-response (S-R) behaviors. Studies have implicated DA's role in reward-driven learning and also its role in setting the overall level of vigor or frequency of response. Here, a neurocomputational model is developed which models DA's influence on a set of brain regions believed to be involved in the learning and execution of S-R tasks, including frontal cortex, basal ganglia, and cingulate cortex. An `actor' component of the model is trained, using `babble' (random behavior selection) and `critic' (rewarding and punishing) components of the model, to perform acceptance/rejection responses upon presentation of color stimuli in the context of recently presented auditory tones. The model behaves like an autonomous organism learning (and relearning) through `trial-and-error'. The focus of the study, the impact of hypo- and hyper-normal DA activity on this model, is investigated by three different dopaminergic pathways--two striatal and one prefrontal cortical--being manipulated independently during the learning and performance of the color response task. Hypo-DA conditions, analogous to Parkinsonism, cause slowing and reduction of frequency of learned responses, and, at extremes, degrade the learning (either initial or reversal) of the task. Hyper-DA conditions, analogous to psychostimulant effects, cause more rapid response times, but also can lead to perseveration of incorrect learning of response on the task. The presence of these effects often depends on which DA-ergic pathway is manipulated, however, which has implications for interpretation of the pharmacological experimental data. The proposed model embodies an integrative theory of dopamine function which suggests that the base rate of DA cell activity encodes the overall `activity-oriented motivation' of the organism, with hunger and/or expectation of reward driving both response vigor and tendency to generate an explorative `babble' response. This more `tonic' feature of DA functionality coexists naturally with the more extensively-studied `phasic' reward-learning features. The model may provide better insights on the role of DA system dysfunction in the cognitive and motivational symptoms of disorders such as Parkinsonism, psychostimulant abuse, ADHD, OCD, and schizophrenia, accounting for deficits in both learning and performance of tasks

Dopamine precursor depletion improves punishment prediction during reversal learning in healthy females but not males

Contains fulltext : 88802.pdf (publisher's version ) (Closed access

Complying with norms. a neurocomputational exploration

The subject matter of this thesis can be summarized by a triplet of questions and answers. Showing what these questions and answers mean is, in essence, the goal of my project. The triplet goes like this: Q: How can we make progress in our understanding of social norms and norm compliance? A: Adopting a neurocomputational framework is one effective way to make progress in our understanding of social norms and norm compliance. Q: What could the neurocomputational mechanism of social norm compliance be? A: The mechanism of norm compliance probably consists of Bayesian - Reinforcement Learning algorithms implemented by activity in certain neural populations. Q: What could information about this mechanism tell us about social norms and social norm compliance? A: Information about this mechanism tells us that: a1: Social norms are uncertainty-minimizing devices. a2: Social norm compliance is one trick that agents employ to interact coadaptively and smoothly in their social environment. Most of the existing treatments of norms and norm compliance (e.g. Bicchieri 2006; Binmore 1993; Elster 1989; Gintis 2010; Lewis 1969; Pettit 1990; Sugden 1986; Ullmann‐Margalit 1977) consist in what Cristina Bicchieri (2006) refers to as “rational reconstructions.” A rational reconstruction of the concept of social norm “specifies in which sense one may say that norms are rational, or compliance with a norm is rational” (Ibid., pp. 10-11). What sets my project apart from these types of treatments is that it aims, first and foremost, at providing a description of some core aspects of the mechanism of norm compliance. The single most original idea put forth in my project is to bring an alternative explanatory framework to bear on social norm compliance. This is the framework of computational cognitive neuroscience. The chapters of this thesis describe some ways in which central issues concerning social norms can be fruitfully addressed within a neurocomputational framework. In order to qualify and articulate the triplet above, my strategy consists firstly in laying down the beginnings of a model of the mechanism of norm compliance behaviour, and then zooming in on specific aspects of the model. Such a model, the chapters of this thesis argue, explains apparently important features of the psychology and neuroscience of norm compliance, and helps us to understand the nature of the social norms we live by

Functional implications of dopamine D1 vs. D2 receptors: A ‘prepare and select’ model of the striatal direct vs. indirect pathways

AbstractThe functions of the D1- and D2-dopamine receptors in the basal ganglia have remained somewhat enigmatic, with a number of competing theories relating to the interactions of the ‘direct’ and ‘indirect pathways’. Computational models have been good at simulating properties of the system, but are typically divorced from the underlying neural architecture. In this article we propose a new model which re-addresses response selection at the level of the basal ganglia. At the core of this response selection system the D1 DA receptor-expressing striatal pathways ‘prepare’ the set of possible appropriate responses. The D2DR-expressing striatal pathways then shape and ‘select’ from this initial response set framework.This article is part of a Special Issue entitled: Ventral Tegmentum & Dopamine

Lost in Translation? On the Need for Convergence in Animal and Human Studies on the Role of Dopamine in Diet-Induced Obesity

Purpose of Review: Animal and human studies suggest that diet-induced obesity and plasticity in the central dopaminergic system are linked. However, it is unclear whether observed changes depend on diet or obesity, and whether they are specific to brain regions and cognitive functions. Here, we focus on neural and cognitive changes in frontostriatal circuits. Recent Findings: Both diet and obesity affect dopaminergic transmission. However, site and direction of effects are inconsistent across species and studies. Non-specific changes are observed spanning all frontostriatal loops, from sensory input to motivated behaviour. Given the impact of peripheral signals on central dopaminergic signalling and the interaction between the frontostriatal loops, modulation of dopamine likely propagates through all loops and, thus, affects behaviour on various levels of complexity. Summary: To improve convergence between animal and human studies on diet-induced obesity, animal studies should include sophisticated cognitive measures and diets resembling human obesogenic diets, and human studies should adopt diet interventions and longitudinal designs.Peer reviewe

Obesity is associated with insufficient behavioral adaptation

Obesity is one of the major health concerns nowadays according to the World Health Organisation (WHO global status report on noncommunicable diseases 2010). Thus, there is an urgent need for understanding obesity-associated alterations in food-related and general cognition and their underlying structural and functional correlates within the central nervous system (CNS). Neuroscientific research of the past decade has mainly focussed on obesity-related differences within homeostatic and hedonic processing of food stimuli. Therein, alterations during anticipation and consumption of food-reward stimuli in obese compared with lean subjects have been highlighted. This points at an altered adaptation of eating behavior in obese individuals. This thesis investigates if adaptation of behavior is attenuated in obese compared to lean individuals in learning-related processes beyond the food domain. In five consecutive experimental studies, we show that obese participants reveal reduced adaptation of behavior within and outside the food context. With the help of MRI, we relate these behavioral findings to alterations in structure and function of the fronto-striatal dopaminergic system in obesity. In more detail, reduced behavioral adaptation seems to be associated with attenuated utilization of negative prediction errors in obese individuals. Within the brain, this relates to reduced functional coupling between subcortical dopaminergic target regions (ventral striatum) and executive cortical structures (supplementary motor area) in obesity, as revealed by fMRI analysis

Developing an fMRI paradigm for studying reinforcement learning with gustatory stimuli

One of the main challenges for global public health in the modern world is the rising prevalence of obesity. Obtaining a better understanding of the dysregulated feeding behaviour that leads to obesity, by investigating the decision making and learning processes underlying it, could advance our capabilities in battling the obesity epidemic. Consequently, our aim in this study is to design an experiment that could evaluate these processes. We examined ten healthy participants using a modified version of the "probabilistic selection task". We used gustatory stimuli as a replacement for monetary rewards, to assess the effect of nutritional rewards on the learning behaviour. We subsequently analysed the behavioural results with computational modelling and combined this with imaging data simultaneously acquired with a functional magnetic resonance imaging (fMRI) multiband sequence. All participants in this study succeeded in interpreting and interacting with the gustatory stimuli appropriately. Performance on the task was affected by the subjective valuation of the reward. Participants whose motivation to drink the reward and liking of its taste decreased during the task presented difficulties correctly choosing the more rewarding cues. Computational modelling of the behaviour found that the so-called asymmetric learning model, in which positive and negative reinforcement are differently weighted, best explained the group. The acquired fMRI data was suboptimal and we did not detect the neurological activity we expected in the reward system, which is central to our scientific question. Thus, our study shows it is possible to implement the PST with gustatory stimuli. However, to evaluate the corresponding neurological activity, our fMRI configuration requires improvement. An optimised system could be used in further studies to improve our understanding of the neurobiological mechanisms of learning that lead to obesity and elucidate the role of food as a distinctive reinforcer

Computational Models of Interoception and Body Regulation

To survive, organisms must effectively respond to the challenge of maintaining their physiological integrity in the face of an ever-changing environment. Preserving this homeostasis critically relies on adaptive behavior. In this review, we consider recent frameworks that extend classical homeostatic control via reflex arcs to include more flexible forms of adaptive behavior that take interoceptive context, experiences, and expectations into account. Specifically, we define a landscape for computational models of interoception, body regulation, and forecasting, address these models' unique challenges in relation to translational research efforts, and discuss what they can teach us about cognition as well as physical and mental health