Independent circuits in basal ganglia and cortex for the processing of reward and precision feedback

In order to understand human decision making it is necessary to understand how the brain uses feedback to guide goal-directed behavior. The ventral striatum (VS) appears to be a key structure in this function, responding strongly to explicit reward feedback. However, recent results have also shown striatal activity following correct task performance even in the absence of feedback. This raises the possibility that, in addition to processing external feedback, the dopamine-centered reward circuit might regulate endogenous reinforcement signals, like those triggered by satisfaction in accurate task performance. Here we use functional magnetic resonance imaging (fMRI) to test this idea. Participants completed a simple task that garnered both reward feedback and feedback about the precision of performance. Importantly, the design was such that we could manipulate information about the precision of performance within different levels of reward magnitude. Using parametric modulation and functional connectivity analysis we identified brain regions sensitive to each of these signals. Our results show a double dissociation: frontal and posterior cingulate regions responded to explicit reward but were insensitive to task precision, whereas the dorsal striatum - and putamen in particular - was insensitive to reward but responded strongly to precision feedback in reward-present trials. Both types of feedback activated the VS, and sensitivity in this structure to precision feedback was predicted by personality traits related to approach behavior and reward responsiveness. Our findings shed new light on the role of specific brain regions in integrating different sources of feedback to guide goal-directed behavior

Many hats: intra-trial and reward-level dependent bold activity in the striatum and premotor cortex

2012 Spring.Includes bibliographical references.Lesion, drug, single-cell recording, as well as human fMRI studies, suggest dopaminergic projections from VTA/SNc (ventral tagmental area/substantia nigra pars compacta) and cortically driven striatal activity plays a key role in associating sensory events with rewarding actions both by facilitating reward processing and prediction (i.e. reinforcement learning) and biasing and later updating action selection. We, for the first time, isolated BOLD signal changes for stimulus, pre-response, response and feedback delivery at three reward levels. This design allowed us to estimate the degree of involvement of individual striatal regions across these trial components, the reward sensitivity of each component and allowed for a novel comparison of potential (and potentially competing) reinforcement learning computations. Striatal and lateral premotor cortex regions of interest (ROIs) significant activations were universally observed (excepting the ventral striatum) during stimulus presentation, pre-response, response and feedback delivery, confirming these areas importance in all aspects of visuomotor learning. The head of the caudate showed a precipitous drop in activity pre-response, while in the body of the caudate showed no significant changes in activity. The putamen peaked in activity during response. Activation in the lateral premotor cortex was strongest during stimulus presentation, but the drop off was followed by a trend of increasing activity as feedback approached. Both the head and body of the caudate as well as the putamen displayed reward-level sensitivity only during stimulus, while the ventral striatum showed reward sensitivity at both stimulus and feedback. The lack of reward sensitivity surrounding response is inconsistent with theories that the head and ventral striatum encode the value of actions. Which of the three examined reinforcement learning models correlated best with BOLD signal changes varied as a function of trial component and ROI suggesting these regions computations vary depending on task demand

How may the basal ganglia contribute to auditory categorization and speech perception?

Listeners must accomplish two complementary perceptual feats in extracting a message from speech. They must discriminate linguistically-relevant acoustic variability and generalize across irrelevant variability. Said another way, they must categorize speech. Since the mapping of acoustic variability is language-specific, these categories must be learned from experience. Thus, understanding how, in general, the auditory system acquires and represents categories can inform us about the toolbox of mechanisms available to speech perception. This perspective invites consideration of findings from cognitive neuroscience literatures outside of the speech domain as a means of constraining models of speech perception. Although neurobiological models of speech perception have mainly focused on cerebral cortex, research outside the speech domain is consistent with the possibility of significant subcortical contributions in category learning. Here, we review the functional role of one such structure, the basal ganglia. We examine research from animal electrophysiology, human neuroimaging, and behavior to consider characteristics of basal ganglia processing that may be advantageous for speech category learning. We also present emerging evidence for a direct role for basal ganglia in learning auditory categories in a complex, naturalistic task intended to model the incidental manner in which speech categories are acquired. To conclude, we highlight new research questions that arise in incorporating the broader neuroscience research literature in modeling speech perception, and suggest how understanding contributions of the basal ganglia can inform attempts to optimize training protocols for learning non-native speech categories in adulthood

A Comparison of the neural correlates that underlie rule-based and information-integration category learning

The influential Competition between Verbal and Implicit Systems (COVIS) model proposes that category learning is driven by two competing neural systems – an explicit, verbal, system, and a procedural-based, implicit, system. In the current fMRI study, participants learned either a conjunctive, rule-based, category structure that is believed to engage the explicit system, or an information-integration category structure that is thought to preferentially recruit the implicit system. The rule-based and information-integration category structures were matched for participant error rate, the number of relevant stimulus dimensions and category separation. Under these conditions, considerable overlap in brain activation, including the prefrontal cortex, basal ganglia, and the hippocampus, was found between the rule-based and information-integration category structures. Contrary to the predictions of COVIS, the medial temporal lobes and in particular the hippocampus, key regions for explicit memory, were found to be more active in the information-integration condition than in the rule-based condition. No regions were more activated in rule-based than information-integration category learning. The implications of these results for theories of category learning are discussed.The support of a South West Doctoral Training Centre (SWDTC) Economic and Social Research Council (ESRC) Studentship Award (ES/J50015X/1) to the first author is appreciatively acknowledged. We also thank Todd Maddox for supplying the stimuli used in this study and Greg Ashby for his comments on this work. The participation of University of Exeter student volunteers is also greatly appreciated

Dopaminergic and Non-Dopaminergic Value Systems in Conditioning and Outcome-Specific Revaluation

Animals are motivated to choose environmental options that can best satisfy current needs. To explain such choices, this paper introduces the MOTIVATOR (Matching Objects To Internal Values Triggers Option Revaluations) neural model. MOTIVATOR describes cognitiveemotional interactions between higher-order sensory cortices and an evaluative neuraxis composed of the hypothalamus, amygdala, and orbitofrontal cortex. Given a conditioned stimulus (CS), the model amygdala and lateral hypothalamus interact to calculate the expected current value of the subjective outcome that the CS predicts, constrained by the current state of deprivation or satiation. The amygdala relays the expected value information to orbitofrontal cells that receive inputs from anterior inferotemporal cells, and medial orbitofrontal cells that receive inputs from rhinal cortex. The activations of these orbitofrontal cells code the subjective values of objects. These values guide behavioral choices. The model basal ganglia detect errors in CS-specific predictions of the value and timing of rewards. Excitatory inputs from the pedunculopontine nucleus interact with timed inhibitory inputs from model striosomes in the ventral striatum to regulate dopamine burst and dip responses from cells in the substantia nigra pars compacta and ventral tegmental area. Learning in cortical and striatal regions is strongly modulated by dopamine. The model is used to address tasks that examine food-specific satiety, Pavlovian conditioning, reinforcer devaluation, and simultaneous visual discrimination. Model simulations successfully reproduce discharge dynamics of known cell types, including signals that predict saccadic reaction times and CS-dependent changes in systolic blood pressure.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Institutes of Health (R29-DC02952, R01-DC007683); National Science Foundation (IIS-97-20333, SBE-0354378); Office of Naval Research (N00014-01-1-0624

Neuroimaging in Human Category Learning: A Comparison Between Functional Near-Infrared Spectroscopy (fNIR) and Functional Magnetic Resonance Imaging (fMRI)

The objective of this thesis is to examine the validity of functional near-infrared spectroscopy (fNIR) to examine brain regions involved in rule based (RB) and information integration (II) category learning. We predicted similar patterns of activation found by past studies that used fMRI scans. Our goal was to test if fNIR would be able to detect changes in blood oxygenation levels of participants who learned to categorize (learners) vs. those that did not (non learners). The stimulus set comprised of lines that differed in length and orientation. Participants had to learn to categorize by trial and error based on the feedback provided. Behavioral and neuroimaging data was recorded for both RB and II conditions. Results showed an upward trend in response accuracy over trials for participants identified as learners. Furthermore, blood oxygenation levels reported by fNIR indicated a systematic increase in oxygen consumption for learners as compared to non learners. These areas of increased prefrontal cortex activity recorded by fNIR correspond to the same areas found to be involved in categorization by fMRI. This paper reviews the background of category learning, explores various neuroimaging techniques in categorization research, and investigates the efficacy of fNIR as a relatively new neuroimaging modality by comparing it to fMRI

Animal models of compulsive eating behavior

In industrialized nations, overeating is a significant problem leading to overweight, obesity, and a host of related disorders; the increase in these disorders has prompted a significant amount of research aimed at understanding their etiology. Eating disorders are multifactorial conditions involving genetic, metabolic, environmental, and behavioral factors. Considering that compulsive eating in the face of adverse consequences characterizes some eating disorders, similar to the way in which compulsive drug intake characterizes drug-addiction, it might be considered an addiction in its own right. Moreover, numerous review articles have recently drawn a connection between the neural circuits activated in the seeking/intake of palatable food and drugs of abuse. Based on this observation, “food addiction” has emerged as an area of intense scientific research and accumulating evidence suggests it is possible to model some aspects of food addiction in animals. The development of well-characterized animal models would advance our understanding of the etiologic neural factors involved in eating disorders, such as compulsive overeating, and it would permit to propose targeted pharmacological therapies. However, to date, little evidence has been reported of continued food seeking and intake despite its harmful consequences in rats and mice

Role Of The Dorsal Striatum In Learning and Decision Making

The striatum, the input region of the basal ganglia, has been shown to mediate many cognitive functions. The striatum itself can be functionally segregated into dorsal (DS) and ventral striatum (VS). For more than 60 years, DS has been reported to mediate stimulus-response learning, though evidence has been accruing pointing to a role in decision making. These literatures have been growing independently and an aim of this thesis was to bridge these two bodies of knowledge. We directly investigated the role of DS in stimulus-response learning versus decision making using functional magnetic resonance imaging (fMRI) in patients with Parkinson’s disease (Chapter 2) and obsessive compulsive disorder (Chapter 3). In Chapter 4, the role of DS in stimulus-response habit learning was tested in healthy individuals using fMRI. In three separate experiments (Chapters 2-4), all of the results strongly support the notion that DS mediates decision making and not learning. DS is implicated in many disorders ranging from Parkinson’s disease, obsessive compulsive disorder and addiction, and clarifying the role of DS in cognitive function is paramount for understanding substrates of disease and developing treatments

Are Bipolar Disorder and Schizophrenia Neuroanatomically Distinct? An Anatomical Likelihood Meta-analysis

Objective: There is renewed debate on whether modern diagnostic classification should adopt a dichotomous or dimensional approach to schizophrenia and bipolar disorder. This study synthesizes data from voxel-based studies of schizophrenia and bipolar disorder to estimate the extent to which these conditions have a common neuroanatomical phenotype. Methods: A post-hoc meta-analytic estimation of the extent to which bipolar disorder, schizophrenia, or both conditions contribute to brain gray matter differences compared to controls was achieved using a novel application of the conventional anatomical likelihood estimation (ALE) method. 19 schizophrenia studies (651 patients and 693 controls) were matched as closely as possible to 19 bipolar studies (540 patients and 745 controls). Result: Substantial overlaps in the regions affected by schizophrenia and bipolar disorder included regions in prefrontal cortex, thalamus, left caudate, left medial temporal lobe, and right insula. Bipolar disorder and schizophrenia jointly contributed to clusters in the right hemisphere, but schizophrenia was almost exclusively associated with additional gray matter deficits (left insula and amygdala) in the left hemisphere. Limitation: The current meta-analytic method has a number of constraints. Importantly, only studies identifying differences between controls and patient groups could be included in this analysis. Conclusion: Bipolar disorder shares many of the same brain regions as schizophrenia. However, relative to neurotypical controls, lower gray matter volume in schizophrenia is more extensive and includes the amygdala. This fresh application of ALE accommodates multiple studies in a relatively unbiased comparison. Common biological mechanisms may explain the neuroanatomical overlap between these major disorders, but explaining why brain differences are more extensive in schizophrenia remains challenging