Many hats: intra-trial and reward-level dependent bold activity in the striatum and premotor cortex

2012 Spring.Includes bibliographical references.Lesion, drug, single-cell recording, as well as human fMRI studies, suggest dopaminergic projections from VTA/SNc (ventral tagmental area/substantia nigra pars compacta) and cortically driven striatal activity plays a key role in associating sensory events with rewarding actions both by facilitating reward processing and prediction (i.e. reinforcement learning) and biasing and later updating action selection. We, for the first time, isolated BOLD signal changes for stimulus, pre-response, response and feedback delivery at three reward levels. This design allowed us to estimate the degree of involvement of individual striatal regions across these trial components, the reward sensitivity of each component and allowed for a novel comparison of potential (and potentially competing) reinforcement learning computations. Striatal and lateral premotor cortex regions of interest (ROIs) significant activations were universally observed (excepting the ventral striatum) during stimulus presentation, pre-response, response and feedback delivery, confirming these areas importance in all aspects of visuomotor learning. The head of the caudate showed a precipitous drop in activity pre-response, while in the body of the caudate showed no significant changes in activity. The putamen peaked in activity during response. Activation in the lateral premotor cortex was strongest during stimulus presentation, but the drop off was followed by a trend of increasing activity as feedback approached. Both the head and body of the caudate as well as the putamen displayed reward-level sensitivity only during stimulus, while the ventral striatum showed reward sensitivity at both stimulus and feedback. The lack of reward sensitivity surrounding response is inconsistent with theories that the head and ventral striatum encode the value of actions. Which of the three examined reinforcement learning models correlated best with BOLD signal changes varied as a function of trial component and ROI suggesting these regions computations vary depending on task demand

Independent circuits in basal ganglia and cortex for the processing of reward and precision feedback

In order to understand human decision making it is necessary to understand how the brain uses feedback to guide goal-directed behavior. The ventral striatum (VS) appears to be a key structure in this function, responding strongly to explicit reward feedback. However, recent results have also shown striatal activity following correct task performance even in the absence of feedback. This raises the possibility that, in addition to processing external feedback, the dopamine-centered reward circuit might regulate endogenous reinforcement signals, like those triggered by satisfaction in accurate task performance. Here we use functional magnetic resonance imaging (fMRI) to test this idea. Participants completed a simple task that garnered both reward feedback and feedback about the precision of performance. Importantly, the design was such that we could manipulate information about the precision of performance within different levels of reward magnitude. Using parametric modulation and functional connectivity analysis we identified brain regions sensitive to each of these signals. Our results show a double dissociation: frontal and posterior cingulate regions responded to explicit reward but were insensitive to task precision, whereas the dorsal striatum - and putamen in particular - was insensitive to reward but responded strongly to precision feedback in reward-present trials. Both types of feedback activated the VS, and sensitivity in this structure to precision feedback was predicted by personality traits related to approach behavior and reward responsiveness. Our findings shed new light on the role of specific brain regions in integrating different sources of feedback to guide goal-directed behavior

Decision-making after continuous wins or losses in a randomized guessing task: implications for how the prior selection results affect subsequent decision-making

BACKGROUND: Human decision-making is often affected by prior selections and their outcomes, even in situations where decisions are independent and outcomes are unpredictable. METHODS: In this study, we created a task that simulated real-life non-strategic gambling to examine the effect of prior outcomes on subsequent decisions in a group of male college students. RESULTS: Behavioral performance showed that participants needed more time to react after continuous losses (LOSS) than continuous wins (WIN) and discontinuous outcomes (CONTROL). In addition, participants were more likely to repeat their selections in both WIN and LOSS conditions. Functional MRI data revealed that decisions in WINs were associated with increased activation in the mesolimbic pathway, but decreased activation in the inferior frontal gyrus relative to LOSS. Increased prefrontal cortical activation was observed during LOSS relative to WIN and CONTROL conditions. CONCLUSION: Taken together, the behavioral and neuroimaging findings suggest that participants tended to repeat previous selections during LOSS trials, a pattern resembling the gambler’s fallacy. However, during WIN trials, participants tended to follow their previous lucky decisions, like the ‘hot hand’ fallacy

AN EXAMINATION OF CONCURRENT DISCRIMINATION LEARNING WITHIN INDIVIDUALS WITH PARKINSON’S DISEASE

The main focus of this research is to further understand memory formation by examining the role of the basal ganglia in learning. Broadly, this study examines how the basal ganglia may play a role in a task that has been associated with declarative memory mechanisms, in this case the concurrent discrimination task (CDT). Specifically, we examine how performance is affected on the CDT when structures of the basal ganglia are compromised by recruiting individuals with Parkinson’s disease (PD). Past work examining the performance of individuals with PD on a CDT have had contradicting results and have proposed that participants may adopt different strategies that rely variously either on declarative or non-declarative strategy (Moody et. al., 2010). We aimed to reduce strategy differences by making changes in stimuli, increasing the number of stimuli significantly, increasing the number of learning blocks, and making all participants explicitly aware of the task structure and goals. By making the goals explicit, we predicted that we would engage a declarative mechanism in both PD and control individuals. To examine declarative memory formation we used the Remember Know task (RK). However, since used a significantly larger set size of stimuli we hypothesized that individuals with PD would perform significantly worse on the CDT than control individuals. The current study reveals that there are no significant differences in performance between individuals with PD and control participants on both the CDT and RK task. We attribute these results to design of our paradigm and stimuli which may have influenced individuals to engage in declarative strategies to perform the CDT reasonably well

Information content and reward processing in the human striatum during performance of a declarative memory task

Negative feedback can signal poor performance, but it also provides information that can help learners reach the goal of task mastery. The primary aim of this study was to test the hypothesis that the amount of information provided by negative feedback during a paired-associate learning task influences feedback-related processing in the caudate nucleus. To do this, we manipulated the number of response options: With two options, positive and negative feedback provide equal amounts of information, whereas with four options, positive feedback provides more information than does negative feedback. We found that positive and negative feedback activated the caudate similarly when there were two response options. With four options, the caudate’s response to negative feedback was reduced. A secondary goal was to investigate the link between brain-based measures of feedback-related processing and behavioral indices of learning. Analysis of the posttest measures showed that trials with positive feedback were associated with higher posttest confidence ratings. Additionally, when positive feedback was delivered, caudate activity was greater for trials with high than with low posttest confidence. This experiment demonstrated the context sensitivity of feedback processing and provided evidence that feedback processing in the striatum can contribute to the strengthening of the representations available within declarative memory

The role of the left head of caudate in suppressing irrelevant words

Suppressing irrelevant words is essential to successful speech production and is expected to involve general control mechanisms that reduce interference from task-unrelated processing. To investigate the neural mechanisms that suppress visual word interference, we used fMRI and a Stroop task, using a block design with an event-related analysis. Participants indicated with a finger press whether a visual stimulus was colored pink or blue. The stimulus was either the written word "BLUE," the written word "PINK," or a string of four Xs, with word interference introduced when the meaning of the word and its color were "incongruent" (e.g., BLUE in pink hue) relative to congruent (e.g., BLUE in blue) or neutral (e.g., XXXX in pink). The participants also made color decisions in the presence of spatial interference rather than word interference (i.e., the Simon task). By blocking incongruent, congruent, and neutral trials, we identified activation related to the mechanisms that suppress interference as that which was greater at the end relative to the start of incongruency. This highlighted the role of the left head of caudate in the control of word interference but not spatial interference. The response in the left head of caudate contrasted to bilateral inferior frontal activation that was greater at the start than at the end of incongruency, and to the dorsal anterior cingulate gyrus which responded to a change in the motor response. Our study therefore provides novel insights into the role of the left head of caudate in the mechanisms that suppress word interference

Neural Correlates of Feedback Processing in Visuo-Tactile Crossmodal Paired-Associate Learning

Previous studies have examined the neural correlates for crossmodal paired-associate (PA) memory and the temporal dynamics of its formation. However, the neural dynamics for feedback processing of crossmodal PA learning remain unclear. To examine this process, we recorded event-related scalp electrical potentials for PA learning of unimodal visual-visual pairs and crossmodal visual-tactile pairs when participants performed unimodal and crossmodal tasks. We examined event-related potentials (ERPs) after the onset of feedback in the tasks for three effects: feedback type (positive feedback vs. negative feedback), learning (as the learning progressed) and the task modality (crossmodal vs. unimodal). The results were as follows: (1) feedback type: the amplitude of P300 decreased with incorrect trials and the P400/N400 complex was only present in incorrect trials; (2) learning: progressive positive voltage shifts in frontal recording sites and negative voltage shifts in central and posterior recording sites were identified as learning proceeded; and (3) task modality: compared with the unimodal PA learning task, positive voltage shifts in frontal sites and negative voltage shifts in posterior sites were found in the crossmodal PA learning task. To sum up, these results shed light on cortical excitability related to feedback processing of crossmodal PA learning

The basal ganglia and training of arithmetic fluency

The role of dopamine neurons in reward processing is well-established, as is the observation of reward-related responses in the striatum, a region to which these midbrain dopamine neurons project. The reward-prediction error signals generated in the midbrain may play a role in the striatum in learning, as they help to shape expectations about future events based on prior experiences. The goal of the current experiment was to use principles of striatal function in order to optimize learning in an arithmetic domain. We created a training program that we believed would lead to increased arithmetic fluency by maximally engaging the striatum, through the use of contingent feedback, uncertainty regarding performance, and incentives for correct responses. Both experimental and control participants, who completed training focusing on arithmetic calculation and digit-entry respectively, showed improvement on a task involving the addition of a double-digit and a single-digit number following training, as successful performance on the task required accurate computations and entry of the solution within a narrow response window. We conducted functional magnetic resonance imaging before and after training while participants performed this task, in order to examine the effect of feedback on activity in the caudate nucleus and to determine if learning signals generated by the striatum during arithmetic training are able to modify quantity representations in parietal cortex. Results indicated activation of both the caudate nucleus and the hIPS region. Activation of the caudate nucleus replicated previous work, as it showed the prototypical pattern of activity that distinguished between positive and negative feedback. Activation of the hIPS region was not surprising, given the focus on arithmetic calculation, but this region also exhibited feedback-sensitive activation that differed between sessions and groups, possibly indicating the common influence of a reinforcement learning system

How may the basal ganglia contribute to auditory categorization and speech perception?

Listeners must accomplish two complementary perceptual feats in extracting a message from speech. They must discriminate linguistically-relevant acoustic variability and generalize across irrelevant variability. Said another way, they must categorize speech. Since the mapping of acoustic variability is language-specific, these categories must be learned from experience. Thus, understanding how, in general, the auditory system acquires and represents categories can inform us about the toolbox of mechanisms available to speech perception. This perspective invites consideration of findings from cognitive neuroscience literatures outside of the speech domain as a means of constraining models of speech perception. Although neurobiological models of speech perception have mainly focused on cerebral cortex, research outside the speech domain is consistent with the possibility of significant subcortical contributions in category learning. Here, we review the functional role of one such structure, the basal ganglia. We examine research from animal electrophysiology, human neuroimaging, and behavior to consider characteristics of basal ganglia processing that may be advantageous for speech category learning. We also present emerging evidence for a direct role for basal ganglia in learning auditory categories in a complex, naturalistic task intended to model the incidental manner in which speech categories are acquired. To conclude, we highlight new research questions that arise in incorporating the broader neuroscience research literature in modeling speech perception, and suggest how understanding contributions of the basal ganglia can inform attempts to optimize training protocols for learning non-native speech categories in adulthood