Reinforcement magnitudes modulate subthalamic beta band activity in patients with Parkinson's disease

We set out to investigate whether beta oscillations in the human basal ganglia are modulated during reinforcement learning. Based on previous research, we assumed that beta activity might either reflect the magnitudes of individuals' received reinforcements (reinforcement hypothesis), their reinforcement prediction errors (dopamine hypothesis) or their tendencies to repeat versus adapt responses based upon reinforcements (status-quo hypothesis). We tested these hypotheses by recording local field potentials (LFPs) from the subthalamic nuclei of 19 Parkinson's disease patients engaged in a reinforcement-learning paradigm. We then correlated patients' reinforcement magnitudes, reinforcement prediction errors and response repetition tendencies with task-related power changes in their LFP oscillations. During feedback presentation, activity in the frequency range of 14 to 27 Hz (beta spectrum) correlated positively with reinforcement magnitudes. During responding, alpha and low beta activity (6 to 18 Hz) was negatively correlated with previous reinforcement magnitudes. Reinforcement prediction errors and response repetition tendencies did not correlate significantly with LFP oscillations. These results suggest that alpha and beta oscillations during reinforcement learning reflect patients' observed reinforcement magnitudes, rather than their reinforcement prediction errors or their tendencies to repeat versus adapt their responses, arguing both against an involvement of phasic dopamine and against applicability of the status-quo theory

Direct-Pathway Striatal Neurons Regulate the Retention of Decision-Making Strategies

The dorsal striatum has been implicated in reward-based decision making, but the role played by specific striatal circuits in these processes is essentially unknown. Using cell phenotype-specific viral vectors to express engineered G-protein-coupled DREADD (designer receptors exclusively activated by designer drugs) receptors, we enhanced G(i/o)- or G(s)-protein-mediated signaling selectively in direct-pathway (striatonigral) neurons of the dorsomedial striatum in Long–Evans rats during discrete periods of training of a high versus low reward-discrimination task. Surprisingly, these perturbations had no impact on reward preference, task performance, or improvement of performance during training. However, we found that transiently increasing G(i/o) signaling during training significantly impaired the retention of task strategies used to maximize reward obtainment during subsequent preference testing, whereas increasing G(s) signaling produced the opposite effect and significantly enhanced the encoding of a high-reward preference in this decision-making task. Thus, the fact that the endurance of this improved performance was significantly altered over time—long after these neurons were manipulated—indicates that it is under bidirectional control of canonical G-protein-mediated signaling in striatonigral neurons during training. These data demonstrate that cAMP-dependent signaling in direct-pathway neurons play a well-defined role in reward-related behavior; that is, they modulate the plasticity required for the retention of task-specific information that is used to improve performance on future renditions of the task

Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning

SummaryThe division of human learning systems into reward and punishment opponent modules is still a debated issue. While the implication of ventral prefrontostriatal circuits in reward-based learning is well established, the neural underpinnings of punishment-based learning remain unclear. To elucidate the causal implication of brain regions that were related to punishment learning in a previous functional neuroimaging study, we tested the effects of brain damage on behavioral performance, using the same task contrasting monetary gains and losses. Cortical and subcortical candidate regions, the anterior insula and dorsal striatum, were assessed in patients presenting brain tumor and Huntington disease, respectively. Both groups exhibited selective impairment of punishment-based learning. Computational modeling suggested complementary roles for these structures: the anterior insula might be involved in learning the negative value of loss-predicting cues, whereas the dorsal striatum might be involved in choosing between those cues so as to avoid the worst

Adolescent-specific patterns of behavior and neural activity during social reinforcement learning

Humans are sophisticated social beings. Social cues from others are exceptionally salient, particularly during adolescence. Understanding how adolescents interpret and learn from variable social signals can provide insight into the observed shift in social sensitivity during this period. The current study tested 120 participants between the ages of 8 and 25 years on a social reinforcement learning task where the probability of receiving positive social feedback was parametrically manipulated. Seventy-eight of these participants completed the task during fMRI scanning. Modeling trial-by-trial learning, children and adults showed higher positive learning rates than adolescents, suggesting that adolescents demonstrated less differentiation in their reaction times for peers who provided more positive feedback. Forming expectations about receiving positive social reinforcement correlated with neural activity within the medial prefrontal cortex and ventral striatum across age. Adolescents, unlike children and adults, showed greater insular activity during positive prediction error learning and increased activity in the supplementary motor cortex and the putamen when receiving positive social feedback regardless of the expected outcome, suggesting that peer approval may motivate adolescents towards action. While different amounts of positive social reinforcement enhanced learning in children and adults, all positive social reinforcement equally motivated adolescents. Together, these findings indicate that sensitivity to peer approval during adolescence goes beyond simple reinforcement theory accounts and suggests possible explanations for how peers may motivate adolescent behavior.Psycholog

From the ventral to the dorsal striatum: Devolving views of their roles in drug addiction

AbstractWe revisit our hypothesis that drug addiction can be viewed as the endpoint of a series of transitions from initial voluntarily drug use to habitual, and ultimately compulsive drug use. We especially focus on the transitions in striatal control over drug seeking behaviour that underlie these transitions since functional heterogeneity of the striatum was a key area of Ann Kelley's research interests and one in which she made enormous contributions. We also discuss the hypothesis in light of recent data that the emergence of a compulsive drug seeking habit both reflects a shift to dorsal striatal control over behaviour and impaired prefontal cortical inhibitory control mechanisms. We further discuss aspects of the vulnerability to compulsive drug use and in particular the impact of impulsivity. In writing this review we acknowledge the untimely death of an outstanding scientist and a dear personal friend

Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding

Hard-wired, Pavlovian, responses elicited by predictions of rewards and punishments exert significant benevolent and malevolent influences over instrumentally-appropriate actions. These influences come in two main groups, defined along anatomical, pharmacological, behavioural and functional lines. Investigations of the influences have so far concentrated on the groups as a whole; here we take the critical step of looking inside each group, using a detailed reinforcement learning model to distinguish effects to do with value, specific actions, and general activation or inhibition. We show a high degree of sophistication in Pavlovian influences, with appetitive Pavlovian stimuli specifically promoting approach and inhibiting withdrawal, and aversive Pavlovian stimuli promoting withdrawal and inhibiting approach. These influences account for differences in the instrumental performance of approach and withdrawal behaviours. Finally, although losses are as informative as gains, we find that subjects neglect losses in their instrumental learning. Our findings argue for a view of the Pavlovian system as a constraint or prior, facilitating learning by alleviating computational costs that come with increased flexibility

Conflicts as aversive signals : Investigations on the affective valence of conflict stimuli

According to a recent theory by Matthew Botvinick (2007), conflicts in information processing are aversive. This assumption, its boundaries and its functionality, have been examined in three studies. STUDY 1 tested if the conflict priming effect as found by Dreisbach and Fischer (2012a) can be reproduced with a different dependent variable, offering more clear-cut evidence for conflict aversiveness. Participants had to judge the valence of neutral German words or Chinese pictographs after being primed by conflict or non-conflict Stroop primes in two experiments. Results show that priming with conflict stimuli increases the frequency of negative judgments, thus giving unequivocal evidence for the aversiveness of conflicts. STUDY 2 was designed to test the time characteristics of the conflict priming effect. In three experiments, the results show that conflict priming is present already with a SOA of 200 ms, highlighting the similarity between conflicts and other aversive stimuli. Furthermore, Study 2 showed a reverse priming effect for a SOA of 800 ms with continued prime presentation, maybe due to processes of affective counter-regulation. STUDY 3 investigated whether it is the aversiveness of conflict stimuli that motivates conflict adaptation. In two separate experiments, subjects participated in two response conflict tasks, a color version of the Eriksen Flanker task and a manual version of the Stroop task. Here, the stimuli’s perceptual fluency (i.e., the ease of processing) was manipulated in short blocks of ten trials length. Disfluency (reduced figure-ground contrast) is associated with the experience of negative affect. Results showed that increasing the general stimulus aversiveness by adding disfluency eliminated conflict adaptation instead of increasing it. The results of the three studies are discussed in the light of current emotion and cognitive control research