Dissociating self-generated volition from externally-generated motivation.

Insight into motivational processes may be gained by examining measures of willingness to exert effort for rewards, which have been linked to neuropsychiatric symptoms of anhedonia and apathy. However, while much work has focused on the development of models of motivation based on classic tasks of externally-generated levels of effort for reward, there has been less focus on the question of self-generated motivation or volition. We developed a task that aims to capture separate measures of self-generated and externally-generated motivation, with two task variants for physical and cognitive effort, and sought to test and validate this measure in two populations of healthy volunteers (N = 27 and N = 28). Similar to previous reports, a sigmoid function represented a better overall fit to the effort-reward data than a linear or Weibull model. Individual sigmoid function shapes were governed by two free parameters: bias (the amount of reward needed for effort initiation) and reward insensitivity (the amount of increase in reward needed to accelerate effort expenditure). For both physical and cognitive effort, bias was higher in the self-generated condition, indicating reduced self-generated volitional effort initiation, compared to externally-generated effort initiation, across effort domains. Bias against initial effort initiation in the self-generated condition was related to a specific dimensional measure of anticipatory anhedonia. For physical effort only, reward insensitivity was also higher in the self-generated condition compared to the externally-generated motivation condition, indicating lower self-generated effort acceleration. This work provides a novel objective measure of self-generated motivation that may provide insights into mechanisms of anhedonia and related symptoms

Common and Distinct Functional Brain Networks for Intuitive and Deliberate Decision Making

Reinforcement learning studies in rodents and primates demonstrate that goal-directed and habitual choice behaviors are mediated through different fronto-striatal systems, but the evidence is less clear in humans. In this study, functional magnetic resonance imaging (fMRI) data were collected whilst participants ( n = 20) performed a conditional associative learning task in which blocks of novel conditional stimuli (CS) required a deliberate choice, and blocks of familiar CS required an intuitive choice. Using standard subtraction analysis for fMRI event-related designs, activation shifted from the dorso-fronto-parietal network, which involves dorsolateral prefrontal cortex (DLPFC) for deliberate choice of novel CS, to ventro-medial frontal (VMPFC) and anterior cingulate cortex for intuitive choice of familiar CS. Supporting this finding, psycho-physiological interaction (PPI) analysis, using the peak active areas within the PFC for novel and familiar CS as seed regions, showed functional coupling between caudate and DLPFC when processing novel CS and VMPFC when processing familiar CS. These findings demonstrate separable systems for deliberate and intuitive processing, which is in keeping with rodent and primate reinforcement learning studies, although in humans they operate in a dynamic, possibly synergistic, manner particularly at the level of the striatum.Peer reviewedFinal Published versio

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson\u27s disease: implications for cognitive function.

Cognitive abnormalities are a feature of Parkinson\u27s disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment

Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors

To make sound economic decisions, the brain needs to compute several different value-related signals. These include goal values that measure the predicted reward that results from the outcome generated by each of the actions under consideration, decision values that measure the net value of taking the different actions, and prediction errors that measure deviations from individuals' previous reward expectations. We used functional magnetic resonance imaging and a novel decision-making paradigm to dissociate the neural basis of these three computations. Our results show that they are supported by different neural substrates: goal values are correlated with activity in the medial orbitofrontal cortex, decision values are correlated with activity in the central orbitofrontal cortex, and prediction errors are correlated with activity in the ventral striatum

Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain

Prediction error signals have been reported in human imaging studies in target areas of dopamine neurons such as ventral and dorsal striatum during learning with many different types of reinforcers. However, a key question that has yet to be addressed is whether prediction error signals recruit distinct or overlapping regions of striatum and elsewhere during learning with different types of reward. To address this, we scanned 17 healthy subjects with functional magnetic resonance imaging while they chose actions to obtain either a pleasant juice reward (1 ml apple juice), or a monetary gain (5 cents) and applied a computational reinforcement learning model to subjects' behavioral and imaging data. Evidence for an overlapping prediction error signal during learning with juice and money rewards was found in a region of dorsal striatum (caudate nucleus), while prediction error signals in a subregion of ventral striatum were significantly stronger during learning with money but not juice reward. These results provide evidence for partially overlapping reward prediction signals for different types of appetitive reinforcers within the striatum, a finding with important implications for understanding the nature of associative encoding in the striatum as a function of reinforcer type

Many hats: intra-trial and reward-level dependent bold activity in the striatum and premotor cortex

2012 Spring.Includes bibliographical references.Lesion, drug, single-cell recording, as well as human fMRI studies, suggest dopaminergic projections from VTA/SNc (ventral tagmental area/substantia nigra pars compacta) and cortically driven striatal activity plays a key role in associating sensory events with rewarding actions both by facilitating reward processing and prediction (i.e. reinforcement learning) and biasing and later updating action selection. We, for the first time, isolated BOLD signal changes for stimulus, pre-response, response and feedback delivery at three reward levels. This design allowed us to estimate the degree of involvement of individual striatal regions across these trial components, the reward sensitivity of each component and allowed for a novel comparison of potential (and potentially competing) reinforcement learning computations. Striatal and lateral premotor cortex regions of interest (ROIs) significant activations were universally observed (excepting the ventral striatum) during stimulus presentation, pre-response, response and feedback delivery, confirming these areas importance in all aspects of visuomotor learning. The head of the caudate showed a precipitous drop in activity pre-response, while in the body of the caudate showed no significant changes in activity. The putamen peaked in activity during response. Activation in the lateral premotor cortex was strongest during stimulus presentation, but the drop off was followed by a trend of increasing activity as feedback approached. Both the head and body of the caudate as well as the putamen displayed reward-level sensitivity only during stimulus, while the ventral striatum showed reward sensitivity at both stimulus and feedback. The lack of reward sensitivity surrounding response is inconsistent with theories that the head and ventral striatum encode the value of actions. Which of the three examined reinforcement learning models correlated best with BOLD signal changes varied as a function of trial component and ROI suggesting these regions computations vary depending on task demand

Missing motoric manipulations: rethinking the imaging of the ventral striatum and dopamine in human reward

Human neuroimaging studies of natural rewards and drugs of abuse frequently assay the brain’s response to stimuli that, through Pavlovian learning, have come to be associated with a drug’s rewarding properties. This might be characterized as a ‘sensorial’ view of the brain’s reward system, insofar as the paradigms are designed to elicit responses to a reward’s (drug’s) sight, aroma, or flavor. A different field of research nevertheless suggests that the mesolimbic dopamine system may also be critically involved in the motor behaviors provoked by such stimuli. This brief review and commentary surveys some of the preclinical data supporting this more “efferent” (motoric) view of the brain’s reward system, and discusses what such findings might mean for how human brain imaging studies of natural rewards and drugs of abuse are designed

Striatal responsiveness to reward under threat-of-shock and working memory load: A preliminary study.

Reward and stress are important determinants of motivated behaviors. Striatal regions play a crucial role in both motivation and hedonic processes. So far, little is known on how cognitive effort interacts with stress to modulate reward processes. This study examines how cognitive effort (load) interacts with an unpredictable acute stressor (threat-of-shock) to modulate motivational and hedonic processes in healthy adults. A reward task, involving stress with unpredictable mild electric shocks, was conducted in 23 healthy adults aged 20-37 (mean age: 24.7 ± 0.9; 14 females) during functional magnetic resonance imaging (fMRI). Manipulation included the use of (a) monetary reward for reinforcement, (b) threat-of-shock as the stressor, and (c) a spatial working memory task with two levels of difficulty (low and high load) for cognitive load. Reward-related activation was investigated in a priori three regions of interest, the nucleus accumbens (NAcc), caudate nucleus, and putamen. During anticipation, threat-of-shock or cognitive load did not affect striatal responsiveness to reward. Anticipated reward increased activation in the ventral and dorsal striatum. During feedback delivery, both threat-of-shock and cognitive effort modulated striatal activation. Higher working memory load blunted NAcc responsiveness to reward delivery, while stress strengthened caudate nucleus reactivity regardless reinforcement or load. These findings provide initial evidence that both stress and cognitive load modulate striatal responsiveness during feedback delivery but not during anticipation in healthy adults. Of clinical importance, sustained stress exposure might go along with dysregulated arousal, increasing therefore the risk for the development of maladaptive incentive-triggered motivation. This study brings new insight that might help to build a framework to understand common stress-related disorders, given that these psychiatric disorders involve disturbances of the reward system, cognitive deficits, and abnormal stress reactivity

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region