Nonmonotonicity in Absolute and Relative Preferences for Real and Imagined Rewards

A key assumption in many studies examining valuation of reward is that participants’ preferences for various rewards are meaningfully, monotonically ordered with respect to other possible rewards. However, this assumption has not been systematically tested. Two studies consisting of 74 undergraduates from the University of Nevada, Las Vegas and 122 community members demonstrated nonmonotonic reward preferences when provided with parametrically varied reward magnitudes. Although deficits in reward processing are believed to be a key feature of depression, depressed participants were more willing to work hard for rewards and exhibited more monotonic reward preferences than non-depressed participants. Relative to imaginary rewards, participants were more willing to work for real money and spent more time making decisions when there was a possibility of earning real money

Differences in Behavior and Brain Activity during Hypothetical and Real Choices

Real behaviors are binding consequential commitments to a course of action, such as harming another person, buying an Apple watch, or fleeing from danger. Cognitive scientists are generally interested in the psychological and neural processes that cause such real behavior. However, for practical reasons, many scientific studies measure behavior using only hypothetical or imagined stimuli. Generalizing from such studies to real behavior implicitly assumes that the processes underlying the two types of behavior are similar. We review evidence of similarity and differences in hypothetical and real mental processes. In many cases, hypothetical choice tasks give an incomplete picture of brain circuitry that is active during real choice

Schema and value: Characterizing the role of the rostral and ventral medial prefrontal cortex in episodic future thinking

As humans we are not stuck in an everlasting present. Instead, we can project ourselves into both our personal past and future. Remembering the past and simulating the future are strongly interrelated processes. They are both supported by largely the same brain regions including the rostral and ventral medial prefrontal cortex (mPFC) but also the hippocampus, the posterior cingulate cortex (PCC), as well as other regions in the parietal and temporal cortices. Interestingly, this core network for episodic simulation and episodic memory partially overlaps with a brain network for evaluation and value-based decision making. This is particularly the case for the mPFC. This part of the brain has been associated both with a large number of different cognitive functions ranging from the representation of memory schemas and self-referential processing to the representation of value and affect. As a consequence, a unifying account of mPFC functioning has remained elusive. The present thesis investigates the unique contribution of the mPFC to episodic simulation by highlighting its role in the representation of memory schemas and value. In a first functional MRI and pre-registered behavioral replication study, we demonstrate that the mPFC encodes representations of known people as well as of known locations from participants’ everyday life. We demonstrate that merely imagined encounters with liked vs. disliked people at these locations can change our attitude toward the locations. The magnitude of this simulation-induced attitude change was predicted by activation in the mPFC during the simulations. Specifically, locations simulated with liked people exhibited significantly larger increases in liking than those simulated with disliked people. In a second behavioral study, we examined the mechanisms of simulation-based learning more closely. To this end, participants also simulated encounters with neutral people at neutral locations. Using repeated behavioral assessments of participants’ memory representations, we reveal that simulations cause an integration of memory representations for jointly simulated people and locations. Moreover, compared to the neutral baseline condition we demonstrate a transfer of positive valence from liked and of negative valence from disliked people to their paired locations. We also provide evidence that simulations induce an affective experience that aligns with the valence of the person and that this experience can account for the observed attitude change toward the location. In a final fMRI study, we examine the structure of memory representations encoded in the mPFC. Specifically, we provide evidence for the hypothesis that the mPFC encodes schematic representations of our social and physical environment. We demonstrate that representations of individual exemplars of these environments (i.e., individual people and locations) are closely intertwined with a representation of their value. In sum, our findings show that we can learn from imagined experience much as we learn from actual past experience and that the mPFC plays a key role in simulation-based learning. The mPFC encodes information about our environment in value-weighted schematic representations. These representations can account for the overlap of mnemonic and evaluative functions in the mPFC and might play a key role in simulation-based learning. Our results are in line with a view that our memories of the past serve us in ways that are oriented toward the future. Our ability to simulate potential scenarios allows us to anticipate the future consequences of our choices and thereby fosters farsighted decision making. Thus, our findings help to better characterize the functional role of the mPFC in episodic future simulation and valuation

Analysis of individual differences in neurofeedback training illuminates successful self-regulation of the dopaminergic midbrain

The dopaminergic midbrain is associated with reinforcement learning, motivation and decision-making – functions often disturbed in neuropsychiatric disorders. Previous research has shown that dopaminergic midbrain activity can be endogenously modulated via neurofeedback. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examine whether the activation of particular brain regions associates with successful regulation transfer when feedback is no longer available. Moreover, to elucidate mechanisms underlying effective self-regulation, we study the relation of successful transfer with learning (temporal difference coding) outside the midbrain during neurofeedback training and with individual reward sensitivity in a monetary incentive delay (MID) task. Fifty-nine participants underwent neurofeedback training either in standard (Study 1 N = 15, Study 2 N = 28) or control feedback group (Study 1, N = 16). We find that successful self-regulation is associated with prefrontal reward sensitivity in the MID task (N = 25), with a decreasing relation between prefrontal activity and midbrain learning signals during neurofeedback training and with increased activity within cognitive control areas during transfer. The association between midbrain self-regulation and prefrontal temporal difference and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings provide insights in the control of midbrain activity and may facilitate individually tailoring neurofeedback training

Hierarchical Reinforcement Learning in Behavior and the Brain

Dissertation presented to obtain the Ph.D degree in Biology, NeuroscienceReinforcement learning (RL) has provided key insights to the neurobiology of learning and decision making. The pivotal nding is that the phasic activity of dopaminergic cells in the ventral tegmental area during learning conforms to a reward prediction error (RPE), as speci ed in the temporal-di erence learning algorithm (TD). This has provided insights to conditioning, the distinction between habitual and goal-directed behavior, working memory, cognitive control and error monitoring. It has also advanced the understanding of cognitive de cits in Parkinson's disease, depression, ADHD and of personality traits such as impulsivity.(...

Emotional Cue Effects on Choice Impulsivity

This dissertation project investigated neurophysiological mechanisms contributing to emotional cue effects on temporal discounting (TD). Due to its high intraindividual stability and its close association with a range of maladaptive behaviors and clinical conditions (including addiction) TD has been considered a transdiagnostic marker and potential (future) indicator of treatment response. Improving our understanding of how TD is affected by external factors therefore represents a both vital andextensive endeavor. In a psychophysiological and a brain imaging study, it was shown that affective cues lead to a strong increase in neurophysiological arousal indexed by pupil dilation, heart rate frequencies, and electrodermal activity as well as mesolimbic BOLD activity. However, intertemporal decision-making behavior could not be altered by the visual stimuli. There was also no behavioral influence of physical arousal which was thoroughly tested via extensive computational modelling. The impulsivity-enhancing effects of so-called drug cues often oberservable in addiction disorders seems to be moderated by a different addiction-specific mechanism

Effects of Peer Influence in Adolescence

Adolescence is a period in life that is characterised by substantial changes in the social environment. Compared to childhood, relationships with peers gain more importance and adolescents are particularly sensitive to peer influence. Adolescents, but not adults, show increased levels of risk-taking when in the presence of peers relative to when alone. Experimental studies suggest that heightened levels of risk-taking during adolescence might be specific to affective contexts (e.g. the presence of peers), while risk-taking in non-affective contexts remains stable or decreases. In Chapter 2 of this thesis the development of the impact of two decision variables (risk and valence) on decision-making in a non-affective context during adolescence is investigated in a behavioural risk-taking task. Chapter 3 employs a functional magnetic resonance imaging (fMRI) approach to examine the development of the neural correlates of social influence during adolescence. Previous studies have focussed on peer influence on risk-taking and little is known about the neural mechanisms of peer influence. This thesis examines whether heightened levels of sensitivity to peer influence during adolescence extend to the influence of a peer audience on tasks with either high-level (reasoning) or low-level (perceptual) cognitive components (Chapters 4 and 5). Chapter 4 investigates the effect of the presence of a peer audience on performance in a cognitively challenging task (relational reasoning), the development of this effect during adolescence and whether this effect is dependent on the identity of the audience (peer or non-peer). Chapter 5 examines the effect of the presence of a peer audience on performance in a low-level perceptual task to test whether peer audience effects would also extend to a low-level cognitive task. Chapter 6 investigates the modulation of brain activity during a high-level cognitive task (relational reasoning) by an evaluative peer audience in adolescents and adults