Decision and Reward in Intertemporal Choice: The Roles of Brain Development, Inter-individual Differences and Pharmacological Influences

Human decision making is closely related to reward processing because many decisions rely to a certain degree on the evaluation of different outcome values. Reward-based decisions can be health-related, for example if someone has to compare the outcome value of the instant reward of smoking a cigarette to that of the long term goal of keeping well and fit. Such comparisons do not only rely on the nominal value of the alternatives but also on devaluation of rewards over time. The value of being healthy at older age might outweigh the value of smoking a cigarette but since the payoff of the health-outcome will be delayed, humans tend to decrease the value of this option. Therefore in this example one might choose the immediate reward of smoking a cigarette. The proclivity to devaluate the value of rewards over time has been widely investigated with experimental intertemporal choice tasks, in which subjects have to choose between smaller sooner rewards and larger later rewards. A stronger individual devaluation proclivity (i.e. discounting rate) has been reported to be related to addiction. Research in neuroeconomics has suggested the competing neurobehavioural decision systems (CNDS) theory, proposing that an imbalance between an executive (cortical prefrontal brain areas) and an impulsive (i.e. subcortical areas, such as ventral striatum (VS), amygdala) system in the brain leads to steeper discounting and a higher risk for addiction. Additionally, temporal discounting has been proposed as a transdisease process, i.e., “a process that occurs across a range of disorders, making findings from one disorder relevant to other disorders” (Bickel, Jarmolowicz, Mueller, Koffarnus, & Gatchalian, 2012, Abstract). Thus, the CNDS theory and temporal discounting might also have implications for other health-related behaviour than substance use. So far many factors have been shown to be associated with higher discount rates: for instance, adolescent age, lower intelligence and nicotine dependence. Further, it has been shown that adolescents are at highest risk to start smoking. On the other hand a higher education level has been shown to be associated to lower rates of smoking. Thus, it seems likely that a higher discount rate might be one reason why adolescents experiment with smoking, why lower education is associated to nicotine addiction and why dependent smokers are not successful in smoking cessation. But relatively little is known about the neural processes behind these variables, which could be also seen as exemplary risk- and protective factors regarding addiction. The 3 studies of the thesis at hand were conducted to extend the knowledge about neural processes associated to age, intelligence and smoking in their relation to intertemporal choice. The task was chosen because of its relevance for addiction and a variety of health-related behaviour. The first study was conducted to explore the neural correlates of age related differences between adolescents at age 14 and young adults during intertemporal choices. Additionally, the roles of discounting and choice consistency were investigated. Although adoles-cents discounted delayed rewards more steeply than adults, neural processing of reward value did not differ between groups, when controlling reward values for the individual discount rates. However, a higher discount rate was related to a lower responsivity in the ventral striatum to delayed rewards, independent of age. Concerning decision making, adolescents exhib-ited a lower consistency of choices and less brain activity in a parietal network than adults (i.e. posterior and inferior parietal regions). Thus, reward value processing might be more sensitive to the discount rate than to chronological age. Lower consistency of intertemporal choices might indicate ongoing maturation of parietal brain areas from adolescence to young adulthood. The second study was conducted to reveal the associations between neural processes of decision making and intelligence in adolescents. The results of study 2 revealed networks in the adolescent brain where brain activity was related to crystallised intelligence as well as to intertemporal choice behaviour. Specifically, during decision processing higher crystallised intelligence as well as more consistent decisions were associated with higher brain activity in the posterior parietal cortex. Processing of delayed rewards was also related to crystallised intelligence, i.e. more intelligent adolescents showed higher brain activation in the anterior cingulate cortex (ACC) and the inferior frontal gyrus (IFG), which was in turn related to a lower discount rate. Additionally, associations between the parental education level and crys-tallised intelligence of the adolescent participants of the study and their discount rate were found, indicating that parental education as an environmental factor could be related to a low-er risk for addiction. This protective effect might be mediated by the offspring’s crystallised intelligence and discount rate which are both related to brain activity in parts of the same brain networks (i.e. the IFG). The third study was done to investigate neural processes of intertemporal decisions in smokers and non-smokers. To test whether the effects of smoking on the discount rate are due to chronic or acute nicotine intake, non-smokers were additionally assessed under acute nico-tine administration. Study 3 revealed that the effects of nicotine on intertemporal choice behaviour were related to chronic intake of nicotine in smokers rather than to acute nicotine ad-ministration in non-smokers. Regarding the neural processes, smokers compared to non-smokers showed lower brain activity in the posterior parietal cortex. Comparable but weaker effects were found under acute nicotine in non-smokers. Although acute nicotine administra-tion altered neural processes, behavioural changes might only occur after repeated nicotine intake. However, the study did not preclude that the differences are predrug characteristics. Altogether the studies revealed overlapping neural correlates of intertemporal choices which are related to the individual age, the discount rate, the choice consistency, the individual intelligence as well as acute and chronic nicotine intake. This might provide an integrative view on how inter-individual differences and behaviour during intertemporal choices are based on common neural correlates which in turn might have implications for the development and the maintenance of addiction. Specifically, hyposensitivity towards delayed rewards in the adolescent ventral striatum, which has also been found in smokers compared to non-smokers, is associated with higher discount rates and higher risk for smoking initiation. In contrast, higher activation in the IFG and the ACC in more intelligent individuals during reward value processing might enhance behavioural inhibition and control and, hence, might prevent nicotine addiction. In line with the CNDS theory responsivity in subcortical brain areas (i.e. impulsive system), such as the VS was related to the risk factor of adolescent age, whereas activity in cortical areas (IFG and ACC) was related to the protective factors of high-er crystallised intelligence. Since there was only one study beside the studies of the current thesis reporting results regarding consistency, one can only speculate about implications for health-related behaviour, such as addiction. Consistency might play a role, especially for cessation success. Thus, the findings that adolescents as well as less intelligent individuals were less consistent might point to a higher risk for maintenance of nicotine addiction. The higher brain activity in a fronto-parietal network, which has been shown in studies 1 and 2 in adults as well as in more intelligent adolescents, was related to higher consistency of choices in both studies. Thus, the finding might be a possible neural correlate for the association between the risk factor of ado-lescent age, the protective factor of higher crystallised intelligence, and more consistent deci-sion making. In conclusion the findings of the current thesis contribute to a better understanding of how inter-individual differences and environmental factors might be accompanied by neural processes which in turn might be related to individual development of addiction. Further the results might extend the CNDS theory regarding neural correlates of exemplary risk and pro-tective factors regarding adolescents’ health behaviour and smoking in adults

Neurobiological bases of intertemporal choices: A comprehensive review

Intertemporal choices (ICs) are choices that involve trade-off between costs and benefits that take place at differ ent moments in time. The aim of this article is to present a comprehensive literature review on neurobiological bases of IC. We present the functional models of IC and data from neuroimaging studies, namely ALE analysis. With this paper we intended to show the presence of immediate value preference beyond that predicted by a single-parameter exponential discounting model and its mapping to the dual-systems model for brain function. Studies indicate that individuals tend to show inconsistent preferences depending on the time until the rewards are available and support a perspective that intertemporal evaluation reflects neural mechanisms that differ from other forms of choice, although associated value signals are later represented in the context of a common reward system. The IC induces activations in a “nuclear network” and auxiliary areas including inferior prefrontal cortex, medial prefrontal cortex, temporo-parietal cortex, and peri-splenial posterior cingulate. The network of areas sensitive to value is comprised of several regions that include ventral striatum, medial prefrontal cortex, orbitofrontal cortex, and anterior insula. Evidence from neuroimaging and EEG studies corroborates that choices are determined by a dual evaluation system.info:eu-repo/semantics/publishedVersio

Individual Differences In Value-Based Decision-Making: Learning And Time Preference

Human decisions are strongly influenced by past experience or by the subjective values attributed to available choice options. Although decision processes show some common trends across individuals, they also vary considerably between individuals. The research presented in this dissertation focuses on two domains of decision-making, related to learning and time preference, and examines factors that explain decision-making differences between individuals. First, we focus on a form of reinforcement learning in a dynamic environment. Across three experiments, we investigated whether individual differences in learning were associated with differences in cognitive abilities, personality, and age. Participants made sequential predictions about an on-screen location in a video game. Consistent with previous work, participants showed high variability in their ability to implement normative strategies related to surprise and uncertainty. We found that higher cognitive ability, but not personality, was associated with stronger reliance on the normative factors that should govern learning. Furthermore, learning in older adults (age 60+) was less influenced by uncertainty, but also less influenced by reward, a non-normative factor that has substantial effects on learning across the lifespan. Second, we focus on delay discounting, the tendency to prefer smaller rewards delivered soon over larger rewards delivered after a delay. Delay discounting has been used as a behavioral measure of impulsivity and is associated with many undesirable real-life outcomes. Specifically, we examined how neuroanatomy is associated with individual differences in delay discounting in a large adolescent sample. Using a novel multivariate method, we identified networks where cortical thickness varied consistently across individuals and brain regions. Cortical thickness in several of these networks, including regions such as ventromedial prefrontal cortex, orbitofrontal cortex, and temporal pole, was negatively associated with delay discounting. Furthermore, this brain data predicted differences beyond those typically accounted for by other cognitive variables related to delay discounting. These results suggest that cortical thickness may be a useful brain phenotype of delay discounting and carry unique information about impulsivity. Collectively, this research furthers our understanding of how cognitive abilities, brain structure and healthy aging relate to individual differences in value-based decision-making

Temporal delay discounting in acutely ill and weight-recovered patients with anorexia nervosa

Background. Patients with anorexia nervosa (AN) are characterized by a very low body weight but readily give up immediate rewards (food) for long-term goals (slim figure), which might indicate an unusual level of self-control. This everyday clinical observation may be quantifiable in the framework of the anticipation-discounting dilemma. Method. Using a cross-sectional design, this study compared the capacity to delay reward in 34 patients suffering from acute AN (acAN), 33 weight-recovered AN patients (recAN) and 54 healthy controls. We also used a longitudinal study to reassess 21 acAN patients after short-term weight restoration. A validated intertemporal choice task and a hyperbolic model were used to estimate temporal discounting rates. Results. Confirming the validity of the task used, decreased delay discounting was associated with age and low selfreported impulsivity. However, no group differences in key measures of temporal discounting of monetary rewards were found. Conclusions. Increased cognitive control, which has been suggested as a key characteristic of AN, does not seem to extend the capacity to wait for delayed monetary rewards. Differences between our study and the only previous study reporting decreased delay discounting in adult AN patients may be explained by the different age range and chronicity of acute patients, but the fact that weight recovery was not associated with changes in discount rates suggests that discounting behavior is not a trait marker in AN. Future studies using paradigms with disorder-specific stimuli may help to clarify the role of delay discounting in AN

Decisions, decisions, decisions: the development and plasticity of reinforcement learning, social and temporal decision making in children

Human decision-making is the flexible way people respond to their environment, take actions, and plan toward long-term goals. It is commonly thought that humans rely on distinct decision-making systems, which are either more habitual and reflexive or deliberate and calculated. How we make decisions can provide insight into our social functioning, mental health and underlying psychopathology, and ability to consider the consequences of our actions. Notably, the ability to make appropriate, habitual or deliberate decisions depending on the context, here referred to as metacontrol, remains underexplored in developmental samples. This thesis aims to investigate the development of different decision-making mechanisms in middle childhood (ages 5-13) and to illuminate the potential neurocognitive mechanisms underlying value-based decision-making. Using a novel sequential decision-making task, the first experimental chapter presents robust markers of model-based decision-making in childhood (N = 85), which reflects the ability to plan through a sequential task structure, contrary to previous developmental studies. Using the same paradigm, in a new sample via both behavioral (N = 69) and MRI-based measures (N = 44), the second experimental chapter explores the neurocognitive mechanisms that may underlie model-based decision-making and its metacontrol in childhood and links individual differences in inhibition and cortical thickness to metacontrol. The third experimental chapter explores the potential plasticity of social and intertemporal decision-making in a longitudinal executive function training paradigm (N = 205) and initial relationships with executive functions. Finally, I critically discuss the results presented in this thesis and their implications and outline directions for future research in the neurocognitive underpinnings of decision-making during development

Temporal Decision-Making: Insights from Cognitive Neuroscience

Decisions frequently have consequences that play out over time and these temporal factors can exert strong influences on behavior. For example, decision-makers exhibit delay discounting, behaving as though immediately consumable goods are more valuable than those available only after some delay. With the use of functional magnetic resonance imaging, we are now beginning to characterize the physiological bases of such behavior in humans and to link work on this topic from neuroscience, psychology, and economics. Here we review recent neurocognitive investigations of temporal decision-making and outline the theoretical picture that is beginning to take shape. Taken as a whole, this body of work illustrates the progress made in understanding temporal choice behavior. However, we also note several questions that remain unresolved and areas where future work is needed

The role of simulation in intertemporal choices

One route to understanding the thoughts and feelings of others is by mentally putting one's self in their shoes and seeing the world from their perspective, i.e., by simulation. Simulation is potentially used not only for inferring how others feel, but also for predicting how we ourselves will feel in the future. For instance, one might judge the worth of a future reward by simulating how much it will eventually be enjoyed. In intertemporal choices between smaller immediate and larger delayed rewards, it is observed that as the length of delay increases, delayed rewards lose subjective value; a phenomenon known as temporal discounting. In this article, we develop a theoretical framework for the proposition that simulation mechanisms involved in empathizing with others also underlie intertemporal choices. This framework yields a testable psychological account of temporal discounting based on simulation. Such an account, if experimentally validated, could have important implications for how simulation mechanisms are investigated, and makes predictions about special populations characterized by putative deficits in simulating others

Age differences in intertemporal choice among children, adolescents, and adults

When choosing between sooner–smaller and later–larger rewards (i.e., intertemporal choices), adults typically prefer later–larger rewards more often than children. Intertemporal choice preferences have been implicated in various impulsivity-related psychopathologies, making it important to understand the underlying mechanisms not only in terms of how reward magnitude and delay affect choice but also in terms of how these mechanisms develop across age. We administered an intertemporal choice paradigm to 60 children (8–11 years), 79 adolescents (14–16 years), and 60 young adults (18–23 years). The paradigm systematically varied amounts and delays of the available rewards, allowing us to identify mechanisms underlying age-related differences in patience. Compared with young adults, both children and adolescents made fewer later–larger choices. In terms of underlying mechanisms, variation in delays, absolute reward magnitudes, and relative amount differences affected choice in each age group, indicating that children showed sensitivity to the same choice-relevant factors as young adults. Sensitivity to both absolute reward magnitude and relative amount differences showed a further monotonic age-related increase, whereas no change in delay sensitivity occurred. Lastly, adolescents and young adults weakly displayed a present bias (i.e., overvaluing immediate vs. future rewards; nonsignificant and trend, respectively), whereas children showed a nonsignificant but opposite pattern, possibly indicating that specifically dealing with future rewards changed with age. These findings shed light on the underlying mechanisms that contribute to the development of patience. By decomposing overt choices, our results suggest that the age-related increase in patience may be driven specifically by stronger sensitivity to amount differences with age.</p