A Behavioral and Neural Evaluation of Prospective Decision-Making under Risk

Making the best choice when faced with a chain of decisions requires a person to judge both anticipated outcomes and future actions. Although economic decision-making models account for both risk and reward in single-choice contexts, there is a dearth of similar knowledge about sequential choice. Classical utility-based models assume that decision-makers select and follow an optimal predetermined strategy, regardless of the particular order in which options are presented. An alternative model involves continuously reevaluating decision utilities, without prescribing a specific future set of choices. Here, using behavioral and functional magnetic resonance imaging (fMRI) data, we studied human subjects in a sequential choice task and use these data to compare alternative decision models of valuation and strategy selection. We provide evidence that subjects adopt a model of reevaluating decision utilities, in which available strategies are continuously updated and combined in assessing action values. We validate this model by using simultaneously acquired fMRI data to show that sequential choice evokes a pattern of neural response consistent with a tracking of anticipated distribution of future reward, as expected in such a model. Thus, brain activity evoked at each decision point reflects the expected mean, variance, and skewness of possible payoffs, consistent with the idea that sequential choice evokes a prospective evaluation of both available strategies and possible outcomes

Die Rolle der Zielnähe und der investierten Anstrengung für den erwarteten Wert einer Handlung

In human neuroscientific research, there has been an increasing interest in how the brain computes the value of an anticipated outcome. However, evidence is still missing about which valuation related brain regions are modulated by the proximity to an expected goal and the previously invested effort to reach a goal. The aim of this dissertation is to investigate the effects of goal proximity and invested effort on valuation related regions in the human brain. We addressed this question in two fMRI studies by integrating a commonly used reward anticipation task in differential versions of a Multitrial Reward Schedule Paradigm. In both experiments, subjects had to perform consecutive reward anticipation tasks under two different reward contingencies: in the delayed condition, participants received a monetary reward only after successful completion of multiple consecutive trials. In the immediate condition, money was earned after every successful trial. In the first study, we could demonstrate that the rostral cingulate zone of the posterior medial frontal cortex signals action value contingent to goal proximity, thereby replicating neurophysiological findings about goal proximity signals in a homologous region in non-human primates. The findings of the second study imply that brain regions associated with general cognitive control processes are modulated by previous effort investment. Furthermore, we found the posterior lateral prefrontal cortex and the orbitofrontal cortex to be involved in coding for the effort-based context of a situation. In sum, these results extend the role of the human rostral cingulate zone in outcome evaluation to the continuous updating of action values over a course of action steps based on the proximity to the expected reward. Furthermore, we tentatively suggest that previous effort investment invokes processes under the control of the executive system, and that posterior lateral prefrontal cortex and the orbitofrontal cortex are involved in an effort-based context representation that can be used for outcome evaluation that is dependent on the characteristics of the current situation.Derzeit besteht im Bereich der Neurowissenschaften ein großes Interesse daran aufzuklären, auf welche Weise verschiedene Variablen die Wertigkeit eines erwarteten Handlungsziels beeinflussen bzw. welche Hirnregionen an der Repräsentation der Wertigkeit eines Handlungsziels beteiligt sind. Die meisten Untersuchungen beziehen sich dabei auf Einflussgrößen wie die erwartete Belohnungshöhe, die Wahrscheinlichkeit, mit der ein bestimmtes Ereignis eintritt, oder die Dauer bis zum Erhalt einer Belohnung. Bisher liegen jedoch kaum Untersuchungen vor bezüglich zweier anderer Variablen, die ebenfalls den erwarteten Wert eines Handlungsergebnisses beeinflussen. Das sind (a) die Nähe zu dem erwarteten Ziel und (b) die bisher investierte Anstrengung, um ein Ziel zu erreichen. Das Ziel der vorliegenden Dissertation ist zu untersuchen, wie die Nähe zum Ziel und die bisher investierte Anstrengung Gehirnregionen beeinflussen, die mit der Repräsentation von Wertigkeit im Zusammenhang stehen. Dazu führten wir zwei fMRT-Studien durch, in denen wir eine klassische Belohnungs-Antizipationsaufgabe in unterschiedliche Versionen eines „Multitrial Reward Schedule“ Paradigmas integriert haben. Das bedeutet, dass die Probanden Belohnungs-Antizipationsaufgaben unter zwei unterschiedlichen Belohnungskontingenzen bearbeiteten: In der verzögerten Bedingung erhielten die Probanden einen Geldbetrag nach der erfolgreichen Bearbeitung von mehreren aufeinanderfolgenden Aufgaben, in der direkten Bedingung dagegen nach jeder korrekt ausgeführten Aufgabe. In der ersten Studie konnte eine sukzessiv ansteigende Aktivität in Abhängigkeit zur Zielnähe in der rostralen cingulären Zone identifiziert werden. Das deutet darauf hin, dass dieses Areal den Wert einer Handlung in Abhängigkeit zur Nähe zum Ziel kodiert. Die Ergebnisse der zweiten Studie zeigten, dass die bisher investierte Anstrengung kortikale Regionen moduliert, die klassischerweise mit kognitiven Kontrollfunktionen in Zusammenhang gebracht werden. Außerdem repräsentierten der posteriore laterale präfrontale Cortex und der orbitofrontale Cortex den motivationalen Kontext eines Trials anhand des Risikos des Verlustes von bisher investierter Anstrengung. Insgesamt weisen diese Befunde darauf hin, dass die rostrale cinguläre Zone eine entscheidende Rolle spielt für die Kontrolle sequenzieller Handlungsstufen, die auf eine verzögerte Belohnung ausgerichtet sind. Diese Kontrollfunktion scheint auf der kontinuierlichen Aktualisierung des Wertes einer Handlungsstufe zu basieren, der von der aktuellen Zielnähe bestimmt wird. Die Befunde der zweiten Studie lassen darauf schließen, dass sich die bisher investierte Anstrengung zur Erreichung eines Handlungsziels auf die Bereitstellung von allgemeinen kognitiven Ressourcen auswirkt. Das Risiko des Verlustes von bisher investierter Anstrengung kann außerdem ein kontextuelles Merkmal der Situation darstellen, das als Bezugsrahmen für die Evaluation des erwarteten Wertes dienen kann

Neurocomputational mechanisms underlying effort-based value integration

In everyday life, we encounter many decisions requiring the consideration of prospective effort, such as taking exercise and altruistic behaviors. Therefore, the ability to accurately weigh effort costs against potential rewards is critical for optimal goal-directed behavior. The common currency theory proposes that values of different options are mapped to a common scale by a neural network to ensure efficient decision-making across different cost types. This theory provides a general framework to explain how rewards and costs are integrated and has gained popularity in decision-making associated with other types of cost, such as risk and delay. Although a few studies have examined the computational and neural mechanisms underlying effort-based value integration, it remains unclear if effort discounts prospective outcomes in a similar way to other costs, and, at the neural level, it is still under debate if effort-based value integration engages a general valuation neural network as suggested by the common currency theory, or instead relies on a specific network compared with other cost domains. In this dissertation, I address these questions across a meta-analytic study and two empirical studies. In study 1 (Lopez-Gamundi et al., 2021) of this dissertation, we conducted two separate meta-analyses to examine consistent neural correlates of effort-reward integration or raw effort requirement in related fMRI studies. We found that the vmPFC activity scaled positively with net value but negatively with raw effort. On the other hand, the dmPFC was also identified in both analyses, but its activity scaled negatively with net value and positively with raw effort. These findings are generally consistent with previous findings in other cost domains. In study 2 (Yao et al., 2022), to directly test if the common currency theory could be applied to value integration during effort-based decision-making, we reanalyzed the choice behavior and fMRI data of an open-access dataset, which included both effort-based and risky (one-option) decision-making tasks. Using computational modeling, we found that effort and risk showed distinct discounting effects on prospective outcomes. At the neural level, we conducted multivariate decoding analyses and found that a large cluster including both the vmPFC and dmPFC represented subjective value independent of cost types. In study 3 (Yao et al., 2022), we examined the replicability of the findings of Study 2 in an independent sample of participants. Moreover, to maintain similar overall acceptance rates between tasks, we estimated participant-specific indifference points for all combinations of rewards and costs (effort or risk) before scanning and manipulated the amounts of smaller rewards around these indifference points during scanning. We confirmed that effort and risk distinctively devalued rewards. At the neural level, we found that the dmPFC represented subjective value in a task-independent manner. Taken together, these findings highlight the role of the dmPFC in subjective value computation across effort-based and risky decision-making. Finally, I discuss how these results may reconcile the ongoing debates on the neural mechanisms underlying effort-reward integration and outline potential implications for the common currency theory.Im Alltag sind wir mit vielen Entscheidungen konfrontiert, die eine Abwägung des voraussichtlichen Aufwands erfordern, wie z. B. sportliche Aktivitäten und altruistisches Verhalten. Daher ist die Fähigkeit, die Kosten des Aufwands mit den potenziellen Belohnungen genau abzuwägen, entscheidend für optimales zielgerichtetes Verhalten. Die Theorie der gemeinsamen Währung besagt, dass die Werte verschiedener Optionen auf einer gemeinsamen Skala abgebildet werden, um eine effiziente Entscheidungsfindung über verschiedene Kostenarten hinweg zu gewährleisten. Diese Theorie bietet einen allgemeinen Rahmen, um zu erklären, wie Belohnungen und Kosten integriert werden, und hat bei der Entscheidungsfindung im Zusammenhang mit anderen Kostenarten wie Risiko und Verzögerung an Popularität gewonnen. Obwohl einige Studien die komputationalen und neuronalen Mechanismen untersucht haben, die der aufwandsbasierten Integration subjektiver Werte zugrunde liegen, bleibt unklar, ob Aufwand erwartete Ereignisse in ähnlicher Weise wie andere Kosten diskontiert. Weiterhin wird auf neuronaler Ebene diskutiert, ob die aufwandsbasierte Wertintegration auf ein allgemeines neuronales Bewertungsnetzwerk zurückzuführen ist, wie es die Theorie der gemeinsamen Währung nahelegt oder stattdessen auf ein spezifisches Netzwerk im Vergleich zu anderen Arten von Kosten zurückgreift. In dieser Dissertation befasse ich mich mit diesen Fragen im Rahmen einer Meta-Analyse früherer Studien und zweier empirischer Studien. In der ersten Dissertationsstudie (Lopez-Gamundi et al., 2021) haben wir zwei separate Meta-Analysen durchgeführt, um neuronale Korrelate der Anstrengungs-Belohnungs-Integration bzw. reiner Anstrengungsanforderung in verwandten fMRI-Studien zu untersuchen. Wir fanden heraus, dass die Aktivität des ventromedialen präfrontalen Kortex (vmPFC) positiv mit subjektiven Werten, aber negativ mit reinen Aufwandsanforderungen skaliert. Andererseits wurde der dorsomediale präfrontale Kortex (dmPFC) in beiden Analysen identifiziert, zeigte aber ein entgegengesetztes Aktivitätsmuster. Diese Ergebnisse stimmen im Allgemeinen mit früheren Erkenntnissen bei anderen Arten von Kosten überein. In Studie 2 (Yao et al., 2022) haben wir das Wahlverhalten und die Daten eines frei zugänglichen Datensatzes mittels funktioneller Magnetresonanztomografie (fMRT), der sowohl anstrengungsbasierte als auch riskante (eine Option) Entscheidungsaufgaben enthielt, erneut analysiert, um direkt zu prüfen, ob die Theorie der gemeinsamen Währung auf die Wertintegration bei anstrengungsbasierten Entscheidungen angewendet werden kann. Mithilfe von Computermodellen fanden wir heraus, dass Anstrengung und Risiko unterschiedliche Diskontierungseffekte auf prospektive Ergebnisse haben. Auf neuronaler Ebene führten wir multivariate Dekodierungsanalysen durch und fanden heraus, dass ein großes Cluster, das sowohl den vmPFC als auch den dmPFC umfasst, den subjektiven Wert unabhängig von der Art der Kosten repräsentiert. In Studie 3 (Yao et al., 2022) untersuchten wir die Replizierbarkeit der Ergebnisse aus Studie 2 mit einer unabhängigen Stichprobe von Versuchspersonen. Um ein ähnliches Entscheidungsverhalten zwischen den Aufgaben sicherzustellen, haben wir vor dem Scannen teilnehmerspezifische Indifferenzpunkte für alle Kombinationen von Belohnungen und Kosten (Aufwand oder Risiko) geschätzt. Zudem haben wir während des Scannens die Beträge kleinerer Belohnungen um diese Indifferenzpunkte manipuliert. Hierdurch konnten wir bestätigten, dass Aufwand und Risiko die Belohnungen deutlich abwerteten. Auf neuronaler Ebene stellten wir fest, dass der dmPFC den subjektiven Wert aufgabenunabhängig repräsentiert. Zusammengenommen betonen die Ergebnisse die Rolle des dmPFC bei der subjektiven Wertberechnung unter Anstrengung und Risiko. Abschließend erörtere ich, wie diese Ergebnisse die laufenden Debatten über die neuronalen Mechanismen der Integration von Aufwand und Belohnung in Einklang bringen können und skizziere mögliche Implikationen für die Theorie der gemeinsamen Währung

Neural Basis for Economic Saving Strategies in Human Amygdala-Prefrontal Reward Circuits.

Economic saving is an elaborate behavior in which the goal of a reward in the future directs planning and decision-making in the present. Here, we measured neural activity while subjects formed simple economic saving strategies to accumulate rewards and then executed their strategies through choice sequences of self-defined lengths. Before the initiation of a choice sequence, prospective activations in the amygdala predicted subjects' internal saving plans and their value up to two minutes before a saving goal was achieved. The valuation component of this planning activity persisted during execution of the saving strategy and predicted subjects' economic behavior across different tasks and testing days. Functionally coupled amygdala and prefrontal cortex activities encoded distinct planning components that signaled the transition from saving strategy formation to execution and reflected individual differences in saving behavior. Our findings identify candidate neural mechanisms for economic saving in amygdala and prefrontal cortex and suggest a novel planning function for the human amygdala in directing strategic behavior toward self-determined future rewards

The subjective value of cognitive effort is encoded by a domain-general valuation network

Cognitive control is necessary for goal-directed behavior, yet people treat cognitive control demand as a cost, which discounts the value of rewards in a similar manner as other costs, such as delay or risk. It is unclear, however, whether the subjective value (SV) of cognitive effort is encoded in the same putatively domain-general brain valuation network implicated in other cost domains, or instead engages a distinct frontoparietal network, as implied by recent studies. Here, we provide rigorous evidence that the valuation network, with core foci in the ventromedial prefrontal cortex and ventral striatum, also encodes SV during cognitive effort-based decision-making in healthy, male and female adult humans. We doubly dissociate this network from frontoparietal regions that are instead recruited as a function of decision difficulty. We show that the domain-general valuation network jointly and independently encodes both reward benefits and cognitive effort costs. We also demonstrate that cognitive effort SV signals predict choice and are influenced by state and trait motivation, including sensitivity to reward and anticipated task performance. These findings unify cognitive effort with other cost domains, and suggest candidate neural mechanisms underlying state and trait variation in willingness to expend cognitive effort

Prospective effort and choice

We constantly face the challenge of selecting among actions in pursuit of our goals. Behavioral theories suggest the ubiquity of these choices necessitate a valuation process that integrates expected costs and outcomes. Increased sensitivity to costs in value-based choices, such as reduced willingness to tolerate risk, wait or work for rewards, features prominently in the symptomatology of mental illness. Contrary to classical theories of choice that do not distinguish among cost type, the recent unification of experimental economics, psychology and neuroscience describes the influence of specific costs upon behavioral and neural correlates of subjective value. Despite substantial progress in the understanding of the basis of subjective valuation under delay and risk, the specific influence of effort, the energetic cost of an action, remains largely unknown. Limited existing accounts hypothesize that cost sensitivity during subjective valuation results from separable neural systems related to risk, delay and effort. This dissertation evaluates evidence for distinct neural representation of effort and presents a set of experiments designed to refine normative accounts of effort-based choice. First, I review the neural basis of economic choice under risk and delay. Second, I review limited accounts economic choice under effort. I describe a novel prospective effort task designed and validated to examine effort-based valuation and address potential confounds present in previous studies. In the first experiment, I report novel evidence for discounting of neural activity related to value by prospective effort and conjoint sensitivity to effort costs and expected outcomes in brain regions related to selection and generation of actions. In a second experiment, I examined the influence of prospective effort costs upon delay discounting preferences. This experiment did not find modulation of individual preferences by prospective effort costs. Finally, I discuss our results in the context of existing accounts and potential extensions of the prospective effort paradigm. Overall, I show that prospective effort imposes a specific cost, reflected in behavioral and neural correlates of value, and presents a novel approach to further the understanding of motivated behavior.Neuroscienc

The Neural Representation of Value and individual Differences in Human Intertemporal Choice

Intertemporal choices, or decisions that involve tradeoffs between rewards and time, are ubiquitous in our daily lives. The tendency to devalue, or discount, future rewards has been linked to maladaptive long-term health and financial outcomes. Despite their broad clinical relevance, individual differences in discounting preferences are poorly understood. In this thesis, we make progress on the understanding of the neural basis of these decisions and factors that affect individual differences. The first two chapters focus on neurobiology. Chapter 2 investigates the decision-related variables that best explain the observed patterns of BOLD activity in ventromedial prefrontal cortex (VMPFC) and ventral striatum (VS) during intertemporal choice. We find that these regions carry different signals and likely contribute to different stages of the choice process. Across the brain, we find four kinds of value-responsive regions, each carrying different combinations of value-related signals. Next, we examine whether we can predict participants\u27 choices from any or all of these groups of regions, and find that we can predict choice from most value-responsive regions, with interesting exceptions. In Chapter 3, we identify a novel brain predictor of individual differences in discounting. When participants are making judgments about how far away some number of days feels, discount rates, measured a week later, can be predicted from how VMPFC and VS respond as a function of temporal distance. This difference in the basic response to delayed time intervals could be a target for interventions aiming to reduce discount rates. In the final chapter, we find several behavioral manipulations that are able to reduce discount rates persistently and to a significant degree. We find that there is a general lack of knowledge about the normative strategy in the monetary discounting task, and that providing information about this strategy - to accept all delayed offers that provide higher interest rates than one could obtain elsewhere - reduces discounting significantly, for at least one month. Information about peers\u27 strategies for making these decisions also reduces discounting. Taken together, this work advances our understanding of individual differences in discounting and further suggests interventions that could be used to reduce discounting

Differences in Behavior and Brain Activity during Hypothetical and Real Choices

Real behaviors are binding consequential commitments to a course of action, such as harming another person, buying an Apple watch, or fleeing from danger. Cognitive scientists are generally interested in the psychological and neural processes that cause such real behavior. However, for practical reasons, many scientific studies measure behavior using only hypothetical or imagined stimuli. Generalizing from such studies to real behavior implicitly assumes that the processes underlying the two types of behavior are similar. We review evidence of similarity and differences in hypothetical and real mental processes. In many cases, hypothetical choice tasks give an incomplete picture of brain circuitry that is active during real choice

Training motor responses to food: A novel treatment for obesity targeting implicit processes.

PublishedREVIEWJOURNAL ARTICLEThis is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.The present review first summarizes results from prospective brain imaging studies focused on identifying neural vulnerability factors that predict excessive weight gain. Next, findings from cognitive psychology experiments evaluating various interventions involving food response inhibition training or food response facilitation training are reviewed that appear to target these neural vulnerability factors and that have produced encouraging weight loss effects. Findings from both of these reviewed research fields suggest that interventions that reduce reward and attention region responses to high calorie food cues and increase inhibitory region responses to high calorie food cues could prove useful in the treatment of obesity. Based on this review, a new conceptual model is presented to describe how different cognitive training procedures may contribute to modifying eating behavior and important directions for future research are offered. It is concluded that there is a need for evaluating the effectiveness of more intensive food response training interventions and testing whether adding such training to extant weight loss interventions increases their efficacy