Encoding of Marginal Utility across Time in the Human Brain

Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an intertemporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging, we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behavior. Furthermore, during choice, we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness

Escalating risk and the moderating effect of resistance to peer influence on the P200 and feedback-related negativity

Young people frequently socialize together in contexts that encourage risky decision making, pointing to a need for research into how susceptibility to peer influence is related to individual differences in the neural processing of decisions during sequentially escalating risk. We applied a novel analytic approach to analyze EEG activity from college-going students while they completed the Balloon Analogue Risk Task (BART), a well-established risk-taking propensity assessment. By modeling outcome-processing-related changes in the P200 and feedback-related negativity (FRN) sequentially within each BART trial as a function of pump order as an index of increasing risk, our results suggest that analyzing the BART in a progressive fashion may provide valuable new insights into the temporal neurophysiological dynamics of risk taking. Our results showed that a P200, localized to the left caudate nucleus, and an FRN, localized to the left dACC, were positively correlated with the level of risk taking and reward. Furthermore, consistent with our hypotheses, the rate of change in the FRN was higher among college students with greater self-reported resistance to peer influence

Activity in Inferior Parietal and Medial Prefrontal Cortex Signals the Accumulation of Evidence in a Probability Learning Task

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes

AMYGDALA AND VENTRAL STRIATAL REACTIVITY IN ADOLESCENTS AT HIGH-RISK FOR DEPRESSION

Previous research has shown that depression clusters within families. Adolescents from these families (i.e., high-risk) have approximately a three-fold increased risk of developing depression, an earlier mean age at onset, and greater lifetime morbidity in comparison with low-risk adolescents. Understanding the developmental pathways and mechanisms of susceptibility to depression, especially at the level of neurobiological circuits, is critical for the development of more effective intervention and prevention strategies, particularly in high-risk adolescents. The current study examined the functional reactivity of affect- and reward-related neural circuitries in high-risk and low-risk adolescents, as well as the functional coupling between regions of PFC and amygdala and ventral striatum. Adolescents (aged 12-15 years)—stratified according to familial history of depression (i.e., high- and low-risk)—completed two fMRI paradigms known to reliably elicit threat-related amygdala and reward-related ventral striatal reactivity, respectively. Using a conservative threshold, employed because of the very large sample size (> 300 adolescents), the present analyses failed to detect significant differences between these groups at the level of the amygdala and ventral striatum. When a more liberal threshold was applied, hypothesized differences were observed for both the amygdala reactivity paradigm and the ventral striatal reactivity paradigm: high-risk adolescents displayed relatively greater amygdala reactivity and relatively blunted VS reactivity compared to low-risk adolescents. Additionally, these data offer some evidence to suggest that alterations in functional connectivity of the threat-related amygdala reactivity network (but not reward-related VS reactivity) may vary as a function of risk status during adolescence

Art for reward's sake: Visual art recruits the ventral striatum

A recent study showed that people evaluate products more positively when they are physically associated with art images than similar non-art images. Neuroimaging studies of visual art have investigated artistic style and esthetic preference but not brain responses attributable specifically to the artistic status of images. Here we tested the hypothesis that the artistic status of images engages reward circuitry, using event-related functional magnetic resonance imaging (fMRI) during viewing of art and non-art images matched for content. Subjects made animacy judgments in response to each image. Relative to non-art images, art images activated, on both subject- and item-wise analyses, reward-related regions: the ventral striatum, hypothalamus and orbitofrontal cortex. Neither response times nor ratings of familiarity or esthetic preference for art images correlated significantly with activity that was selective for art images, suggesting that these variables were not responsible for the art-selective activations. Investigation of effective connectivity, using time-varying, wavelet-based, correlation-purged Granger causality analyses, further showed that the ventral striatum was driven by visual cortical regions when viewing art images but not non-art images, and was not driven by regions that correlated with esthetic preference for either art or non -art images. These findings are consistent with our hypothesis, leading us to propose that the appeal of visual art involves activation of reward circuitry based on artistic status alone and independently of its hedonic value

Recurrent, Robust and Scalable Patterns Underlie Human Approach and Avoidance

BACKGROUND. Approach and avoidance behavior provide a means for assessing the rewarding or aversive value of stimuli, and can be quantified by a keypress procedure whereby subjects work to increase (approach), decrease (avoid), or do nothing about time of exposure to a rewarding/aversive stimulus. To investigate whether approach/avoidance behavior might be governed by quantitative principles that meet engineering criteria for lawfulness and that encode known features of reward/aversion function, we evaluated whether keypress responses toward pictures with potential motivational value produced any regular patterns, such as a trade-off between approach and avoidance, or recurrent lawful patterns as observed with prospect theory. METHODOLOGY/PRINCIPAL FINDINGS. Three sets of experiments employed this task with beautiful face images, a standardized set of affective photographs, and pictures of food during controlled states of hunger and satiety. An iterative modeling approach to data identified multiple law-like patterns, based on variables grounded in the individual. These patterns were consistent across stimulus types, robust to noise, describable by a simple power law, and scalable between individuals and groups. Patterns included: (i) a preference trade-off counterbalancing approach and avoidance, (ii) a value function linking preference intensity to uncertainty about preference, and (iii) a saturation function linking preference intensity to its standard deviation, thereby setting limits to both. CONCLUSIONS/SIGNIFICANCE. These law-like patterns were compatible with critical features of prospect theory, the matching law, and alliesthesia. Furthermore, they appeared consistent with both mean-variance and expected utility approaches to the assessment of risk. Ordering of responses across categories of stimuli demonstrated three properties thought to be relevant for preference-based choice, suggesting these patterns might be grouped together as a relative preference theory. Since variables in these patterns have been associated with reward circuitry structure and function, they may provide a method for quantitative phenotyping of normative and pathological function (e.g., psychiatric illness).National Institute on Drug Abuse (14118, 026002, 026104, DABK39-03-0098, DABK39-03-C-0098); The MGH Phenotype Genotype Project in Addiction and Mood Disorder from the Office of National Drug Control Policy - Counterdrug Technology Assessment Center; MGH Department of Radiology; the National Center for Research Resources (P41RR14075); National Institute of Neurological Disorders and Stroke (34189, 05236

The neural foundations of financial decision making

Diese Diplomarbeit beschäftigt sich mit den neuronalen Korrelaten von Investorenentscheidungen (''Neurofinance''). Die prägnante Erkenntnis, daß finanzielle Entscheidungen mit Unsicherheiten verknüpft sind, dient als Ausgangsthese des ersten Kapitels dieser Arbeit. Es enthält zunächst eine Vorstellung und anschließend eine synoptische Gegenüberstellung der Erwartungsnutzentheorie und der Prospect Theorie. Weiters werden systematische Verzerrungen im menschlichen Entscheidungsverhalten ausführlich betrachtet. Das zweite Kapitel behandelt die funktionelle Anatomie des Cerebrum, der Schwerpunkt der Untersuchung liegt aber in der Darstellung der dopaminergen Innervation des Mittelhirns. Essentiell ist nicht zuletzt aufgrund der extremen Komplexität (sub-)kortikaler Strukturen die Abgrenzung des frontalen Assoziationskortex gegenüber anderen neuronalen Strukturen. Insbesondere gilt festzuhalten, daß der dorsale und laterale frontale Assoziationskortex zu den phylogenetisch jüngsten Hirnregionen zählen. Das letzte Kapitel dieser Arbeit beschäftigt sich mit der Wahrnehmung von Risiken und Belohnungen im Kontext finanzieller Entscheidungen. Es werden jüngere Studien mit funktionellen bildgebenden Verfahren vorgestellt, die einerseits auf der Erwartungsnutzentheorie, und andererseits auf dem Erwartungswert-Varianz-Prinzip aufbauend, sich mit der Lokalisierung von neuronalen Korrelaten dieser Modelle befassen. Die Forschungsergebnisse bestätigen nicht exklusiv das Vorhandensein der spatio-temporalen Separation in der Wahrnehmung von Risiken und Belohnungen, vielmehr zeigen sie, daß es sich um eine entweder/oder Abbildung handelt. Im zweiten Teil des dritten Kapitels wird, basierend auf neuropsychologischen Untersuchungen von Patienten mit selektiven bilateralen Läsionen die eine differenzierte Erregung der Amygdala und des orbitofrontalen Kortex feststellen, gezeigt, daß emotionale Variablen bei der rationalen Entscheidungsfindung unverzichtbar sind

The perception of the affective expressions of anger and fear: evidence for fast and slow processing of threatening faces

Ein Gesichtsausdruck von Ärger wird leichter erkannt und löst eine stärkere affektive Reaktion in einem Betrachter aus, wenn Blickkontakt vorliegt. Ein angstvoller Ausdruck hat hingegen eine stärkere Wirkung, wenn der Blick der betrachteten Person auf die Umgebung gerichtet ist. Eine Interpretation dieses Befundes zielt auf die Bedrohlichkeit der betrachteten Gesichter ab: Ärger legt bei direktem Blick eine unmittelbare Bedrohung nahe, Angst weist bei abgewandtem Blick auf eine mögliche Bedrohung in einer geteilten Umwelt hin. Die Verarbeitung von Bedrohungsinformationen aus Gesichtern könnte durch ein spezialisiertes mentales und hirnphysiologisches Organ, ein sogenanntes „Modul“ erfolgen, das unabhängig von Bewusstsein operiert und in dem die Amygdala eine zentrale Rolle einnimmt. Die Amygdala ist eine Hirnstruktur, die mit schneller Gefahrenerkennung und unwillkürlichen Schutzreaktionen in Verbindung gebracht wird. Unklar war bislang, ob sich die genannten Reize tatsächlich durch eine vergleichsweise stärkere Bedrohlichkeit auszeichnen, und ob der obige Interaktionsbefund – hinsichtlich der Aktivierung der Amygdala und des Verhalten von Versuchspersonen – auch in Abwesenheit voll ausgeprägten Reizbewusstseins vorliegt. Das Projekt hat diese Fragen untersucht. Zunächst konnte gezeigt werden, dass die Stärke der Schreckreaktion, die während der Betrachtung von Bildern ausgelöst wird, die Bedrohlichkeit widerspiegelt, die Versuchspersonen subjektiv durch sie empfinden. Anhand der Schreckreaktion als einem objektiven Bedrohungsmaß konnte anschließend bestätigt werden, dass Ärgerausdrücke mit direktem Blick (Blickkontakt) und Angstausdrücke mit abgewandtem Blick eine stärkere Bedrohung hervorrufen als die beiden komplementären Reizbedingungen. In einem dritten Experiment wurden diese Reize schließlich so präsentiert, dass sie keinen voll ausgeprägten Bewusstheitseindruck hervorrufen konnten. Dies geschah durch ihre sehr kurze Darbietung einschließlich eines unmittelbar darauf folgenden zweiten Reizes, der den Eindruck des ersten zeitlich überlagerte. Trotz dieser erschwerten Sehbedingungen zeigten Versuchspersonen im Erkennen der Reizkategorien Unterschiede, die mit der bevorzugten Verarbeitung besonders bedrohlicher Gesichter – Ärgerausdrücke mit direktem und Angstausdrücke mit abgewandtem Blick – kompatibel sind. Zusätzlich wurde in diesem dritten Experiment die Beteiligung verschiedener Hirnstrukturen untersucht. Für die Amygdala zeigte sich ein uneinheitlicher Befund: Während Ärgerausdrücke die Amygdala unabhängig von einem bewussten Seheindruck zu aktivieren schienen, waren Angstausdrücke offenbar auf einen solchen angewiesen. Dieses zweite Ergebnis passt zu Befunden anderer Forscher, die eine Aktivierung der Amygdala durch beobachtete Angst (im Gegensatz zu Ärger) mit der aufmerksamkeitsbasierten Lokalisation möglicher Gefahren in Verbindung bringen. Somit legt diese Arbeit nahe, dass zumindest für Ärgerausdrücke der Einfluss der Blickrichtung auf ihre Verarbeitung auf der Funktionsweise eines spezialisierten Moduls zur Bedrohungsermittlung beruht. Blickrichtungseinflüsse auf die Angstwahrnehmung hingegen scheinen anteilig auf bewusster Reizverarbeitung zu beruhen, auch wenn diese ebenfalls zu gesteigerter Bedrohlichkeit führen kann. Für die funktionale Rolle der Amygdala muss dies kein Widerspruch sein, da sie eine komplexe Struktur ist und daher an unterschiedlichen Hirnfunktionen beteiligt sein kann.The literature reports an interaction effect of facial expression (anger/fear) and gaze direction (direct/averted) on perceptual (detection scores), affective (ratings), and neurophysiological (BOLD response of the amygdala) measures: direct-gaze angry and averted fearful faces are particularly salient. The purpose of this work was to assess, to what extent this effect is mediated by threat. This was done by examining startle potentiation by the overt (experiment 1/2), as well as signal detection and brain activation by the covert (backward masked) presentation of affective faces with varying in gaze direction. The results suggest the advantage of direct-gaze anger (vs. averted) to be mediated by threat. Regarding fear the situation is twofold: fast perceptual effects (detection despite masking) seem to have occurred, but they are not mediated by activation of the amygdala. The latter was observed, but only after perceptual learning had taken place. This finding relates amygdalar activation by fearful faces to the processing of the more general self-relevance of these stimuli. Therefore the data suggest the amygdala to be involved in both: the specific detection of threat as well as the more general processing of biological or social significance

Prediction error dependent changes in brain connectivity during associative learning

One of the fundaments of associative learning theories is that surprising events drive learning by signalling the need to update one’s beliefs. It has long been suggested that plasticity of connection strengths between neurons underlies the learning of predictive associations: Neural units encoding associated entities change their connectivity to encode the learned associative strength. Surprisingly, previous imaging studies have focused on correlations between regional brain activity and variables of learning models, but neglected how these variables changes in interregional connectivity. Dynamic Causal Models (DCMs) of neuronal populations and their effective connectivity form a novel technique to investigate such learning dependent changes in connection strengths. In the work presented here, I embedded computational learning models into DCMs to investigate how computational processes are reflected by changes in connectivity. These novel models were then used to explain fMRI data from three associative learning studies. The first study integrated a Rescorla-Wagner model into a DCM using an incidental learning paradigm where auditory cues predicted the presence/absence of visual stimuli. Results showed that even for behaviourally irrelevant probabilistic associations, prediction errors drove the consolidation of connection strengths between the auditory and visual areas. In the second study I combined a Bayesian observer model and a nonlinear DCM, using an fMRI paradigm where auditory cues differentially predicted visual stimuli, to investigate how predictions about sensory stimuli influence motor responses. Here, the degree of striatal prediction error activity controlled the plasticity of visuo-motor connections. In a third study, I used a nonlinear DCM and data from a fear learning study to demonstrate that prediction error activity in the amygdala exerts a modulatory influence on visuo-striatal connections. Though postulated by many models and theories about learning, to our knowledge the work presented in this thesis constitutes the first direct report that prediction errors can modulate connection strength