Decision Making and Reward in Frontal Cortex: Complementary Evidence From Neurophysiological and Neuropsychological Studies

Patients with damage to the prefrontal cortex (PFC)—especially the ventral and medial parts of PFC—often show a marked inability to make choices that meet their needs and goals. These decision-making impairments often reflect both a deficit in learning concerning the consequences of a choice, as well as deficits in the ability to adapt future choices based on experienced value of the current choice. Thus, areas of PFC must support some value computations that are necessary for optimal choice. However, recent frameworks of decision making have highlighted that optimal and adaptive decision making does not simply rest on a single computation, but a number of different value computations may be necessary. Using this framework as a guide, we summarize evidence from both lesion studies and single-neuron physiology for the representation of different value computations across PFC areas

Distributional reinforcement learning in prefrontal cortex

The prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories center on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in the prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses. In the present study, we show that distributional RL also better explains macaque anterior cingulate cortex neuronal responses, suggesting that it is a common mechanism for reward-guided learning

Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation

There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity

The role of the medial frontal cortex in updating action representations

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Promoting Adult Identity Development: A Feasibility Study Of A University-Based Identity Intervention Program

Targeting identity as an outcome variable, the research described here reports a preliminary evaluation of the feasibility of extending to adult university-based populations, the use of intervention change strategies (co-participatory and transformative) developed for community-based positive youth development interventions. Participants (N = 43) were enrolled in an elective personal growth psychology course at a public university. Identity exploration significantly increased, and identity distress significantly decreased. In addition, there was a significant decrease in foreclosed identity statuses and a significant increase in achieved identity statuses. Qualitative indices also suggest that the course was well received and course objectives were achieved. Copyright © Taylor & Francis Group, LLC

In the blink of an eye: Value and novelty drive saccades

Evidence accumulation is an essential component of value-based decisions. Recent human studies suggest that overt attention correlates with evidence accumulation necessary for optimal decisions. However, the influence of covert attention on decision-making remains relatively unexplored. To investigate this issue, two monkeys were trained to perform a decision-making task where they chose between two stimuli, which were either ‘Overtrained’ or learned that day (‘Novel’). Subjects could freely saccade during choice evaluation and indicated their decision by moving a joystick. Saccades were made within 170 ms of stimulus presentation and were strongly driven by both value and novelty, implying covert stimulus evaluation prior to saccade. This effect was strongest for ‘Overtrained’ choices, but rapidly emerged during learning of ‘Novel’ choices. Though novel stimuli attracted initial saccades, final decisions were guided only by value; implying attentional value comparison processes are at least partially dissociable from value comparison processes that govern final decisions. While subjects made highly optimal decisions, they frequently viewed only one stimulus; final choice was thus best explained by assuming covert evidence accumulation. Our results suggest that the primate brain contains multiple value comparison systems for guiding attention toward highly valuable or novel information while simultaneously optimizing final decision value

Encoding of Reward and Space During a Working Memory Task in the Orbitofrontal Cortex and Anterior Cingulate Sulcus

Several lines of research indicate that emotional and motivational information may be useful in guiding the allocation of attentional resources. Two areas of the frontal lobe that are particularly implicated in the encoding of motivational information are the orbital prefrontal cortex (PFo) and the dorsomedial region of prefrontal cortex, specifically the anterior cingulate sulcus (PFcs). However, it remains unclear whether these areas use this information to influence spatial attention. We used single-unit neurophysiology to examine whether, at the level of individual neurons, there was evidence for integration between reward information and spatial attention. We trained two subjects to perform a task that required them to attend to a spatial location across a delay under different expectancies of reward for correct performance. We balanced the order of presentation of spatial and reward information so we could assess the neuronal encoding of the two pieces of information independently and conjointly. We found little evidence for encoding of the spatial location in either PFo or PFcs. In contrast, both areas encoded the expected reward. Furthermore, PFo consistently encoded reward more quickly than PFcs, although reward encoding was subsequently more prevalent and stronger in PFcs. These results suggest a differential contribution of PFo and PFcs to reward encoding, with PFo potentially more important for initially determining the value of rewards predicted by sensory stimuli. They also suggest that neither PFo nor PFcs play a direct role in the control of spatial attention