Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis

Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies

μ Opioid Antagonist Naltrexone Partially Abolishes the Antidepressant Placebo Effect and Reduces Orbitofrontal Cortex Encoding of Reinforcement

Background: Like placebo analgesia, the antidepressant placebo effect appears to involve cortical and subcortical endogenous opioid signaling, yet the mechanism through which opioid release affects mood remains unclear. The orbitofrontal cortex (OFC)—which integrates various attributes of a stimulus to predict associated outcomes—has been implicated in placebo effects and is rich in μ opioid receptors. We hypothesized that naltrexone blockade of μ opioid receptors would blunt OFC-dependent antidepressant placebo effects. Methods: Twenty psychotropic-free patients with major depressive disorder completed a randomized, double-blind, placebo-controlled crossover study of 1 oral dose of 50 mg of naltrexone or matching placebo immediately before completing 2 sessions of the antidepressant placebo functional magnetic resonance imaging task. This task manipulates placebo-associated expectancies and their reinforcement while assessing expected and actual mood improvement. Results: Behaviorally, manipulations of antidepressant placebo expectancies and their reinforcement had positive, interactive effects on participants’ expectancy and mood ratings. The high-expectancy condition recruited the dorsolateral and ventrolateral prefrontal cortex, as well as dorsal attention stream regions. Interestingly, increased dorsolateral and ventrolateral prefrontal cortex brain responses appeared to attenuate the antidepressant placebo effect. The administration of 1 oral dose of naltrexone, compared with placebo, partially abolished the interaction of the expectancy and reinforcement manipulation on mood and blocked reinforcement-induced responses in the right central OFC. Conclusions: Our results show preliminary evidence for the role of μ opioid central OFC modulation in antidepressant placebo effects by positively biasing the value of placebo based on reinforcement and enhancing subsequent hedonic experiences.12 month embargo; available online 06 March 2021This item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Personality and Suicidal Behavior in Old Age: A Systematic Literature Review

BackgroundSuicide rates generally peak in the second half of life and are particularly high in older men; however, little is known about the contribution of dispositional factors to late-life suicide. Maladaptive personality traits have been strongly implicated in suicide among younger adults, but the extent to which they continue to play a role in late-life suicidal behavior is unclear. We also do not know whether specific personality profiles interact with the stressors of aging to cause suicidal behavior.MethodsWe sought to synthesize the data on personality pathology in late-life suicidal ideation and behavior via a systematic review using the PubMed, Google Scholar, PsycInfo, Scopus, Ovid, Web of Science, Embase, and Cochrane search engines. The included key words related to three descriptors: “personality,” “suicide,” and “elderly.” Included articles evaluated personality based on the Five-Factor Model (FFM) or ICD/DSM diagnostic criteria in older samples with minimum age cutoffs of 50 years or older. Our original search identified 1,183 articles, of which 31 were retained.ResultsIncluded studies were heterogeneous in their design and personality measurements. Studies of categorical personality disorders were particularly scarce and suggested a stronger association with late-life suicidal ideation than with death by suicide. Only obsessive–compulsive and avoidant personality traits were associated with death by suicide in old age, but only in studies that did not control for depression. All personality constructs were positively linked to suicidal ideation, except for histrionic personality, which emerged as a negative predictor. Studies employing the FFM also indicated that older adults who died by suicide were less likely to display a maladaptive personality profile than elderly suicide attempters and younger suicide victims, having both lower levels of neuroticism and higher levels of conscientiousness than these comparison groups. Nevertheless, older suicide victims displayed lower levels of openness to experience than younger victims in two samples.ConclusionMaladaptive personality manifests in milder, subthreshold, and more heterogeneous forms in late-life vs. early-life suicide. An inability to adapt to the changes occurring in late life may help explain the association between suicide in old age and higher conscientiousness as well as obsessive–compulsive and avoidant personality disorders

Status, rivalry and admiration-seeking in narcissism and depression: A behavioral study.

Humans seek admiration to boost their social rank and engage in rivalry to protect it when fearing defeat. Traits such as narcissism and affective states such as depression are thought to influence perception of rank and motivation for dominance in opposite ways, but evidence of the underlying behavioral mechanisms is scant. We investigated the effects of dimensionally-assessed narcissism and depression on behavioral responses to social defeat in a rigged video game tournament designed to elicit rivalry (stealing points from opponents) and admiration-seeking (paying for rank). We tested an undergraduate sample (N = 70, mean age = 21.5 years) and a clinical sample of predominantly depressed elderly (N = 85, mean age = 62.6 years). Both rivalry and admiration-seeking increased with time on task and were particularly enhanced in individuals high in narcissism. Participants engaged in more rivalry when pitted against high-ranked opponents, but depression partially mitigated this tendency. Our findings provide behavioral evidence that narcissism manifests in increased rivalry and admiration-seeking during social contests. Depression does not suppress general competitiveness but selectively inhibits upward-focused rivalry