Ventromedial Prefrontal Cortex Activation Is Associated with Memory Formation for Predictable Rewards

During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC

A Common Mechanism for Adaptive Scaling of Reward and Novelty

Declarative memory is remarkably adaptive in the way it maintains sensitivity to relative novelty in both unknown and highly familiar environments. However, the neural mechanisms underlying this contextual adaptation are poorly understood. On the basis of emerging links between novelty processing and reinforcement learning mechanisms, we hypothesized that responses to novelty will be adaptively scaled according to expected contextual probabilities of new and familiar events, in the same way that responses to prediction errors for rewards are scaled according to their expected range. Using functional magnetic resonance imaging in humans, we show that the influence of novelty and reward on memory formation in an incidental memory task is adaptively scaled and furthermore that the BOLD signal in orbital prefrontal and medial temporal cortices exhibits concomitant scaled adaptive coding. These findings demonstrate a new mechanism for adjusting gain and sensitivity in declarative memory in accordance with contextual probabilities and expectancies of future events. Hum Brain Mapp, 2010. © 2010 Wiley-Liss, Inc

Anticipation of novel environments enhances memory for incidental information

Novelty is a potent driver of learning, but little is known about whether anticipation of novelty can enhance memory for incidental information. Here, participants incidentally encountered objects while they actively navigated toward novel or previously familiarized virtual rooms. Across immediate and delayed surprise memory tests, participants showed superior recollection for incidental objects encountered while anticipating novel as compared with familiarized rooms. Furthermore, memory for incidental objects correlated positively with between-participants average curiosity about novel rooms but negatively with within-participants trial-specific curiosity. Our findings contribute to the growing literature on how salient processes impact memory for incidental material

The role of hippocampal mossy cells in novelty detection

At the encounter with a novel environment, contextual memory formation is greatly enhanced, accompanied with increased arousal and active exploration. Although this phenomenon has been widely observed in animal and human daily life, how the novelty in the environment is detected and contributes to contextual memory formation has lately started to be unveiled. The hippocampus has been studied for many decades for its largely known roles in encoding spatial memory, and a growing body of evidence indicates a differential involvement of dorsal and ventral hippocampal divisions in novelty detection. In this brief review article, we discuss the recent findings of the role of mossy cells in the ventral hippocampal moiety in novelty detection and put them in perspective with other novelty-related pathways in the hippocampus. We propose a mechanism for novelty-driven memory acquisition in the dentate gyrus by the direct projection of ventral mossy cells to dorsal dentate granule cells. By this projection, the ventral hippocampus sends novelty signals to the dorsal hippocampus, opening a gate for memory encoding in dentate granule cells based on information coming from the entorhinal cortex. We conclude that, contrary to the presently accepted functional independence, the dorsal and ventral hippocampi cooperate to link the novelty and contextual information, and this dorso-ventral interaction is crucial for the novelty-dependent memory formation

Altered neural activity to monetary reward/loss processing in episodic migraine

The dysfunctions of the mesolimbic cortical reward circuit have been proposed to contribute to migraine pain. Although supporting empirical evidence was mainly found in connection with primary rewards or in chronic migraine where the pain experience is (almost) constant. Our goal however was to investigate the neural correlates of secondary reward/loss anticipation and consumption using the monetary incentive delay task in 29 episodic migraine patients and 41 headache-free controls. Migraine patients showed decreased activation in one cluster covering the right inferior frontal gyrus during reward consumption compared to controls. We also found significant negative correlation between the time of the last migraine attack before the scan and activation of the parahippocampal gyrus and the right hippocampus yielded to loss anticipation. During reward/loss consumption, a relative increase in the activity of the visual areas was observed the more time passed between the last attack and the scan session. Our results suggest intact reward/loss anticipation but altered reward consumption in migraine, indicating a decreased reactivity to monetary rewards. The findings also raise the possibility that neural responses to loss anticipation and reward/loss consumption could be altered by the proximity of the last migraine attack not just during pre-ictal periods, but interictally as well

Assessment of sensation seeking personality type using behavioral and functional neuroimaging measures

2020 Fall.Includes bibliographical references.Sensation seeking personality type, in which an individual has the propensity to engage in risky behaviors while searching for an optimal level of stimulation, is associated with a variety of negative health outcomes, such as higher rates of substance misuse, gambling, and self-harm. It is important to develop methods to identify those at higher risk of engaging in such health risk behaviors. Historically, sensation seeking has been primarily measured using self-report surveys. Providing additional measures of sensation seeking, such as through behavioral assessment or biomarkers, would aid our measurement of the sensation seeking personality type. The present work sought to create a new behavioral measure of sensation seeking personality type, the Sensation Seeking Dot Probe Task (SSDP), that measures an individual's attentional bias towards sensation seeking imagery. Further, the SSDP task was combined with functional Near Infrared Spectroscopy, which utilizes the spectral differences of hemoglobin in the brain to measure neural activity, to identify neural correlates of attention to sensation seeking imagery and relate them to the Sensation Seeking Personality Type scale. I hypothesized that the SSDP would be as effective in identifying sensation seeking as the self-report scale, and that attention to sensation seeking images would correlate with changes in neural activity in the prefrontal cortex and orbitofrontal cortex (regions associated with executive control and decision making) that would be greater in high sensation seeking individuals. While the SSDP did not find significant differences in accuracy or reaction time, the typical measures used in attentional bias dot-probe tasks, there was a significant difference in selection of sensation seeking imagery when paired with neutral control imagery. There were also significantly different changes in activity during sensation seeking congruent tasks in areas of the lateral prefrontal cortex for high sensation seeking individuals. These results suggest functional and behavioral differences measurable in high sensation seekers, and future tasks can use these findings to lead to a greater understanding of the personality type

Novelty, Prediction Error and Memory Encoding: Limitations of the Pimms Framework

The Predictive Interactive Multiple Memory Systems (PIMMS) framework has been used to explain how novelty, or more precisely “prediction error”, boosts memory encoding. In this thesis, I explored several other phenomena in the animal and human literature that PIMMS cannot yet explain but should. PIMMS predicts that unexpected information will be better encoded than expected information. However recent work has suggested that expected information can also be better remembered than less expected information. By using a range of expectancies for the location of objects with an immersive virtual reality (iVR) kitchen, I showed that memory is a “U-shaped” function of expectancy, with best memory for highly expected or highly unexpected locations relative to intermediate levels of expectancy. Using OSF-registered Bayesian inference, this U-shape was consistent across four experiments. While the advantage for highly unexpected locations is consistent with PIMMS, the advantage for highly expected locations is not. Importantly, the advantage for expected locations was not simply due to a guessing bias when the location was forgotten, suggesting that the advantage arises during encoding rather than just at retrieval. This U-shape is consistent with another framework - the SLIMM framework - which proposes that different brain regions support the two ends of the U-shape, such that the advantage for unexpected information should be associated with recollection of contextual information via a medial temporal lobe system (like in PIMMS), while the advantage for expected information should be associated with a feeling of familiarity based on rapid cortical consolidation enabled by a medial prefrontal cortex system. However, when I asked participants to indicate recollection or familiarity at retrieval, both ends of the U-shape continuum were associated with higher recollection, while there was no detectable effect of expectancy on familiarity. I consider why this SLIMM prediction may therefore be incorrect. Another finding in the literature concerns the effect of novelty on unrelated information shortly preceding or succeeding the novel experience. PIMMS says nothing about this penumbra effect, which has been related to plasticity-related proteins triggered by the novel experience (so-called “behavioural tagging”). Since participants report that their first iVR experience is highly novel, I submitted a Registered Report to test whether iVR affected memory for unrelated words that were encountered prior to entering the iVR room. In short, the finding was that there is no evidence that novelty improves memory performance for information learned before experiencing something novel. Possible reasons for the failure of finding an effect were discussed. A final limitation of PIMMS I considered was the effect of “boundaries” in continuous stimuli, which are known to affect memory for the temporal order of information. While boundaries might be generated by prediction errors, PIMMS is silent on how they affect temporal order memory. Using a movie featuring a series of rooms, I tested whether memory for the temporal order of objects encountered in those rooms is affected by doorways between rooms and/or by surprising/perceptual changes within a room. Unfortunately, I was unable to replicate a previous report where temporal order memory was worse for pairs of objects in different rooms (i.e., either side of a doorway) than objects in the same room, let alone either sides of a surprising/perceptual change within a room. Taken together, my findings and the literature demonstrate the multiple potential factors that determine how novelty affects memory encoding (and consolidation), which require a more comprehensive theoretical framework than currently available

성공기억에서의 해마의 특징적 뇌 기전

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 뇌인지과학과, 2020. 8. 정천기.One of the most intriguing of the human brain's complex functions is the ability to store information provided by experience and to retrieve much of it at will. This capability of memory processing is critical to humans survival – that is, humans guide their actions based on a given stimulus (e.g., item) in an environment, and can do so even when the stimulus is no longer present owing to the memory of the stimulus. A fundamental question of memory is why some experiences are remembered whereas others are forgotten. Since Scoville and Milners characterization of patient H.M., who demonstrated severe recognition memory deficits following damage to the medial temporal lobe (MTL), the hippocampus has been extensively studied as one of the key neural substrates for memory. In line with this, several experiments have been conducted on exploring the roles of the hippocampus in various ways. One is confirming the causality of the hippocampus in the memory process using direct electrical stimulation to the hippocampal region. The other is investigating the neural correlates of hippocampus using intracranial electroencephalography (iEEG) field potential and single neurons action potential known as spike recorded directly from the hippocampus. The present thesis is focused on providing direct electrophysiological evidence of human hippocampus in episodic memory that may help fill the gap that remained in the field for several years. Here, I will show how direct hippocampal stimulation affect human behavior and present characterized neural correlates of successful memory in the hippocampus. In the first study, building on the previous findings on the hippocampus, I sought to address whether the hippocampus would show functional causality with memory tasks and elicit different neural characteristics depending on memory tasks applied. I found hippocampal stimulation modulated memory performance in a task-dependent manner, improving associative memory performance, while impairing item memory performance. These results of the task-specific memory modulation suggest that the associative task elicited stronger theta oscillations than the single-item task. In the second study, I tested whether successful memory formation relies on the hippocampal neuronal activity that engaged preceding an event. I found that hippocampal pre-stimulus spiking activity (elicited by a cue presented just before a word) predicted subsequent memory. Stimulus activity during encoding (during-stimulus) also showed a trend of predicting subsequent memory but was simply a continuation of pre-stimulus activity. These findings indicate that successful memory formation in human is predicted by a pre-stimulus activity and suggests that the preparatory mobilization of neural processes before encoding benefits episodic memory performance. Throughout the study, the current finding suggests the possibility that the intervals of poor memory encoding can be identified even before the stimulus presented and may be rescued with targeted stimulation to the hippocampus even before the stimulus presented.인간의 복잡한 뇌 기능 중 흥미로운 하나는 경험에 의거하여 정보를 저장하고 의지에 따라 저장된 정보를 재인하는 기억 능력 이다. 인간은 주어진 자극에 기반하여 행동을 정하며 심지어 자극이 없는 상황에서도 자극에 대한 기억을 바탕으로 행동을 결정하기 때문에 기억 능력은 생존에 있어 매우 결정적이며, 이러한 기억과 관련된 가장 기본적인 질문은 기억의 저장 메커니즘, 즉, 어떤 기억은 저장이 되고 어떤 기억은 잊혀지는 가일 것이다. 스코빌과 밀너가 처음 보고한 기억상실증 환자 H.M.은 측두영역의 손상을 입은 후 심각한 인지 기억 능력의 장애를 보였고, 이후 사람 뇌의 해마 영역은 기억을 관장하는 뇌의 중요한 영역 중 하나로 널리 연구되었다. 해마가 기억에 미치는 영향과 역할에 대해서는 다양한 방법으로 실험이 진행되어 왔다. 그 중의 하나는 뇌에 직접적인 전기자극을 가해 기억 과정 중 해마의 역할을 확인하는 방법인데, 이는 뇌전증 환자의 모델을 통해 사람의 뇌에 접근이 가능해지면서 이루어져 왔다. 두 번째 방법은 전기생리학적 방법을 통하는 것인데 세포 외 활동 전위인 스파이크를 통해 성공기억에서의 뉴런의 활동성을 밝히는 것이다. 이 논문은 이 분야에서 오랫동안 논란이 되었고 부족했던 성공 기억에 관련된 해마의 역할과 기전을 물리적 자극 및 신경세포의 신호를 측정해서 전기생리학적 특성을 제시하는데 초점을 맞추고 있다. 논문에서 본 저자는 사람의 성공기억형성과 재인에 대해 뇌 자극과 단위세포활동을 보고할 것이다. 해마와 기억의 인과관계 및 기억 과정 중의 해마의 뇌 기전과 관련된 기존의 실험적, 행동적 발견들에 근거하여 본 저자는 (ㄱ) 해마에 직접적인 전기 자극을 주고 기억 수행능력의 차이 및 기억 과제에 따른 해마의 신경 기전을 밝히고, (ㄴ) 성공 기억이 형성되는 과정에서 나타나는 신경세포의 발화 패턴의 특성을 살펴보았다. 본 연구를 통해 저자는 향후 기억의 형성 과정에서, 자극이 제시되는 구간뿐 만 아니라 자극이 주어지기 전 단계에서도 해마를 타깃 하여 전기 자극을 줌으로써 기억 실패로 이어질 수 있는 자극을 성공 기억으로 저장할 수 있도록 유도할 수 있을 것이라 기대한다.SECTION 1. INTRODUCTION 1 CHAPTER 1: Human Memory System 1 1.1. The hippocampus and memory 2 1.2. The structure of the hippocampus 3 CHAPTER 2: Human Memory Research: how to see a memory 4 2.1 Clinical rationale for invasive recordings with intracranial electrodes 4 2.2. Human intracranial EEG 6 2.3. Single unit activity recording and spike sorting in human 7 2.4. Direct brain stimulation study 9 CHAPTER 3: Human Memory Research: hippocampal activity for understanding successful memory formation 11 3.1. Functional role of human intracranial oscillatory activity in successful memory mechanism 11 3.1.1. Theta Oscillations 11 3.1.2. Gamma oscillations 13 3.2. Brain stimulation for memory enhancement 14 3.3. Single unit activity study in memory 15 CHAPTER 4: Purpose of the Present Study 17 SECTION 2. EXPERIMENTAL STUDY 19 CHAPTER 5: The importance of the hippocampal oscillatory activity for successful memory: direct brain stimulation study 19 5.1. Abstract 20 5.2. Introduction 22 5.3. Materials and Methods 25 5.3.1. Patients 25 5.3.2. Electrode localization 25 5.3.3. Memory task 29 5.3.4. Brain stimulation 30 5.3.5. Neuropsychological memory test 31 5.3.6. Analysis of memory performance and electrophysiological data 32 5.4. Results 37 5.4.1. Hippocampal stimulation improves associative memory but impairs item memory 37 5.4.2. Stimulation-induced memory enhancement is reflected in increased theta power during retrieval 38 5.4.3. Associative memory elicits higher theta power than item memory during encoding 42 5.4.4. Successful memory encoding elicits higher theta power in both memory task 44 5.4.5. Stimulation-mediated memory effect is greater in subject with poorer baseline cognitive function 46 5.5. Discussion 48 5.5.1. Summary 48 5.5.2. Task-dependent effects of hippocampal stimulation on memory 49 5.5.3. Theta activity as a neural signature for memory enhancement 51 5.5.4. Clinical implications 52 5.5.5. Limitations 54 5.5.6. Conclusion 55 CHAPTER 6: Hippocampal pre-stimulus activity predicts later memory success 57 6.1. Abstract 58 6.2. Introduction 59 6.3. Materials and Methods 62 6.3.1. Patients 62 6.3.2. Electrodes 63 6.3.3. Task and Stimuli 64 6.3.4. Electrophysiological recordings and Spike sorting 65 6.3.5. Analysis of iEEG field potentials 66 6.4. Results 68 6.4.1. Behavioral results 68 6.4.2. Spiking properties of hippocampal neurons 68 6.4.3. Hippocampal pre-stimulus activity correlates with successful memory 70 6.4.4. Hippocampal pre-stimulus spiking activity correlates with high gamma field potentials 74 6.5. Discussion 78 6.5.1. Summary 78 6.5.2. Comparison with previous findings 78 6.5.3. Possible mechanism underlying pre-stimulus activity 79 6.5.4. Conclusion 82 SECTION 3. GENERAL CONCLUSION 83 CHAPTER 7: General Conclusion and Perspective 83 Bibliography 84 Abstract in Korean (국문초록) 93Docto

Motivated cognition: Neural and computational mechanisms of curiosity, attention and intrinsic motivation

International audienceBased on a synthesis of findings from psychology, neuroscience, and machine learning, we propose a unified theory of curiosity as a form of motivated cognition. Curiosity, we propose, is comprised of a family of mechanisms that range in complexity from simple heuristics based on novelty, salience, or surprise, to drives based on reward and uncertainty reduction and finally, to self-directed metacognitive processes. These mechanisms, we propose, have evolved to allow agents to discover useful regularities in the world ! steering them toward niches of maximal learning progress and away from both random and highly familiar tasks. We emphasize that curiosity arises organically in conjunction with cogni- tion and motivation, being generated by cognitive processes and in turn, motivating them. We hope that this view will spur the systematic study of curiosity as an integral aspect of cognition and decision making during development and adulthood