Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization

Artificial autonomous agents and robots interacting in complex environments are required to continually acquire and fine-tune knowledge over sustained periods of time. The ability to learn from continuous streams of information is referred to as lifelong learning and represents a long-standing challenge for neural network models due to catastrophic forgetting. Computational models of lifelong learning typically alleviate catastrophic forgetting in experimental scenarios with given datasets of static images and limited complexity, thereby differing significantly from the conditions artificial agents are exposed to. In more natural settings, sequential information may become progressively available over time and access to previous experience may be restricted. In this paper, we propose a dual-memory self-organizing architecture for lifelong learning scenarios. The architecture comprises two growing recurrent networks with the complementary tasks of learning object instances (episodic memory) and categories (semantic memory). Both growing networks can expand in response to novel sensory experience: the episodic memory learns fine-grained spatiotemporal representations of object instances in an unsupervised fashion while the semantic memory uses task-relevant signals to regulate structural plasticity levels and develop more compact representations from episodic experience. For the consolidation of knowledge in the absence of external sensory input, the episodic memory periodically replays trajectories of neural reactivations. We evaluate the proposed model on the CORe50 benchmark dataset for continuous object recognition, showing that we significantly outperform current methods of lifelong learning in three different incremental learning scenario

Memories for Life: A Review of the Science and Technology

This paper discusses scientific, social and technological aspects of memory. Recent developments in our understanding of memory processes and mechanisms, and their digital implementation, have placed the encoding, storage, management and retrieval of information at the forefront of several fields of research. At the same time, the divisions between the biological, physical and the digital worlds seem to be dissolving. Hence opportunities for interdisciplinary research into memory are being created, between the life sciences, social sciences and physical sciences. Such research may benefit from immediate application into information management technology as a testbed. The paper describes one initiative, Memories for Life, as a potential common problem space for the various interested disciplines

Long-term stability of the hippocampal neural code as a substrate for episodic memory

The hippocampus supports the initial formation and recall of episodic memories, as well as the consolidation of short-term into long-term memories. The ability of hippocampal neurons to rapidly change their connection strengths during learning and maintain these changes over long time-scales may provide a mechanism supporting memory. However, little evidence currently exists concerning the long-term stability of information contained in hippocampal neuronal activity, likely due to limitations in recording extracellular activity in vivo from the same neurons across days. In this thesis I employ calcium imaging in freely moving mice to longitudinally track the activity of large ensembles of hippocampal neurons. Using this technology, I explore the proposal that long-term stability of hippocampal information provides a substrate for episodic memory in three different ways. First, I tested the hypothesis that hippocampal activity should remain stable across days in the absence of learning. I found that place cells – hippocampal neurons containing information about a mouse’s position – maintain a coherent map relative to each other across long time-scales but exhibit instability in how they anchor to the external world. Furthermore, I found that coherent maps were frequently used to represent a different environment and incorporated learning via changes in a subset of neurons. Next, I examined how learning a spatial alternation task impacts neuron stability. I found that splitter neurons whose activity patterns reflected an animal’s future or past trajectory emerged relatively slowly when compared to place cells. However, splitter neurons remained more consistently active and relayed more consistent spatial information across days than did place cells, suggesting that the utility of information provided by a neuron influences its long term stability. Last, I investigated how protein synthesis, known to be necessary for long-term maintenance of changes in hippocampal neuron connection strengths and for proper memory consolidation, influences their activity patterns across days. I found that along with blocking memory consolidation, inhibiting protein synthesis induced a profound, long-lasting decrease in neuronal activity up to two days later. These results combined demonstrate the importance of rapid, lasting changes in the hippocampal neuronal code to supporting long-term memory

Scene construction impairments in Alzheimer's disease – A unique role for the posterior cingulate cortex

Episodic memory dysfunction represents one of the most prominent and characteristic clinical features of patients with Alzheimer's disease (AD), attributable to the degeneration of medial temporal and posterior parietal regions of the brain. Recent studies have demonstrated marked impairments in the ability to envisage personally relevant events in the future in AD. It remains unclear, however, whether AD patients can imagine fictitious scenes free from temporal constraints, a process that is proposed to rely fundamentally upon the integrity of the hippocampus. The objective of the present study was to investigate the capacity for atemporal scene construction, and its associated neural substrates, in AD. Fourteen AD patients were tested on the scene construction task and their performance was contrasted with 14 age- and education-matched healthy older Control participants. Scene construction performance was strikingly compromised in the AD group, with significant impairments evident for provision of contextual details, spatial coherence, and the overall richness of the imagined experience. Voxel-based morphometry analyses based on structural MRI revealed significant associations between scene construction capacity and atrophy in posterior parietal and lateral temporal brain structures in AD. In contrast, scene construction performance in Controls was related to integrity of frontal, parietal, and medial temporal structures, including the parahippocampal gyrus and posterior hippocampus. The posterior cingulate cortex (PCC) emerged as the common region implicated for scene construction performance across participant groups. Our study highlights the importance of regions specialised for spatial and contextual processing for the construction of atemporal scenes. Damage to these regions in AD compromises the ability to construct novel scenes, leading to the recapitulation of content from previously experienced events

Schema and value: Characterizing the role of the rostral and ventral medial prefrontal cortex in episodic future thinking

As humans we are not stuck in an everlasting present. Instead, we can project ourselves into both our personal past and future. Remembering the past and simulating the future are strongly interrelated processes. They are both supported by largely the same brain regions including the rostral and ventral medial prefrontal cortex (mPFC) but also the hippocampus, the posterior cingulate cortex (PCC), as well as other regions in the parietal and temporal cortices. Interestingly, this core network for episodic simulation and episodic memory partially overlaps with a brain network for evaluation and value-based decision making. This is particularly the case for the mPFC. This part of the brain has been associated both with a large number of different cognitive functions ranging from the representation of memory schemas and self-referential processing to the representation of value and affect. As a consequence, a unifying account of mPFC functioning has remained elusive. The present thesis investigates the unique contribution of the mPFC to episodic simulation by highlighting its role in the representation of memory schemas and value. In a first functional MRI and pre-registered behavioral replication study, we demonstrate that the mPFC encodes representations of known people as well as of known locations from participants’ everyday life. We demonstrate that merely imagined encounters with liked vs. disliked people at these locations can change our attitude toward the locations. The magnitude of this simulation-induced attitude change was predicted by activation in the mPFC during the simulations. Specifically, locations simulated with liked people exhibited significantly larger increases in liking than those simulated with disliked people. In a second behavioral study, we examined the mechanisms of simulation-based learning more closely. To this end, participants also simulated encounters with neutral people at neutral locations. Using repeated behavioral assessments of participants’ memory representations, we reveal that simulations cause an integration of memory representations for jointly simulated people and locations. Moreover, compared to the neutral baseline condition we demonstrate a transfer of positive valence from liked and of negative valence from disliked people to their paired locations. We also provide evidence that simulations induce an affective experience that aligns with the valence of the person and that this experience can account for the observed attitude change toward the location. In a final fMRI study, we examine the structure of memory representations encoded in the mPFC. Specifically, we provide evidence for the hypothesis that the mPFC encodes schematic representations of our social and physical environment. We demonstrate that representations of individual exemplars of these environments (i.e., individual people and locations) are closely intertwined with a representation of their value. In sum, our findings show that we can learn from imagined experience much as we learn from actual past experience and that the mPFC plays a key role in simulation-based learning. The mPFC encodes information about our environment in value-weighted schematic representations. These representations can account for the overlap of mnemonic and evaluative functions in the mPFC and might play a key role in simulation-based learning. Our results are in line with a view that our memories of the past serve us in ways that are oriented toward the future. Our ability to simulate potential scenarios allows us to anticipate the future consequences of our choices and thereby fosters farsighted decision making. Thus, our findings help to better characterize the functional role of the mPFC in episodic future simulation and valuation