Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization

Artificial autonomous agents and robots interacting in complex environments are required to continually acquire and fine-tune knowledge over sustained periods of time. The ability to learn from continuous streams of information is referred to as lifelong learning and represents a long-standing challenge for neural network models due to catastrophic forgetting. Computational models of lifelong learning typically alleviate catastrophic forgetting in experimental scenarios with given datasets of static images and limited complexity, thereby differing significantly from the conditions artificial agents are exposed to. In more natural settings, sequential information may become progressively available over time and access to previous experience may be restricted. In this paper, we propose a dual-memory self-organizing architecture for lifelong learning scenarios. The architecture comprises two growing recurrent networks with the complementary tasks of learning object instances (episodic memory) and categories (semantic memory). Both growing networks can expand in response to novel sensory experience: the episodic memory learns fine-grained spatiotemporal representations of object instances in an unsupervised fashion while the semantic memory uses task-relevant signals to regulate structural plasticity levels and develop more compact representations from episodic experience. For the consolidation of knowledge in the absence of external sensory input, the episodic memory periodically replays trajectories of neural reactivations. We evaluate the proposed model on the CORe50 benchmark dataset for continuous object recognition, showing that we significantly outperform current methods of lifelong learning in three different incremental learning scenario

Social Cognition for Human-Robot Symbiosis—Challenges and Building Blocks

The next generation of robot companions or robot working partners will need to satisfy social requirements somehow similar to the famous laws of robotics envisaged by Isaac Asimov time ago (Asimov, 1942). The necessary technology has almost reached the required level, including sensors and actuators, but the cognitive organization is still in its infancy and is only partially supported by the current understanding of brain cognitive processes. The brain of symbiotic robots will certainly not be a “positronic” replica of the human brain: probably, the greatest part of it will be a set of interacting computational processes running in the cloud. In this article, we review the challenges that must be met in the design of a set of interacting computational processes as building blocks of a cognitive architecture that may give symbiotic capabilities to collaborative robots of the next decades: (1) an animated body-schema; (2) an imitation machinery; (3) a motor intentions machinery; (4) a set of physical interaction mechanisms; and (5) a shared memory system for incremental symbiotic development. We would like to stress that our approach is totally un-hierarchical: the five building blocks of the shared cognitive architecture are fully bi-directionally connected. For example, imitation and intentional processes require the “services” of the animated body schema which, on the other hand, can run its simulations if appropriately prompted by imitation and/or intention, with or without physical interaction. Successful experiences can leave a trace in the shared memory system and chunks of memory fragment may compete to participate to novel cooperative actions. And so on and so forth. At the heart of the system is lifelong training and learning but, different from the conventional learning paradigms in neural networks, where learning is somehow passively imposed by an external agent, in symbiotic robots there is an element of free choice of what is worth learning, driven by the interaction between the robot and the human partner. The proposed set of building blocks is certainly a rough approximation of what is needed by symbiotic robots but we believe it is a useful starting point for building a computational framework

Continual Lifelong Learning with Neural Networks: A Review

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor skills as well as to long-term memory consolidation and retrieval. Consequently, lifelong learning capabilities are crucial for autonomous agents interacting in the real world and processing continuous streams of information. However, lifelong learning remains a long-standing challenge for machine learning and neural network models since the continual acquisition of incrementally available information from non-stationary data distributions generally leads to catastrophic forgetting or interference. This limitation represents a major drawback for state-of-the-art deep neural network models that typically learn representations from stationary batches of training data, thus without accounting for situations in which information becomes incrementally available over time. In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. We discuss well-established and emerging research motivated by lifelong learning factors in biological systems such as structural plasticity, memory replay, curriculum and transfer learning, intrinsic motivation, and multisensory integration

A Biosymtic (Biosymbiotic Robotic) Approach to Human Development and Evolution. The Echo of the Universe.

In the present work we demonstrate that the current Child-Computer Interaction paradigm is not potentiating human development to its fullest – it is associated with several physical and mental health problems and appears not to be maximizing children’s cognitive performance and cognitive development. In order to potentiate children’s physical and mental health (including cognitive performance and cognitive development) we have developed a new approach to human development and evolution. This approach proposes a particular synergy between the developing human body, computing machines and natural environments. It emphasizes that children should be encouraged to interact with challenging physical environments offering multiple possibilities for sensory stimulation and increasing physical and mental stress to the organism. We created and tested a new set of computing devices in order to operationalize our approach – Biosymtic (Biosymbiotic Robotic) devices: “Albert” and “Cratus”. In two initial studies we were able to observe that the main goal of our approach is being achieved. We observed that, interaction with the Biosymtic device “Albert”, in a natural environment, managed to trigger a different neurophysiological response (increases in sustained attention levels) and tended to optimize episodic memory performance in children, compared to interaction with a sedentary screen-based computing device, in an artificially controlled environment (indoors) - thus a promising solution to promote cognitive performance/development; and that interaction with the Biosymtic device “Cratus”, in a natural environment, instilled vigorous physical activity levels in children - thus a promising solution to promote physical and mental health

Interaction Histories and Short-Term Memory: Enactive Development of Turn-Taking Behaviours in a Childlike Humanoid Robot

In this article, an enactive architecture is described that allows a humanoid robot to learn to compose simple actions into turn-taking behaviours while playing interaction games with a human partner. The robot’s action choices are reinforced by social feedback from the human in the form of visual attention and measures of behavioural synchronisation. We demonstrate that the system can acquire and switch between behaviours learned through interaction based on social feedback from the human partner. The role of reinforcement based on a short-term memory of the interaction was experimentally investigated. Results indicate that feedback based only on the immediate experience was insufficient to learn longer, more complex turn-taking behaviours. Therefore, some history of the interaction must be considered in the acquisition of turn-taking, which can be efficiently handled through the use of short-term memory.Peer reviewedFinal Published versio

Reconnaissance et stabilité d'une mémoire épisodique influencée par les émotions artificielles pour un robot autonome

Les robots de service devront répondre aux besoins d'humains au quotidien. Nos milieux de vie diffèrent par leur configuration, les conditions environnementales, les objets qui s'y trouvent, les personnes présentes et les événements pouvant y survenir. Un grand défi de la robotique autonome est de permettre aux robots de s'adapter à n'importe quelle situation tout en étant efficace et sécuritaire dans l'exécution de tâches. À cette fin, une mémoire épisodique a le rôle d'emmagasiner et de classer les expériences d'un agent intelligent en lien avec les éléments du contexte spatio-temporel d'apprentissage. Ainsi, une mémoire épisodique s'avère un élément essentiel pour permettre au robot de mémoriser ses expériences dans le but de les réutiliser lors de situations similaires. Toutefois, pour qu'une mémoire épisodique puisse être utilisée par un robot autonome, elle doit pouvoir exploiter l'information provenant de capteurs asynchrones et bruités. De plus, elle doit pouvoir être influencée différemment selon l'importance des expériences vécues. Le but de ce projet de recherche est de concevoir et d'intégrer à un robot mobile une mémoire épisodique construite à partir d'un apprentissage non supervisé et qui favorise la mémorisation des expériences les plus pertinentes afin d'améliorer l'efficacité du robot dans l'exécution de sa tâche. À la base, l'approche repose sur des réseaux de neurones utilisant la Théorie de résonance adaptative (ART, pour Adaptive Resonance Theory). Deux réseaux ART sont placés en cascade afin de catégoriser, respectivement, les contextes spatiaux, appelés événements, et les séquences d'événements, appelées épisodes. Le modèle résultant, EM-ART (Episodic Memory-ART), utilise un module d'émotions artificielles afin d'influencer la dynamique d'apprentissage et d'utilisation des réseaux ART en favorisant la mémorisation et le rappel des expériences associées à de fortes intensités émotionnelles. Le rappel d'épisodes permet de prédire et d'anticiper les événements futurs, contribuant à améliorer l'adaptabilité du robot pour effectuer sa tâche. EM-ART est validé sur le robot IRL-1/TR dans un scénario de livraison d'objets. Les expérimentations réalisées en milieu réel permettent d'isoler les caractéristiques du modèle telles que la prédiction d'événements, la création d'épisodes et l'influence des émotions. Des simulations construites à partir de données réelles permettent aussi d'observer l'évolution de la structure du modèle sur une plus grande période de temps et dans des séquences différentes. Les résultats démontrent que le modèle EM-ART permet une récupération d'épisodes plus hâtive lorsque ceux-ci sont associés à une intensité émotionnelle élevée, permettant à IRL-1/TR d'utiliser la destination de sa dernière livraison pour accomplir la livraison en cours. Selon la séquence des expériences soumis au modèle, un plus grand nombre d'épisodes est créé si les premières expériences ne sont pas associées à des émotions élevées, puisqu'ils sont négligées en mémoire au détriment de la création de nouveaux épisodes plus distinctifs. Il en résulte une capacité faisant évoluer l'intelligence du robot à celle d'une entité capable d'apprendre de ses expériences évaluées selon sa propre perspective

A neural network model of normal and abnormal learning and memory consolidation

The amygdala and hippocampus interact with thalamocortical systems to regulate cognitive-emotional learning, and lesions of amygdala, hippocampus, thalamus, and cortex have different effects depending on the phase of learning when they occur. In examining eyeblink conditioning data, several questions arise: Why is the hippocampus needed for trace conditioning where there is a temporal gap between the conditioned stimulus offset and the onset of the unconditioned stimulus, but not needed for delay conditioning where stimuli temporally overlap and co-terminate? Why do amygdala lesions made before or immediately after training decelerate conditioning while those made later have no impact on conditioned behavior? Why do thalamic lesions degrade trace conditioning more than delay conditioning? Why do hippocampal lesions degrade recent learning but not temporally remote learning? Why do cortical lesions degrade temporally remote learning, and cause amnesia, but not recent or post-lesion learning? How is temporally graded amnesia caused by ablation of medial prefrontal cortex? How are mechanisms of motivated attention and the emergent state of consciousness linked during conditioning? How do neurotrophins, notably Brain Derived Neurotrophic Factor (BDNF), influence memory formation and consolidation? A neural model, called neurotrophic START, or nSTART, proposes answers to these questions. The nSTART model synthesizes and extends key principles, mechanisms, and properties of three previously published brain models of normal behavior. These three models describe aspects of how the brain can learn to categorize objects and events in the world; how the brain can learn the emotional meanings of such events, notably rewarding and punishing events, through cognitive-emotional interactions; and how the brain can learn to adaptively time attention paid to motivationally important events, and when to respond to these events, in a context-appropriate manner. The model clarifies how hippocampal adaptive timing mechanisms and BDNF may bridge the gap between stimuli during trace conditioning and thereby allow thalamocortical and corticocortical learning to take place and be consolidated. The simulated data arise as emergent properties of several brain regions interacting together. The model overcomes problems of alternative memory models, notably models wherein memories that are initially stored in hippocampus move to the neocortex during consolidation