Computational Theories of Curiosity-Driven Learning

What are the functions of curiosity? What are the mechanisms of curiosity-driven learning? We approach these questions about the living using concepts and tools from machine learning and developmental robotics. We argue that curiosity-driven learning enables organisms to make discoveries to solve complex problems with rare or deceptive rewards. By fostering exploration and discovery of a diversity of behavioural skills, and ignoring these rewards, curiosity can be efficient to bootstrap learning when there is no information, or deceptive information, about local improvement towards these problems. We also explain the key role of curiosity for efficient learning of world models. We review both normative and heuristic computational frameworks used to understand the mechanisms of curiosity in humans, conceptualizing the child as a sense-making organism. These frameworks enable us to discuss the bi-directional causal links between curiosity and learning, and to provide new hypotheses about the fundamental role of curiosity in self-organizing developmental structures through curriculum learning. We present various developmental robotics experiments that study these mechanisms in action, both supporting these hypotheses to understand better curiosity in humans and opening new research avenues in machine learning and artificial intelligence. Finally, we discuss challenges for the design of experimental paradigms for studying curiosity in psychology and cognitive neuroscience. Keywords: Curiosity, intrinsic motivation, lifelong learning, predictions, world model, rewards, free-energy principle, learning progress, machine learning, AI, developmental robotics, development, curriculum learning, self-organization.Comment: To appear in "The New Science of Curiosity", ed. G. Gordon, Nova Science Publisher

Attentional mechanisms for socially interactive robots – a survey

This review intends to provide an overview of the state of the art in the modeling and implementation of automatic attentional mechanisms for socially interactive robots. Humans assess and exhibit intentionality by resorting to multisensory processes that are deeply rooted within low-level automatic attention-related mechanisms of the brain. For robots to engage with humans properly, they should also be equipped with similar capabilities. Joint attention, the precursor of many fundamental types of social interactions, has been an important focus of research in the past decade and a half, therefore providing the perfect backdrop for assessing the current status of state-of-the-art automatic attentional-based solutions. Consequently, we propose to review the influence of these mechanisms in the context of social interaction in cutting-edge research work on joint attention. This will be achieved by summarizing the contributions already made in these matters in robotic cognitive systems research, by identifying the main scientific issues to be addressed by these contributions and analyzing how successful they have been in this respect, and by consequently drawing conclusions that may suggest a roadmap for future successful research efforts

TOWARDS THE GROUNDING OF ABSTRACT CATEGORIES IN COGNITIVE ROBOTS

The grounding of language in humanoid robots is a fundamental problem, especially in social scenarios which involve the interaction of robots with human beings. Indeed, natural language represents the most natural interface for humans to interact and exchange information about concrete entities like KNIFE, HAMMER and abstract concepts such as MAKE, USE. This research domain is very important not only for the advances that it can produce in the design of human-robot communication systems, but also for the implication that it can have on cognitive science. Abstract words are used in daily conversations among people to describe events and situations that occur in the environment. Many scholars have suggested that the distinction between concrete and abstract words is a continuum according to which all entities can be varied in their level of abstractness. The work presented herein aimed to ground abstract concepts, similarly to concrete ones, in perception and action systems. This permitted to investigate how different behavioural and cognitive capabilities can be integrated in a humanoid robot in order to bootstrap the development of higher-order skills such as the acquisition of abstract words. To this end, three neuro-robotics models were implemented. The first neuro-robotics experiment consisted in training a humanoid robot to perform a set of motor primitives (e.g. PUSH, PULL, etc.) that hierarchically combined led to the acquisition of higher-order words (e.g. ACCEPT, REJECT). The implementation of this model, based on a feed-forward artificial neural networks, permitted the assessment of the training methodology adopted for the grounding of language in humanoid robots. In the second experiment, the architecture used for carrying out the first study was reimplemented employing recurrent artificial neural networks that enabled the temporal specification of the action primitives to be executed by the robot. This permitted to increase the combinations of actions that can be taught to the robot for the generation of more complex movements. For the third experiment, a model based on recurrent neural networks that integrated multi-modal inputs (i.e. language, vision and proprioception) was implemented for the grounding of abstract action words (e.g. USE, MAKE). Abstract representations of actions ("one-hot" encoding) used in the other two experiments, were replaced with the joints values recorded from the iCub robot sensors. Experimental results showed that motor primitives have different activation patterns according to the action's sequence in which they are embedded. Furthermore, the performed simulations suggested that the acquisition of concepts related to abstract action words requires the reactivation of similar internal representations activated during the acquisition of the basic concepts, directly grounded in perceptual and sensorimotor knowledge, contained in the hierarchical structure of the words used to ground the abstract action words.This study was financed by the EU project RobotDoC (235065) from the Seventh Framework Programme (FP7), Marie Curie Actions Initial Training Network

Final report key contents: main results accomplished by the EU-Funded project IM-CLeVeR - Intrinsically Motivated Cumulative Learning Versatile Robots

This document has the goal of presenting the main scientific and technological achievements of the project IM-CLeVeR. The document is organised as follows: 1. Project executive summary: a brief overview of the project vision, objectives and keywords. 2. Beneficiaries of the project and contacts: list of Teams (partners) of the project, Team Leaders and contacts. 3. Project context and objectives: the vision of the project and its overall objectives 4. Overview of work performed and main results achieved: a one page overview of the main results of the project 5. Overview of main results per partner: a bullet-point list of main results per partners 6. Main achievements in detail, per partner: a throughout explanation of the main results per partner (but including collaboration work), with also reference to the main publications supporting them

PCT and beyond: toward a computational framework for ‘intelligent’ communicative systems

Recent years have witnessed increasing interest in ‘intelligent’ autonomous machines such as robots. However, there is a long way to go before autonomous systems reach the level of capabilities required for even the simplest of tasks involving human-robot interaction - especially if it involves communicative behavior such as speech and language. The field of Artificial Intelligence (AI) has made great strides in these areas, and has graduated from high-level rule-based paradigms to embodied architectures whose operations are grounded in real physical environments. What is still missing, however, is an overarching theory of intelligent communicative behavior that informs system-level design decisions. This chapter introduces a framework that extends the principles of Perceptual Control Theory (PCT) toward a remarkably symmetric architecture for a needs-driven communicative agent. It is concluded that, if behavior is the control of perception (the central tenet of PCT), then perception (for communicative agents) is the simulation of behavior

World model learning and inference

Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world

The use of emotions in the implementation of various types of learning in a cognitive agent

Les tuteurs professionnels humains sont capables de prendre en considération des événements du passé et du présent et ont une capacité d'adaptation en fonction d'événements sociaux. Afin d'être considéré comme une technologie valable pour l'amélioration de l'apprentissage humain, un agent cognitif artificiel devrait pouvoir faire de même. Puisque les environnements dynamiques sont en constante évolution, un agent cognitif doit pareillement évoluer et s'adapter aux modifications structurales et aux phénomènes nouveaux. Par conséquent, l'agent cognitif idéal devrait posséder des capacités d'apprentissage similaires à celles que l'on retrouve chez l'être humain ; l'apprentissage émotif, l'apprentissage épisodique, l'apprentissage procédural, et l'apprentissage causal. Cette thèse contribue à l'amélioration des architectures d'agents cognitifs. Elle propose 1) une méthode d'intégration des émotions inspirée du fonctionnement du cerveau; et 2) un ensemble de méthodes d'apprentissage (épisodique, causale, etc.) qui tiennent compte de la dimension émotionnelle. Le modèle proposé que nous avons appelé CELTS (Conscious Emotional Learning Tutoring System) est une extension d'un agent cognitif conscient dans le rôle d'un tutoriel intelligent. Il comporte un module de gestion des émotions qui permet d'attribuer des valences émotionnelles positives ou négatives à chaque événement perçu par l'agent. Deux voies de traitement sont prévues : 1) une voie courte qui permet au système de répondre immédiatement à certains événements sans un traitement approfondis, et 2) une voie longue qui intervient lors de tout événement qui exige la volition. Dans cette perspective, la dimension émotionnelle est considérée dans les processus cognitifs de l'agent pour la prise de décision et l'apprentissage. L'apprentissage épisodique dans CELTS est basé sur la théorie du Multiple Trace Memory consolidation qui postule que lorsque l'on perçoit un événement, l'hippocampe fait une première interprétation et un premier apprentissage. Ensuite, l'information acquise est distribuée aux différents cortex. Selon cette théorie, la reconsolidation de la mémoire dépend toujours de l'hippocampe. Pour simuler de tel processus, nous avons utilisé des techniques de fouille de données qui permettent la recherche de motifs séquentiels fréquents dans les données générées durant chaque cycle cognitif. L'apprentissage causal dans CELTS se produit à l'aide de la mémoire épisodique. Il permet de trouver les causes et les effets possibles entre différents événements. Il est mise en œuvre grâce à des algorithmes de recherche de règles d'associations. Les associations établies sont utilisées pour piloter les interventions tutorielles de CELTS et, par le biais des réponses de l'apprenant, pour évaluer les règles causales découvertes. \ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : agents cognitifs, émotions, apprentissage épisodique, apprentissage causal

The active inference approach to ecological perception: general information dynamics for natural and artificial embodied cognition

The emerging neurocomputational vision of humans as embodied, ecologically embedded, social agents—who shape and are shaped by their environment—offers a golden opportunity to revisit and revise ideas about the physical and information-theoretic underpinnings of life, mind, and consciousness itself. In particular, the active inference framework (AIF) makes it possible to bridge connections from computational neuroscience and robotics/AI to ecological psychology and phenomenology, revealing common underpinnings and overcoming key limitations. AIF opposes the mechanistic to the reductive, while staying fully grounded in a naturalistic and information-theoretic foundation, using the principle of free energy minimization. The latter provides a theoretical basis for a unified treatment of particles, organisms, and interactive machines, spanning from the inorganic to organic, non-life to life, and natural to artificial agents. We provide a brief introduction to AIF, then explore its implications for evolutionary theory, ecological psychology, embodied phenomenology, and robotics/AI research. We conclude the paper by considering implications for machine consciousness