Spatial Learning and Localization in Animals: A Computational Model and Its Implications for Mobile Robots

The ability to acquire a representation of spatial environment and the ability to localize within it are essential for successful navigation in a-priori unknown environments. The hippocampal formation is believed to play a key role in spatial learning and navigation in animals. This paper briefly reviews the relevant neurobiological and cognitive data and their relation to computational models of spatial learning and localization used in mobile robots. It also describes a hippocampal model of spatial learning and navigation and analyzes it using Kalman filter based tools for information fusion from multiple uncertain sources. The resulting model allows a robot to learn a place-based, metric representation of space in a-priori unknown environments and to localize itself in a stochastically optimal manner. The paper also describes an algorithmic implementation of the model and results of several experiments that demonstrate its capabilities

Drama, a connectionist model for robot learning: experiments on grounding communication through imitation in autonomous robots

The present dissertation addresses problems related to robot learning from demonstra¬ tion. It presents the building of a connectionist architecture, which provides the robot with the necessary cognitive and behavioural mechanisms for learning a synthetic lan¬ guage taught by an external teacher agent. This thesis considers three main issues: 1) learning of spatio-temporal invariance in a dynamic noisy environment, 2) symbol grounding of a robot's actions and perceptions, 3) development of a common symbolic representation of the world by heterogeneous agents.We build our approach on the assumption that grounding of symbolic communication creates constraints not only on the cognitive capabilities of the agent but also and especially on its behavioural capacities. Behavioural skills, such as imitation, which allow the agent to co-ordinate its actionn to that of the teacher agent, are required aside to general cognitive abilities of associativity, in order to constrain the agent's attention to making relevant perceptions, onto which it grounds the teacher agent's symbolic expression. In addition, the agent should be provided with the cognitive capacity for extracting spatial and temporal invariance in the continuous flow of its perceptions. Based on this requirement, we develop a connectionist architecture for learning time series. The model is a Dynamical Recurrent Associative Memory Architecture, called DRAMA. It is a fully connected recurrent neural network using Hebbian update rules. Learning is dynamic and unsupervised. The performance of the architecture is analysed theoretically, through numerical simulations and through physical and simulated robotic experiments. Training of the network is computationally fast and inexpensive, which allows its implementation for real time computation and on-line learning in a inexpensive hardware system. Robotic experiments are carried out with different learning tasks involving recognition of spatial and temporal invariance, namely landmark recognition and prediction of perception-action sequence in maze travelling.The architecture is applied to experiments on robot learning by imitation. A learner robot is taught by a teacher agent, a human instructor and another robot, a vocabulary to describe its perceptions and actions. The experiments are based on an imitative strategy, whereby the learner robot reproduces the teacher's actions. While imitating the teacher's movements, the learner robot makes similar proprio and exteroceptions to those of the teacher. The learner robot grounds the teacher's words onto the set of common perceptions they share. We carry out experiments in simulated and physical environments, using different robotic set-ups, increasing gradually the complexity of the task. In a first set of experiments, we study transmission of a vocabulary to designate actions and perception of a robot. Further, we carry out simulation studies, in which we investigate transmission and use of the vocabulary among a group of robotic agents. In a third set of experiments, we investigate learning sequences of the robot's perceptions, while wandering in a physically constrained environment. Finally, we present the implementation of DRAMA in Robota, a doll-like robot, which can imitate the arms and head movements of a human instructor. Through this imitative game, Robota is taught to perform and label dance patterns. Further, Robota is taught a basic language, including a lexicon and syntactical rules for the combination of words of the lexicon, to describe its actions and perception of touch onto its body

The development of numerical cognition in children and artificial systems: a review of the current knowledge and proposals for multi-disciplinary research

Numerical cognition is a distinctive component of human intelligence such that the observation of its practice provides a window into high-level brain function. The modelling of numerical abilities in artificial cognitive systems can help to confirm existing child development hypotheses and define new ones by means of computational simulations. Meanwhile, new research will help to discover innovative principles for the design of artificial agents with advanced reasoning capabilities and clarify the underlying algorithms (e.g. deep learning) that can be highly effective but difficult to understand for humans. This article promotes new investigation by providing a common resource for researchers with different backgrounds, including computer science, robotics, neuroscience, psychology, and education, who are interested in pursuing scientific collaboration on mutually stimulating research on this topic. The article emphasises the fundamental role of embodiment in the initial development of numerical cognition in children. This strong relationship with the body motivates the Cognitive Developmental Robotics (CDR) approach for new research that can (among others) help to standardise data collection and provide open databases for benchmarking computational models. Furthermore, we discuss the potential application of robots in classrooms and argue that the CDR approach can be extended to assist educators and favour mathematical education

Investigations into controllers for adaptive autonomous agents based on artificial neural networks.

This thesis reports the development and study of novel architectures for the simulation of adaptive behaviour based on artificial neural networks. There are two distinct levels of enquiry. At the primary level, the initial aim was to design and implement a unified architecture integrating sensorimotor learning and overall control. This was intended to overcome shortcomings of typical behaviour-based approaches in reactive control settings. It was achieved in two stages. Initially, feedforward neural networks were used at the sensorimotor level of a modular architecture and overall control was provided by an algorithm. The algorithm was then replaced by a recurrent neural network. For training, a form of reinforcement learning was used. This posed an intriguing composite of the well-known action selection and credit assignment problems. The solution was demonstrated in two sets of simulation studies involving variants of each architecture. These studies also showed: firstly that the expected advantages over the standard behaviour-based approach were realised, and secondly that the new integrated architecture preserved these advantages, with the added value of a unified control approach. The secondary level of enquiry addressed the more foundational question of whether the choice of processing mechanism is critical if the simulation of adaptive behaviour is to progress much beyond the reactive stage in more than a trivial sense. It proceeded by way of a critique of the standard behaviourbased approach to make a positive assessment of the potential for recurrent neural networks to fill such a role. The findings were used to inform further investigations at the primary level of enquiry. These were based on a framework for the simulation of delayed response learning using supervised learning techniques. A further new architecture, based on a second-order recurrent neural network, was designed for this set of studies. It was then compared with existing architectures. Some interesting results are presented to indicate the appropriateness of the design and the potential of the approach, though limitations in the long run are not discounted

Methods and Apparatus for Autonomous Robotic Control

Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on "stovepiped," or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements

When robots weep : a computational approach to affective learning

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (p. 245-262).This thesis presents a unified computational framework for the study of emotion that integrates several concepts and mechanisms which have been traditionally deemed to be integral components of intelligent behavior. We introduce the notion of affect programs as the primary theoretical constructs for investigating the function and the mechanisms of emotion, and instantiate these in a variety of embodied agents, including physical and simulated robots. Each of these affect programs establishes a functionally distinct mode of operation for the robots, that is activated when specific environmental contingencies are appraised. These modes involve the coordinated adjustment and entrainment of several different systems-including those governing perception, attention, motivation regulation, action selection, learning, and motor control-as part of the implementation of specialized solutions that take advantage of the regularities found in highly recurrent and prototypical environmental contingencies. We demonstrate this framework through multiple experimental scenarios that explore important features of the affect program abstraction and its function, including the demonstration of affective behavior, evaluative conditioning, incentive salience, and affective learning.by Juan David Velásquez.Ph.D

Mental Imagery in Humanoid Robots

Mental imagery presents humans with the opportunity to predict prospective happenings based on own intended actions, to reminisce occurrences from the past and reproduce the perceptual experience. This cognitive capability is mandatory for human survival in this folding and changing world. By means of internal representation, mental imagery offers other cognitive functions (e.g., decision making, planning) the possibility to assess information on objects or events that are not being perceived. Furthermore, there is evidence to suggest that humans are able to employ this ability in the early stages of infancy. Although materialisation of humanoid robot employment in the future appears to be promising, comprehensive research on mental imagery in these robots is lacking. Working within a human environment required more than a set of pre-programmed actions. This thesis aims to investigate the use of mental imagery in humanoid robots, which could be used to serve the demands of their cognitive skills as in humans. Based on empirical data and neuro-imaging studies on mental imagery, the thesis proposes a novel neurorobotic framework which proposes to facilitate humanoid robots to exploit mental imagery. Through conduction of a series of experiments on mental rotation and tool use, the results from this study confirm this potential. Chapters 5 and 6 detail experiments on mental rotation that investigate a bio-constrained neural network framework accounting for mental rotation processes. They are based on neural mechanisms involving not only visual imagery, but also affordance encoding, motor simulation, and the anticipation of the visual consequences of actions. The proposed model is in agreement with the theoretical and empirical research on mental rotation. The models were validated with both a simulated and physical humanoid robot (iCub), engaged in solving a typical mental rotation task. The results show that the model is able to solve a typical mental rotation task and in agreement with data from psychology experiments, they also show response times linearly dependent on the angular disparity between the objects. Furthermore, the experiments in chapter 6 propose a novel neurorobotic model that has a macro-architecture constrained by knowledge on brain, which encompasses a rather general mental rotation mechanism and incorporates a biologically plausible decision making mechanism. The new model is tested within the humanoid robot iCub in tasks requiring to mentally rotate 2D geometrical images appearing on a computer screen. The results show that the robot has an enhanced capacity to generalize mental rotation of new objects and shows the possible effects of overt movements of the wrist on mental rotation. These results indicate that the model represents a further step in the identification of the embodied neural mechanisms that might underlie mental rotation in humans and might also give hints to enhance robots' planning capabilities. In Chapter 7, the primary purpose for conducting the experiment on tool use development through computational modelling refers to the demonstration that developmental characteristics of tool use identified in human infants can be attributed to intrinsic motivations. Through the processes of sensorimotor learning and rewarding mechanisms, intrinsic motivations play a key role as a driving force that drives infants to exhibit exploratory behaviours, i.e., play. Sensorimotor learning permits an emergence of other cognitive functions, i.e., affordances, mental imagery and problem-solving. Two hypotheses on tool use development are also conducted thoroughly. Secondly, the experiment tests two candidate mechanisms that might underlie an ability to use a tool in infants: overt movements and mental imagery. By means of reinforcement learning and sensorimotor learning, knowledge of how to use a tool might emerge through random movements or trial-and-error which might reveal a solution (sequence of actions) of solving a given tool use task accidentally. On the other hand, mental imagery was used to replace the outcome of overt movements in the processes of self-determined rewards. Instead of determining a reward from physical interactions, mental imagery allows the robots to evaluate a consequence of actions, in mind, before performing movements to solve a given tool use task. Therefore, collectively, the case of mental imagery in humanoid robots was systematically addressed by means of a number of neurorobotic models and, furthermore, two categories of spatial problem solving tasks: mental rotation and tool use. Mental rotation evidently involves the employment of mental imagery and this thesis confirms the potential for its exploitation by humanoid robots. Additionally, the studies on tool use demonstrate that the key components assumed and included in the experiments on mental rotation, namely affordances and mental imagery, can be acquired by robots through the processes of sensorimotor learning.Ministry of Science and Technology, the Thai Governmen