Computational and Robotic Models of Early Language Development: A Review

We review computational and robotics models of early language learning and development. We first explain why and how these models are used to understand better how children learn language. We argue that they provide concrete theories of language learning as a complex dynamic system, complementing traditional methods in psychology and linguistics. We review different modeling formalisms, grounded in techniques from machine learning and artificial intelligence such as Bayesian and neural network approaches. We then discuss their role in understanding several key mechanisms of language development: cross-situational statistical learning, embodiment, situated social interaction, intrinsically motivated learning, and cultural evolution. We conclude by discussing future challenges for research, including modeling of large-scale empirical data about language acquisition in real-world environments. Keywords: Early language learning, Computational and robotic models, machine learning, development, embodiment, social interaction, intrinsic motivation, self-organization, dynamical systems, complexity.Comment: to appear in International Handbook on Language Development, ed. J. Horst and J. von Koss Torkildsen, Routledg

Recurrent neural networks and adaptive motor control

This thesis is concerned with the use of neural networks for motor control tasks. The main goal of the thesis is to investigate ways in which the biological notions of motor programs and Central Pattern Generators (CPGs) may be implemented in a neural network framework. Biological CPGs can be seen as components within a larger control scheme, which is basically modular in design. In this thesis, these ideas are investigated through the use of modular recurrent networks, which are used in a variety of control tasks. The first experimental chapter deals with learning in recurrent networks, and it is shown that CPGs may be easily implemented using the machinery of backpropagation. The use of these CPGs can aid the learning of pattern generation tasks; they can also mean that the other components in the system can be reduced in complexity, say, to a purely feedforward network. It is also shown that incremental learning, or 'shaping' is an effective method for building CPGs. Genetic algorithms are also used to build CPGs; although computational effort prevents this from being a practical method, it does show that GAs are capable of optimising systems that operate in the context of a larger scheme. One interesting result from the GA is that optimal CPGs tend to have unstable dynamics, which may have implications for building modular neural controllers. The next chapter applies these ideas to some simple control tasks involving a highly redundant simulated robot arm. It was shown that it is relatively straightforward to build CPGs that represent elements of pattern generation, constraint satisfaction. and local feedback. This is indirect control, in which errors are backpropagated through a plant model, as well as the ePG itself, to give errors for the controller. Finally, the third experimental chapter takes an alternative approach, and uses direct control methods, such as reinforcement learning. In reinforcement learning, controller outputs have unmodelled effects; this allows us to build complex control systems, where outputs modulate the couplings between sets of dynamic systems. This was shown for a simple case, involving a system of coupled oscillators. A second set of experiments investigates the use of simplified models of behaviour; this is a reduced form of supervised learning, and the use of such models in control is discussed

TOWARDS THE GROUNDING OF ABSTRACT CATEGORIES IN COGNITIVE ROBOTS

The grounding of language in humanoid robots is a fundamental problem, especially in social scenarios which involve the interaction of robots with human beings. Indeed, natural language represents the most natural interface for humans to interact and exchange information about concrete entities like KNIFE, HAMMER and abstract concepts such as MAKE, USE. This research domain is very important not only for the advances that it can produce in the design of human-robot communication systems, but also for the implication that it can have on cognitive science. Abstract words are used in daily conversations among people to describe events and situations that occur in the environment. Many scholars have suggested that the distinction between concrete and abstract words is a continuum according to which all entities can be varied in their level of abstractness. The work presented herein aimed to ground abstract concepts, similarly to concrete ones, in perception and action systems. This permitted to investigate how different behavioural and cognitive capabilities can be integrated in a humanoid robot in order to bootstrap the development of higher-order skills such as the acquisition of abstract words. To this end, three neuro-robotics models were implemented. The first neuro-robotics experiment consisted in training a humanoid robot to perform a set of motor primitives (e.g. PUSH, PULL, etc.) that hierarchically combined led to the acquisition of higher-order words (e.g. ACCEPT, REJECT). The implementation of this model, based on a feed-forward artificial neural networks, permitted the assessment of the training methodology adopted for the grounding of language in humanoid robots. In the second experiment, the architecture used for carrying out the first study was reimplemented employing recurrent artificial neural networks that enabled the temporal specification of the action primitives to be executed by the robot. This permitted to increase the combinations of actions that can be taught to the robot for the generation of more complex movements. For the third experiment, a model based on recurrent neural networks that integrated multi-modal inputs (i.e. language, vision and proprioception) was implemented for the grounding of abstract action words (e.g. USE, MAKE). Abstract representations of actions ("one-hot" encoding) used in the other two experiments, were replaced with the joints values recorded from the iCub robot sensors. Experimental results showed that motor primitives have different activation patterns according to the action's sequence in which they are embedded. Furthermore, the performed simulations suggested that the acquisition of concepts related to abstract action words requires the reactivation of similar internal representations activated during the acquisition of the basic concepts, directly grounded in perceptual and sensorimotor knowledge, contained in the hierarchical structure of the words used to ground the abstract action words.This study was financed by the EU project RobotDoC (235065) from the Seventh Framework Programme (FP7), Marie Curie Actions Initial Training Network

Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.Peer reviewe

Turing Learning: Advances and Applications

Turing Learning is the family of algorithms where models and discriminators are generated in a competitive setting. This thesis concerns the coevolutionary framework of Turing Learning and investigates the advances for improved model accuracy and the applications in robotic systems. Advances proposed in this thesis are as follows: an interactive approach to enable the discriminator to genuinely influence the data sampling process; a hybrid formulation to combine the benefits of the interactive discriminator in improving model accuracy and the advantages of the passive discriminator for reducing training cost; an exclusiveness reward mechanism to promote candidates with the exclusive performance during the coevolutionary process. Applications presented in this thesis are as follows: an approach for a mobile robotic agent to automatically infer its sensor configuration; an approach for the robot agent to automatically calibrate its sensor reading; a novel approach to infer swarm behaviours from their effects on the environment. The interactive approach has been validated in the inference of sensor configuration and calibration model, leading to the self-modelling/self-discovery process of robotic agents. Results suggest an improved model accuracy with the interactive approach in both cases, compared with the passive approach. The hybrid formulation and the exclusiveness reward mechanism have been demonstrated in the inference of the calibration model. Results show that almost half of the training cost can be reduced without a decrease in model accuracy by applying the hybrid formulation. The novel reward mechanism can accelerate the convergence without a decrease in model accuracy. The indirect way of inferring swarm behaviours requires a small amount of training and reveals novel behavioural controllers for individual robots

The Cognitive and Neural Underpinnings of Language Learning and Processing

Language is at the epicenter of our existence, allowing us to communicate and think about complex ideas, feelings, and anything else we want. While language is often regarded as an isolated, unique phenomenon when looking across taxa, it is important to realize that this apparent detachment from the rest of cognition is illusory – our human language abilities are in fact highly dependent upon more basic cognitive processes. Much of this dissertation focuses on the ways in which the process of statistical learning both allows for and constrains language learning and processing. Statistical learning can be thought of as a cognitive process by which learners implicitly form associations between stimuli by tracking and storing the underlying statistical relationships between such elements. To provide insight into the relationship between statistical learning and language, two studies are reported here. I first demonstrate the reliability of paradigms that are frequently used to test this construct, while also showing how individual differences in statistical learning are correlated with biases in language processing. In the second study, I characterize how constraints on the input available to learners can affect their ability to acquire statistically learned grammatical regularities. I also establish that such knowledge is retained over time, by examining performance at a follow-up session two-weeks after training. The third study puts long-held assumptions about the modularity of the brain’s language network to the test by examining neuroplasticity in adult patients with brain tumors. The results of this study show that the right frontal lobe is capable of maintaining language function when there is damage to the left frontal lobe. Together, the findings reported within offer evidence for a language system that is highly sensitive to the distributional properties of the input, and is characterized by processes of entrenchment and plasticity

Drama, a connectionist model for robot learning: experiments on grounding communication through imitation in autonomous robots

The present dissertation addresses problems related to robot learning from demonstra¬ tion. It presents the building of a connectionist architecture, which provides the robot with the necessary cognitive and behavioural mechanisms for learning a synthetic lan¬ guage taught by an external teacher agent. This thesis considers three main issues: 1) learning of spatio-temporal invariance in a dynamic noisy environment, 2) symbol grounding of a robot's actions and perceptions, 3) development of a common symbolic representation of the world by heterogeneous agents.We build our approach on the assumption that grounding of symbolic communication creates constraints not only on the cognitive capabilities of the agent but also and especially on its behavioural capacities. Behavioural skills, such as imitation, which allow the agent to co-ordinate its actionn to that of the teacher agent, are required aside to general cognitive abilities of associativity, in order to constrain the agent's attention to making relevant perceptions, onto which it grounds the teacher agent's symbolic expression. In addition, the agent should be provided with the cognitive capacity for extracting spatial and temporal invariance in the continuous flow of its perceptions. Based on this requirement, we develop a connectionist architecture for learning time series. The model is a Dynamical Recurrent Associative Memory Architecture, called DRAMA. It is a fully connected recurrent neural network using Hebbian update rules. Learning is dynamic and unsupervised. The performance of the architecture is analysed theoretically, through numerical simulations and through physical and simulated robotic experiments. Training of the network is computationally fast and inexpensive, which allows its implementation for real time computation and on-line learning in a inexpensive hardware system. Robotic experiments are carried out with different learning tasks involving recognition of spatial and temporal invariance, namely landmark recognition and prediction of perception-action sequence in maze travelling.The architecture is applied to experiments on robot learning by imitation. A learner robot is taught by a teacher agent, a human instructor and another robot, a vocabulary to describe its perceptions and actions. The experiments are based on an imitative strategy, whereby the learner robot reproduces the teacher's actions. While imitating the teacher's movements, the learner robot makes similar proprio and exteroceptions to those of the teacher. The learner robot grounds the teacher's words onto the set of common perceptions they share. We carry out experiments in simulated and physical environments, using different robotic set-ups, increasing gradually the complexity of the task. In a first set of experiments, we study transmission of a vocabulary to designate actions and perception of a robot. Further, we carry out simulation studies, in which we investigate transmission and use of the vocabulary among a group of robotic agents. In a third set of experiments, we investigate learning sequences of the robot's perceptions, while wandering in a physically constrained environment. Finally, we present the implementation of DRAMA in Robota, a doll-like robot, which can imitate the arms and head movements of a human instructor. Through this imitative game, Robota is taught to perform and label dance patterns. Further, Robota is taught a basic language, including a lexicon and syntactical rules for the combination of words of the lexicon, to describe its actions and perception of touch onto its body

Integration of Gravitational Torques in Cerebellar Pathways Allows for the Dynamic Inverse Computation of Vertical Pointing Movements of a Robot Arm

Several authors suggested that gravitational forces are centrally represented in the brain for planning, control and sensorimotor predictions of movements. Furthermore, some studies proposed that the cerebellum computes the inverse dynamics (internal inverse model) whereas others suggested that it computes sensorimotor predictions (internal forward model).This study proposes a model of cerebellar pathways deduced from both biological and physical constraints. The model learns the dynamic inverse computation of the effect of gravitational torques from its sensorimotor predictions without calculating an explicit inverse computation. By using supervised learning, this model learns to control an anthropomorphic robot arm actuated by two antagonists McKibben artificial muscles. This was achieved by using internal parallel feedback loops containing neural networks which anticipate the sensorimotor consequences of the neural commands. The artificial neural networks architecture was similar to the large-scale connectivity of the cerebellar cortex. Movements in the sagittal plane were performed during three sessions combining different initial positions, amplitudes and directions of movements to vary the effects of the gravitational torques applied to the robotic arm. The results show that this model acquired an internal representation of the gravitational effects during vertical arm pointing movements.This is consistent with the proposal that the cerebellar cortex contains an internal representation of gravitational torques which is encoded through a learning process. Furthermore, this model suggests that the cerebellum performs the inverse dynamics computation based on sensorimotor predictions. This highlights the importance of sensorimotor predictions of gravitational torques acting on upper limb movements performed in the gravitational field

Redundancy reduction in motor control

Research in machine learning and neuroscience has made remarkable progress by investigating statistical redundancy in representations of natural environments, but to date much of this work has focused on sensory information like images and sounds. This dissertation explores the notions of redundancy and efficiency in the motor domain, where several different forms of independence exist. The dissertation begins by discussing redundancy at a conceptual level and presents relevant background material. Next, three main branches of original research are described. The first branch consists of a novel control framework for integrating low-bandwidth sensory updates with model uncertainty and action selection for navigating complex, multi-task environments. The second branch of research applies existing machine learning techniques to movement information and explores the mismatch between these methods for extracting independent components and the forms of redundancy that exist in the motor domain. The third branch of work analyzes full-body, goal-directed reaching movements gathered in a novel laboratory experiment, using explicitly measured information about the goal of each movement to uncover patterns in the movement dynamics. Each branch of research explores redundancy reduction in movement from a different perspective, building up a sort of catalog of the types of information present in movements. Redundancy is discussed throughout as an an important aspect of movement in the natural world. The dissertation concludes by summarizing the contributions of these three branches of work, and discussing promising areas for future work spurred by these investigations. More detailed models of voluntary movements hold promise not only for better treatments, improved prosthetics, smoother animations, and more fluid robots, but also as an avenue for scientific insight into the very foundations of cognition.Computer Science