Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.Peer reviewe

Self-organization in Communicating Groups: the emergence of coordination, shared references and collective intelligence\ud

The present paper will sketch the basic ideas of the complexity paradigm, and then apply them to social systems, and in particular to groups of communicating individuals who together need to agree about how to tackle some problem or how to coordinate their actions. I will elaborate these concepts to provide an integrated foundation for a theory of self-organization, to be understood as a non-linear process of spontaneous coordination between actions. Such coordination will be shown to consist of the following components: alignment, division of labor, workflow and aggregation. I will then review some paradigmatic simulations and experiments that illustrate the alignment of references and communicative conventions between communicating agents. Finally, the paper will summarize the preliminary results of a series of experiments that I devised in order to observe the emergence of collective intelligence within a communicating group, and interpret these observations in terms of alignment, division of labor and workflow

Dance of the bulrushes: building conversations between social creatures

The interactive installation is in vogue. Interaction design and physical installations are accepted fixtures of modern life, and with these technology-driven installations beginning to exert influence on modes of mass communication and general expectations for user experiences, it seems appropriate to explore the variety of interactions that exist. This paper surveys a number of successful projects with a critical eye toward assessing the type of communication and/or conversation generated between interactive installations and human participants. Moreover, this exploration seeks to identify whether specific tactics and/or technologies are particularly suited to engendering layers of dialogue or ‘conversations’ within interactive physical computing installations. It is asserted that thoughtful designs incorporating self-organizational abilities can foster rich dialogues in which participants and the installation collaboratively generate value in the interaction. To test this hypothesis an interactive installation was designed and deployed in locations in and around London. Details of the physical objects and employed technologies are discussed, and results of the installation sessions are shown to corroborate the key tenets of this argument in addition to highlighting other concerns that are specifically relevant to the broad topic of interactive design

Constructing living buildings: a review of relevant technologies for a novel application of biohybrid robotics

Biohybrid robotics takes an engineering approach to the expansion and exploitation of biological behaviours for application to automated tasks. Here, we identify the construction of living buildings and infrastructure as a high-potential application domain for biohybrid robotics, and review technological advances relevant to its future development. Construction, civil infrastructure maintenance and building occupancy in the last decades have comprised a major portion of economic production, energy consumption and carbon emissions. Integrating biological organisms into automated construction tasks and permanent building components therefore has high potential for impact. Live materials can provide several advantages over standard synthetic construction materials, including self-repair of damage, increase rather than degradation of structural performance over time, resilience to corrosive environments, support of biodiversity, and mitigation of urban heat islands. Here, we review relevant technologies, which are currently disparate. They span robotics, self-organizing systems, artificial life, construction automation, structural engineering, architecture, bioengineering, biomaterials, and molecular and cellular biology. In these disciplines, developments relevant to biohybrid construction and living buildings are in the early stages, and typically are not exchanged between disciplines. We, therefore, consider this review useful to the future development of biohybrid engineering for this highly interdisciplinary application.publishe

The dynamic neural field approach to cognitive robotics

This tutorial presents an architecture for autonomous robots to generate behavior in joint action tasks. To efficiently interact with another agent in solving a mutual task, a robot should be endowed with cognitive skills such as memory, decision making, action understanding and prediction. The proposed architecture is strongly inspired by our current understanding of the processing principles and the neuronal circuitr underlying these functionalities in the primate brain. As a mathematical framework, we use a coupled system of dynamic neural fields, each representing the basic functionality of neuronal populations in different brain areas. It implements goal-directed behavior in joint action as a continuous process that builds on the interpretation of observed movements in terms of the partner’s action goal. We validate the architecture in two experimental paradigms: (1) a joint search task; (2) a reproduction of an observed or inferred end state of a grasping–placing sequence. We also review some of the mathematical results about dynamic neural fields that are important for the implementation work.European Commission fp6-IST2, project no. 00374

Autonomous Reinforcement of Behavioral Sequences in Neural Dynamics

We introduce a dynamic neural algorithm called Dynamic Neural (DN) SARSA(\lambda) for learning a behavioral sequence from delayed reward. DN-SARSA(\lambda) combines Dynamic Field Theory models of behavioral sequence representation, classical reinforcement learning, and a computational neuroscience model of working memory, called Item and Order working memory, which serves as an eligibility trace. DN-SARSA(\lambda) is implemented on both a simulated and real robot that must learn a specific rewarding sequence of elementary behaviors from exploration. Results show DN-SARSA(\lambda) performs on the level of the discrete SARSA(\lambda), validating the feasibility of general reinforcement learning without compromising neural dynamics.Comment: Sohrob Kazerounian, Matthew Luciw are Joint first author