Adaptation of the difficulty level in an infant-robot movement contingency study

19th International Workshop of Physical Agents (WAF). Madrid (22-23 Noviembre 2018)ABSTRACT: This paper presents a personalized contingency feedback adaptation system that aims to encourage infants aged 6 to 8 months to gradually increase the peak acceleration of their leg movements. The ultimate challenge is to determine if a socially assistive humanoid robot can guide infant learning using contingent rewards, where the reward threshold is personalized for each infant using a reinforcement learning algorithm. The model learned from the data captured by wearable inertial sensors measuring infant leg movement accelerations in an earlier study. Each infant generated a unique model that determined the behavior of the robot. The presented results were obtained from the distributions of the participants' acceleration peaks and demonstrate that the resulting model is sensitive to the degree of differentiation among the participants; each participant (infant) should have his/her own learned policy.This work was supported by NSF award 1706964 (PI: Smith, Co-PI: Matarić). In addition, this work was developed during an international mobility program at the University of Southern California being also partially funded by the European Union ECHORD++ project (FP7-ICT-601116), the LifeBots project (TIN2015-65686-C5) and THERAPIST project (TIN2012-38079)

Introduction: The Fourth International Workshop on Epigenetic Robotics

As in the previous editions, this workshop is trying to be a forum for multi-disciplinary research ranging from developmental psychology to neural sciences (in its widest sense) and robotics including computational studies. This is a two-fold aim of, on the one hand, understanding the brain through engineering embodied systems and, on the other hand, building artificial epigenetic systems. Epigenetic contains in its meaning the idea that we are interested in studying development through interaction with the environment. This idea entails the embodiment of the system, the situatedness in the environment, and of course a prolonged period of postnatal development when this interaction can actually take place. This is still a relatively new endeavor although the seeds of the developmental robotics community were already in the air since the nineties (Berthouze and Kuniyoshi, 1998; Metta et al., 1999; Brooks et al., 1999; Breazeal, 2000; Kozima and Zlatev, 2000). A few had the intuition – see Lungarella et al. (2003) for a comprehensive review – that, intelligence could not be possibly engineered simply by copying systems that are “ready made” but rather that the development of the system fills a major role. This integration of disciplines raises the important issue of learning on the multiple scales of developmental time, that is, how to build systems that eventually can learn in any environment rather than program them for a specific environment. On the other hand, the hope is that robotics might become a new tool for brain science similarly to what simulation and modeling have become for the study of the motor system. Our community is still pretty much evolving and “under construction” and for this reason, we tried to encourage submissions from the psychology community. Additionally, we invited four neuroscientists and no roboticists for the keynote lectures. We received a record number of submissions (more than 50), and given the overall size and duration of the workshop together with our desire to maintain a single-track format, we had to be more selective than ever in the review process (a 20% acceptance rate on full papers). This is, if not an index of quality, at least an index of the interest that gravitates around this still new discipline

Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.”This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.Peer reviewe

Targeted Movement Pattern Recognition for Infants with Perinatal Brachial Plexus Injury

This thesis presents work toward a novel rehabilitation tool for infants with limited arm movement such as those who sustain a perinatal brachial plexus injury (PBPI). PBPI is a traction injury to peripheral nerves that occurs during the birth process. An injury to the upper trunk of the plexus (C5-6 spinal nerves) partially or fully denervates the skin of the upper arm and muscles of the elbow (i.e., biceps, brachialis) and shoulder (i.e., Deltoid, Sternal Pectoralis Major, Rotator Cuff). Initially, PBPI is typically treated with Passive Range of Motion (PROM) and positioning led by a physical therapist or occupational therapist. If recovery is limited, nerve microsurgery is indicated by 6 months of age. Recovery in infants with PBPI varies from 38 to 80% of infants depending on the initial condition and severity, as well as the rate of reinnervation. Yet, through technology, there may be methods available to increase the rate of recovery, leading to greater use of the affected arm. Infants as young as 3 months of age have been found to increase arm and leg movements through a paradigm of contingent reinforcement (i.e. rewarding desired movement patterns with audiovisual stimulation such as an overhead mobile). Before a device can be fully constructed to provide contingent reinforcement for desired arm movements, those movements must be consistently detectable. Thus, for this thesis project, I studied automatic detection of the arm movements desired for young infants with PBPI using arm acceleration data.Acceleration data was acquired from a wrist-worn sensor as an adult volunteer moved her arm in the desired motion. I implemented a template-matching algorithm based on taking the dot-product of a moving window of 3D acceleration vectors with template acceleration patterns, where the templates were movement samples deemed targeted and rehabilitative by an experienced physical therapist. I found that the algorithm detected rehabilitative movements with an accuracy of ~90%. The algorithm never identified a movement that was deemed as undesirable for this population of infants, by the physical therapist. These results reveal the potential for the template matching algorithm to be used in a contingent reinforcement paradigm capable of activating a toy to encourage infants with PBPI to make targeted rehabilitative arm movements

Final report key contents: main results accomplished by the EU-Funded project IM-CLeVeR - Intrinsically Motivated Cumulative Learning Versatile Robots

This document has the goal of presenting the main scientific and technological achievements of the project IM-CLeVeR. The document is organised as follows: 1. Project executive summary: a brief overview of the project vision, objectives and keywords. 2. Beneficiaries of the project and contacts: list of Teams (partners) of the project, Team Leaders and contacts. 3. Project context and objectives: the vision of the project and its overall objectives 4. Overview of work performed and main results achieved: a one page overview of the main results of the project 5. Overview of main results per partner: a bullet-point list of main results per partners 6. Main achievements in detail, per partner: a throughout explanation of the main results per partner (but including collaboration work), with also reference to the main publications supporting them

Cognitive mechanisms for responding to mimicry from others

Compared to our understanding of neurocognitive processes involved producing mimicry, the downstream consequences of being mimicked are less clear. A wide variety of positive consequences of mimicry, such as liking and helping, have been reported in behavioural research. However, an in-depth review suggests the link from mimicry to liking and other positive outcomes may be fragile. Positive responses to mimicry can break down due to individual factors and social situations where mimicry may be unexpected. It remains unclear how the complex behavioural effects of mimicry relate to neural systems which respond to being mimicked. Mimicry activates regions associated with mirror properties, self-other processing and reward. In this review, we outline three potential models linking these regions with cognitive consequences of being mimicked. The models suggest that positive downstream consequences of mimicry may depend upon self-other overlap, detection of contingency or low prediction error. Finally, we highlight limitations with traditional research designs and suggest alternative methods for achieving highly ecological validity and experimental control. We also highlight unanswered questions which may guide future research

An INNOVATIVE USE of TECHNOLOGY and ASSOCIATIVE LEARNING to ASSESS PRONE MOTOR LEARNING and DESIGN INTERVENTIONS to ENHANCE MOTOR DEVELOPMENT in INFANTS

Since the introduction of the American Academy of Pediatrics Back to Sleep Campaign infants have not met the recommendation to “incorporate supervised, awake “prone play” in their infant’s daily routine to support motor development and minimize the risk of plagiocephaly”. Interventions are needed to increase infants’ tolerance for prone position and prone playtime to reduce the risk of plagiocephaly and motor delays. Associative learning is the ability to understand causal relationship between events. Operant conditioning is a form of associative learning that occurs by associating a behavior with positive or negative consequences. Operant conditions has been utilized to encourage behaviors such as kicking, reaching and sucking in infants by associating these behaviors with positive reinforcement. This dissertation is a compilation of three papers that each represent a study used to investigate a potential play based interventions to encourage prone motor skills in infants. The first paper describes a series of experiment used to develop the Prone Play Activity Center (PPAC) and experimental protocols used in the other studies. The purpose of the second study was to determine the feasibility of a clinical trial comparing usual care (low tech) to a high-tech intervention based on the principles of operant conditioning to increase tolerance for prone and improve prone motor skills. Ten infants participated in the study where parents of infants in the high tech intervention group (n=5) used the PPAC for 3 weeks to practice prone play. Findings from this study suggested the proposed intervention is feasible with some modifications for a future large-scale clinical trial. The purpose of the third study evaluated the ability of 3-6 months old infants to demonstrate AL in prone and remember the association learned a day later. Findings from this study suggested that a majority of infants demonstrated AL in prone with poor retention of the association, 24 hours later. Taken together these 3 papers provide preliminary evidence that a clinical trial of an intervention is feasible and that associative learning could be used to reinforce specific prone motor behaviors in the majority of infants

From locomotion to cognition: Bridging the gap between reactive and cognitive behavior in a quadruped robot

The cognitivistic paradigm, which states that cognition is a result of computation with symbols that represent the world, has been challenged by many. The opponents have primarily criticized the detachment from direct interaction with the world and pointed to some fundamental problems (for instance the symbol grounding problem). Instead, they emphasized the constitutive role of embodied interaction with the environment. This has motivated the advancement of synthetic methodologies: the phenomenon of interest (cognition) can be studied by building and investigating whole brain-body-environment systems. Our work is centered around a compliant quadruped robot equipped with a multimodal sensory set. In a series of case studies, we investigate the structure of the sensorimotor space that the application of different actions in different environments by the robot brings about. Then, we study how the agent can autonomously abstract the regularities that are induced by the different conditions and use them to improve its behavior. The agent is engaged in path integration, terrain discrimination and gait adaptation, and moving target following tasks. The nature of the tasks forces the robot to leave the ``here-and-now'' time scale of simple reactive stimulus-response behaviors and to learn from its experience, thus creating a ``minimally cognitive'' setting. Solutions to these problems are developed by the agent in a bottom-up fashion. The complete scenarios are then used to illuminate the concepts that are believed to lie at the basis of cognition: sensorimotor contingencies, body schema, and forward internal models. Finally, we discuss how the presented solutions are relevant for applications in robotics, in particular in the area of autonomous model acquisition and adaptation, and, in mobile robots, in dead reckoning and traversability detection