An Adaptive Probabilistic Approach to Goal-Level Imitation Learning

Imitation learning has been recognized as a promising technique to teach robots advanced skills. It is based on the idea that robots could learn new behaviors by observing and imitating the behaviors of other skilled actors. We propose an adaptive probabilistic graphical model which copes with three core issues of any imitative behavior: observation, representation and reproduction of skills. Our model, Growing Hierarchical Dynamic Bayesian Network (GHDBN), is hierarchical (i.e. able to characterize structured behaviors at different levels of abstraction), and growing (i.e. skills are learned or updated incrementally - and at each level of abstraction - every time a new observation sequence is available). A GHDBN, once trained, is able to recognize skills being observed and to reproduce them by exploiting the generative power of the model. The system has been successfully tested in simulation, and initial tests have been conducted on a NAO humanoid robot platform

Robot Learning from Human Demonstrations for Human-Robot Synergy

Human-robot synergy enables new developments in industrial and assistive robotics research. In recent years, collaborative robots can work together with humans to perform a task, while sharing the same workplace. However, the teachability of robots is a crucial factor, in order to establish the role of robots as human teammates. Robots require certain abilities, such as easily learning diversified tasks and adapting to unpredicted events. The most feasible method, which currently utilizes human teammate to teach robots how to perform a task, is the Robot Learning from Demonstrations (RLfD). The goal of this method is to allow non-expert users to a programa a robot by simply guiding the robot through a task. The focus of this thesis is on the development of a novel framework for Robot Learning from Demonstrations that enhances the robotsa abilities to learn and perform the sequences of actions for object manipulation tasks (high-level learning) and, simultaneously, learn and adapt the necessary trajectories for object manipulation (low-level learning). A method that automatically segments demonstrated tasks into sequences of actions is developed in this thesis. Subsequently, the generated sequences of actions are employed by a Reinforcement Learning (RL) from human demonstration approach to enable high-level robot learning. The low-level robot learning consists of a novel method that selects similar demonstrations (in case of multiple demonstrations of a task) and the Gaussian Mixture Model (GMM) method. The developed robot learning framework allows learning from single and multiple demonstrations. As soon as the robot has the knowledge of a demonstrated task, it can perform the task in cooperation with the human. However, the need for adaptation of the learned knowledge may arise during the human-robot synergy. Firstly, Interactive Reinforcement Learning (IRL) is employed as a decision support method to predict the sequence of actions in real-time, to keep the human in the loop and to enable learning the usera s preferences. Subsequently, a novel method that modifies the learned Gaussian Mixture Model (m-GMM) is developed in this thesis. This method allows the robot to cope with changes in the environment, such as objects placed in a different from the demonstrated pose or obstacles, which may be introduced by the human teammate. The modified Gaussian Mixture Model is further used by the Gaussian Mixture Regression (GMR) to generate a trajectory, which can efficiently control the robot. The developed framework for Robot Learning from Demonstrations was evaluated in two different robotic platforms: a dual-arm industrial robot and an assistive robotic manipulator. For both robotic platforms, small studies were performed for industrial and assistive manipulation tasks, respectively. Several Human-Robot Interaction (HRI) methods, such as kinesthetic teaching, gamepad or a hands-freea via head gestures, were used to provide the robot demonstrations. The a hands-freea HRI enables individuals with severe motor impairments to provide a demonstration of an assistive task. The experimental results demonstrate the potential of the developed robot learning framework to enable continuous humana robot synergy in industrial and assistive applications

THE EVALUATIVE CHAMELEON: THE VALANCE OF OBSERVED ACTION OUTCOMES DETERMINES AUTOMATIC IMITATION

Humans have a tendency to imitate the actions they observe in others, a process assumed to rely on an automatic bottom-up mapping of observed action features to one’s own motor system. In contrast, imitation in children is goal-directed, aimed at achieving the same outcome as the model. This thesis examines whether such an outcome-dependence can also be observed in automatic imitation. In six experiments, participants watched an actor make movements after observing the same movements and evaluated the valence of these outcomes with either compatible or incompatible responses. Chapter 2 showed that automatic imitation depends on action outcomes and showed that it is (1) independent from the visual perspective from which the action was observed, but (2) does require identification with the model. Chapter 3 showed that this outcome-dependency is observed in observations of human interactions but not when this element is replaced with non-human stimuli. In chapter 4, 2 experiments in which, the participants’ own action kinematics were measured in an alternating reaching task firstly replicated the well-known sIOR effect such that participants were slower to reach to the same target as the previous player. In contrast to other studies on this effect these experiments revealed tentative evidence that the effect depended upon whether the kinematics required to produce the response bore a similarity to the kinematics of the action one has just observed. Together, the findings in this thesis reveal that imitation cannot simply be attributed to a simple bottom-up matching of observed actions to one’s own action repertoires. Instead, similar to goal directed imitation in children, automatic imitation may be guided by hierarchical action-outcome representations that are dynamically established when watching others act