Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation

Imitation learning is an effective approach for autonomous systems to acquire control policies when an explicit reward function is unavailable, using supervision provided as demonstrations from an expert, typically a human operator. However, standard imitation learning methods assume that the agent receives examples of observation-action tuples that could be provided, for instance, to a supervised learning algorithm. This stands in contrast to how humans and animals imitate: we observe another person performing some behavior and then figure out which actions will realize that behavior, compensating for changes in viewpoint, surroundings, object positions and types, and other factors. We term this kind of imitation learning "imitation-from-observation," and propose an imitation learning method based on video prediction with context translation and deep reinforcement learning. This lifts the assumption in imitation learning that the demonstration should consist of observations in the same environment configuration, and enables a variety of interesting applications, including learning robotic skills that involve tool use simply by observing videos of human tool use. Our experimental results show the effectiveness of our approach in learning a wide range of real-world robotic tasks modeled after common household chores from videos of a human demonstrator, including sweeping, ladling almonds, pushing objects as well as a number of tasks in simulation.Comment: Accepted at ICRA 2018, Brisbane. YuXuan Liu and Abhishek Gupta had equal contributio

Programming by Demonstration for in-contact tasks using Dynamic Movement Primitives

Despite the rapid growth in the number of robots in the world, the number of service robots is still very low. The major reasons for this include the robots' lack of world knowledge, sensitivity, safety and flexibility. This thesis experimentally addresses the last three of these issues (sensitivity, safety and flexibility) with reference to advanced, industrial level robotic arms provided with integrated torque sensors at each joint. The aims of this work are twofold. The first one, at a more technical level, is the implementation of a real-time software infrastructure, based on Orocos and ROS, for a general, robust, flexible and modular robot control framework with a relatively high level of abstraction. The second aim is to utilize this software framework for Programming by Demonstration with a class of algorithms known as Dynamic Movement Primitives. Using kinesthetic teaching with one or multiple demonstrations, the robot performs simple sequential in-contact tasks (e. g. writing on a notepad a previously demonstrated sequence of characters). The system is not only able to imitate and generalize from demonstrated trajectories, but also from their associated force profiles during the execution of in-contact tasks. The framework is further extended to successfully recover from perturbations during the execution and to cope with dynamic environments

An Energy-based Approach to Ensure the Stability of Learned Dynamical Systems

Non-linear dynamical systems represent a compact, flexible, and robust tool for reactive motion generation. The effectiveness of dynamical systems relies on their ability to accurately represent stable motions. Several approaches have been proposed to learn stable and accurate motions from demonstration. Some approaches work by separating accuracy and stability into two learning problems, which increases the number of open parameters and the overall training time. Alternative solutions exploit single-step learning but restrict the applicability to one regression technique. This paper presents a single-step approach to learn stable and accurate motions that work with any regression technique. The approach makes energy considerations on the learned dynamics to stabilize the system at run-time while introducing small deviations from the demonstrated motion. Since the initial value of the energy injected into the system affects the reproduction accuracy, it is estimated from training data using an efficient procedure. Experiments on a real robot and a comparison on a public benchmark shows the effectiveness of the proposed approach.Comment: Accepted at the International Conference on Robotics and Automation 202