MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents. However, the demonstrations can be extremely costly and time-consuming to collect. We introduce MimicGen, a system for automatically synthesizing large-scale, rich datasets from only a small number of human demonstrations by adapting them to new contexts. We use MimicGen to generate over 50K demonstrations across 18 tasks with diverse scene configurations, object instances, and robot arms from just ~200 human demonstrations. We show that robot agents can be effectively trained on this generated dataset by imitation learning to achieve strong performance in long-horizon and high-precision tasks, such as multi-part assembly and coffee preparation, across broad initial state distributions. We further demonstrate that the effectiveness and utility of MimicGen data compare favorably to collecting additional human demonstrations, making it a powerful and economical approach towards scaling up robot learning. Datasets, simulation environments, videos, and more at https://mimicgen.github.io .Comment: Conference on Robot Learning (CoRL) 202

From Data to Decision Support in Manufacturing

Digitalization is changing society, industry, and how business is done. For new companies that are more or less born digital, there is the opportunity to use and benefit from the capabilities offered by the new digital technologies, of which data-driven decision-making forms a crucial part. The manufacturing industry is facing the Fourth Industrial Revolution, but most manufacturing organizations are lagging behind in their digital transformation. This is due to the technical and organizational challenges they are experiencing. Based on this current state description and existing gap, the vision, aim, and research questions of this thesis are: Vision - future manufacturing organization to be driven by fact-based decision support based on data rather than of relying mainly on intuition and experience.Aim - to show manufacturing organizations the applicability of digital technologies in digitalizing manufacturing system data to support decision-making and how data sharing may be achieved.Research Question 1. How do manufacturing system lifecycle decisions influence the requirements of data collection towards interoperability? Research Question 2. What makes interoperability standardization applicable to sharing data in a manufacturing system’s lifecycle?This research is applied, addressing real-world problems in manufacturing. For this reason, the main objective is to solve the problem at hand, and data collection methods will be selected that can help address it. This can best be explained by taking a pragmatic worldview and using mixed methods approach that combines quantitative and qualitative methods. The research upon which this thesis is based draws on the results of three research projects involving the active participation of manufacturing companies. The data collection methods included experiments, interviews (focus group and semi-structured), technical development, literature review, and so on. The results are divided into three areas: 1) connected factory, 2) standard representation of machine model data, and 3) the digital twin in smart manufacturing. Connected factory addresses the question of how a mobile connectivity solution, 5G, may be used in a factory setting and demonstrates how the connectivity solution should be planned and how new data from a connected machine may support an operator in decision-making. The standard representation of machine model data demonstrates how an international standard may be used more widely to support the sharing and reuse of information. The digital twin in smart manufacturing investigates the reasons why there are so few real-world examples of this. The findings reveal that a manufacturing system’s lifecycle impacts data requirements, including a need for data accuracy in design, speed of data in operation (to allow operators to act upon events), and availability of historical data in maintenance (for finding root causes and planning). The volume of data was identified as important to all lifecycles. The applicability of standards was found to depend on: 1) the technology providers’ willingness to adapt standards, 2) enforcement by OEMs and larger actors further down a supply chain, 3) the development of standards that consider the user, and 4) when standards are required for infrastructure reasons. Based on the results and findings obtained, it may be stated that it is important to determine what actual manufacturing problem should be addressed by digital technology. There is a tendency to view this change from the perspective of what the digital technology might solve (a technology push), rather than setting aside the solution and focusing on what problem should be solved (a technology pull). This work also reveals the importance of the collaboration between industry and academia making progress in the digital transformation of manufacturing. Proofs-of-concept and demonstrators of how digital technologies might be used are also important tools in helping industry in how they can address a digital transformation

Computational Methods for Cognitive and Cooperative Robotics

In the last decades design methods in control engineering made substantial progress in the areas of robotics and computer animation. Nowadays these methods incorporate the newest developments in machine learning and artificial intelligence. But the problems of flexible and online-adaptive combinations of motor behaviors remain challenging for human-like animations and for humanoid robotics. In this context, biologically-motivated methods for the analysis and re-synthesis of human motor programs provide new insights in and models for the anticipatory motion synthesis. This thesis presents the author’s achievements in the areas of cognitive and developmental robotics, cooperative and humanoid robotics and intelligent and machine learning methods in computer graphics. The first part of the thesis in the chapter “Goal-directed Imitation for Robots” considers imitation learning in cognitive and developmental robotics. The work presented here details the author’s progress in the development of hierarchical motion recognition and planning inspired by recent discoveries of the functions of mirror-neuron cortical circuits in primates. The overall architecture is capable of ‘learning for imitation’ and ‘learning by imitation’. The complete system includes a low-level real-time capable path planning subsystem for obstacle avoidance during arm reaching. The learning-based path planning subsystem is universal for all types of anthropomorphic robot arms, and is capable of knowledge transfer at the level of individual motor acts. Next, the problems of learning and synthesis of motor synergies, the spatial and spatio-temporal combinations of motor features in sequential multi-action behavior, and the problems of task-related action transitions are considered in the second part of the thesis “Kinematic Motion Synthesis for Computer Graphics and Robotics”. In this part, a new approach of modeling complex full-body human actions by mixtures of time-shift invariant motor primitives in presented. The online-capable full-body motion generation architecture based on dynamic movement primitives driving the time-shift invariant motor synergies was implemented as an online-reactive adaptive motion synthesis for computer graphics and robotics applications. The last chapter of the thesis entitled “Contraction Theory and Self-organized Scenarios in Computer Graphics and Robotics” is dedicated to optimal control strategies in multi-agent scenarios of large crowds of agents expressing highly nonlinear behaviors. This last part presents new mathematical tools for stability analysis and synthesis of multi-agent cooperative scenarios.In den letzten Jahrzehnten hat die Forschung in den Bereichen der Steuerung und Regelung komplexer Systeme erhebliche Fortschritte gemacht, insbesondere in den Bereichen Robotik und Computeranimation. Die Entwicklung solcher Systeme verwendet heutzutage neueste Methoden und Entwicklungen im Bereich des maschinellen Lernens und der künstlichen Intelligenz. Die flexible und echtzeitfähige Kombination von motorischen Verhaltensweisen ist eine wesentliche Herausforderung für die Generierung menschenähnlicher Animationen und in der humanoiden Robotik. In diesem Zusammenhang liefern biologisch motivierte Methoden zur Analyse und Resynthese menschlicher motorischer Programme neue Erkenntnisse und Modelle für die antizipatorische Bewegungssynthese. Diese Dissertation präsentiert die Ergebnisse der Arbeiten des Autors im Gebiet der kognitiven und Entwicklungsrobotik, kooperativer und humanoider Robotersysteme sowie intelligenter und maschineller Lernmethoden in der Computergrafik. Der erste Teil der Dissertation im Kapitel “Zielgerichtete Nachahmung für Roboter” behandelt das Imitationslernen in der kognitiven und Entwicklungsrobotik. Die vorgestellten Arbeiten beschreiben neue Methoden für die hierarchische Bewegungserkennung und -planung, die durch Erkenntnisse zur Funktion der kortikalen Spiegelneuronen-Schaltkreise bei Primaten inspiriert wurden. Die entwickelte Architektur ist in der Lage, ‘durch Imitation zu lernen’ und ‘zu lernen zu imitieren’. Das komplette entwickelte System enthält ein echtzeitfähiges Pfadplanungssubsystem zur Hindernisvermeidung während der Durchführung von Armbewegungen. Das lernbasierte Pfadplanungssubsystem ist universell und für alle Arten von anthropomorphen Roboterarmen in der Lage, Wissen auf der Ebene einzelner motorischer Handlungen zu übertragen. Im zweiten Teil der Arbeit “Kinematische Bewegungssynthese für Computergrafik und Robotik” werden die Probleme des Lernens und der Synthese motorischer Synergien, d.h. von räumlichen und räumlich-zeitlichen Kombinationen motorischer Bewegungselemente bei Bewegungssequenzen und bei aufgabenbezogenen Handlungs übergängen behandelt. Es wird ein neuer Ansatz zur Modellierung komplexer menschlicher Ganzkörperaktionen durch Mischungen von zeitverschiebungsinvarianten Motorprimitiven vorgestellt. Zudem wurde ein online-fähiger Synthesealgorithmus für Ganzköperbewegungen entwickelt, der auf dynamischen Bewegungsprimitiven basiert, die wiederum auf der Basis der gelernten verschiebungsinvarianten Primitive konstruiert werden. Dieser Algorithmus wurde für verschiedene Probleme der Bewegungssynthese für die Computergrafik- und Roboteranwendungen implementiert. Das letzte Kapitel der Dissertation mit dem Titel “Kontraktionstheorie und selbstorganisierte Szenarien in der Computergrafik und Robotik” widmet sich optimalen Kontrollstrategien in Multi-Agenten-Szenarien, wobei die Agenten durch eine hochgradig nichtlineare Kinematik gekennzeichnet sind. Dieser letzte Teil präsentiert neue mathematische Werkzeuge für die Stabilitätsanalyse und Synthese von kooperativen Multi-Agenten-Szenarien

Advanced human inspired walking strategies for humanoid robots

Cette thèse traite du problème de la locomotion des robots humanoïdes dans le contexte du projet européen KoroiBot. En s'inspirant de l'être humain, l'objectif de ce projet est l'amélioration des capacités des robots humanoïdes à se mouvoir de façon dynamique et polyvalente. Le coeur de l'approche scientifique repose sur l'utilisation du controle optimal, à la fois pour l'identification des couts optimisés par l'être humain et pour leur mise en oeuvre sur les robots des partenaires roboticiens. Cette thèse s'illustre donc par une collaboration à la fois avec des mathématiciens du contrôle et des spécialistes de la modélisation des primitives motrices. Les contributions majeures de cette thèse reposent donc sur la conception de nouveaux algorithmes temps-réel de contrôle pour la locomotion des robots humanoïdes avec nos collégues de l'université d'Heidelberg et leur intégration sur le robot HRP-2. Deux contrôleurs seront présentés, le premier permettant la locomotion multi-contacts avec une connaissance a priori des futures positions des contacts. Le deuxième étant une extension d'un travail réalisé sur de la marche sur sol plat améliorant les performances et ajoutant des fonctionnalitées au précédent algorithme. En collaborant avec des spécialistes du mouvement humain nous avons implementé un contrôleur innovant permettant de suivre des trajectoires cycliques du centre de masse. Nous présenterons aussi un contrôleur corps-complet utilisant, pour le haut du corps, des primitives de mouvements extraites du mouvement humain et pour le bas du corps, un générateur de marche. Les résultats de cette thèse ont été intégrés dans la suite logicielle "Stack-of-Tasks" du LAAS-CNRS.This thesis covers the topic of humanoid robot locomotion in the frame of the European project KoroiBot. The goal of this project is to enhance the ability of humanoid robots to walk in a dynamic and versatile fashion as humans do. Research and innovation studies in KoroiBot rely on optimal control methods both for the identification of cost functions used by human being and for their implementations on robots owned by roboticist partners. Hence, this thesis includes fruitful collaborations with both control mathematicians and experts in motion primitive modeling. The main contributions of this PhD thesis lies in the design of new real time controllers for humanoid robot locomotion with our partners from the University of Heidelberg and their integration on the HRP-2 robot. Two controllers will be shown, one allowing multi-contact locomotion with a prior knowledge of the future contacts. And the second is an extension of a previous work improving performance and providing additional functionalities. In a collaboration with experts in human motion we designed an innovating controller for tracking cyclic trajectories of the center of mass. We also show a whole body controller using upper body movement primitives extracted from human behavior and lower body movement computed by a walking pattern generator. The results of this thesis have been integrated into the LAAS-CNRS "Stack-of-Tasks" software suit

Human skill capturing and modelling using wearable devices

Industrial robots are delivering more and more manipulation services in manufacturing. However, when the task is complex, it is difficult to programme a robot to fulfil all the requirements because even a relatively simple task such as a peg-in-hole insertion contains many uncertainties, e.g. clearance, initial grasping position and insertion path. Humans, on the other hand, can deal with these variations using their vision and haptic feedback. Although humans can adapt to uncertainties easily, most of the time, the skilled based performances that relate to their tacit knowledge cannot be easily articulated. Even though the automation solution may not fully imitate human motion since some of them are not necessary, it would be useful if the skill based performance from a human could be firstly interpreted and modelled, which will then allow it to be transferred to the robot. This thesis aims to reduce robot programming efforts significantly by developing a methodology to capture, model and transfer the manual manufacturing skills from a human demonstrator to the robot. Recently, Learning from Demonstration (LfD) is gaining interest as a framework to transfer skills from human teacher to robot using probability encoding approaches to model observations and state transition uncertainties. In close or actual contact manipulation tasks, it is difficult to reliabley record the state-action examples without interfering with the human senses and activities. Therefore, wearable sensors are investigated as a promising device to record the state-action examples without restricting the human experts during the skilled execution of their tasks. Firstly to track human motions accurately and reliably in a defined 3-dimensional workspace, a hybrid system of Vicon and IMUs is proposed to compensate for the known limitations of the individual system. The data fusion method was able to overcome occlusion and frame flipping problems in the two camera Vicon setup and the drifting problem associated with the IMUs. The results indicated that occlusion and frame flipping problems associated with Vicon can be mitigated by using the IMU measurements. Furthermore, the proposed method improves the Mean Square Error (MSE) tracking accuracy range from 0.8˚ to 6.4˚ compared with the IMU only method. Secondly, to record haptic feedback from a teacher without physically obstructing their interactions with the workpiece, wearable surface electromyography (sEMG) armbands were used as an indirect method to indicate contact feedback during manual manipulations. A muscle-force model using a Time Delayed Neural Network (TDNN) was built to map the sEMG signals to the known contact force. The results indicated that the model was capable of estimating the force from the sEMG armbands in the applications of interest, namely in peg-in-hole and beater winding tasks, with MSE of 2.75N and 0.18N respectively. Finally, given the force estimation and the motion trajectories, a Hidden Markov Model (HMM) based approach was utilised as a state recognition method to encode and generalise the spatial and temporal information of the skilled executions. This method would allow a more representative control policy to be derived. A modified Gaussian Mixture Regression (GMR) method was then applied to enable motions reproduction by using the learned state-action policy. To simplify the validation procedure, instead of using the robot, additional demonstrations from the teacher were used to verify the reproduction performance of the policy, by assuming human teacher and robot learner are physical identical systems. The results confirmed the generalisation capability of the HMM model across a number of demonstrations from different subjects; and the reproduced motions from GMR were acceptable in these additional tests. The proposed methodology provides a framework for producing a state-action model from skilled demonstrations that can be translated into robot kinematics and joint states for the robot to execute. The implication to industry is reduced efforts and time in programming the robots for applications where human skilled performances are required to cope robustly with various uncertainties during tasks execution

Effective practice in the design of directed independent learning opportunities, : Study skills for academic success: SPOC and MOOC

This compendium, a supplement to the publication Effective practice in the design of directed independent learning opportunities, provides a range of examples of independent learning that will inspire others to review and refresh their own practice. The examples of activity presented here address what for many students will prove to be one of the most challenging aspects of study at the higher level. It is a type of study at the very heart of university education, and is a crucial component for the successful completion of a degree. But, when it comes to its support and development, it is not always particularly well understood. This compendium helps to remedy this, and provides numerous proven examples of good practice from a wide breadth of fields, and from within a range of different institutions