Learning Task Priorities from Demonstrations

Bimanual operations in humanoids offer the possibility to carry out more than one manipulation task at the same time, which in turn introduces the problem of task prioritization. We address this problem from a learning from demonstration perspective, by extending the Task-Parameterized Gaussian Mixture Model (TP-GMM) to Jacobian and null space structures. The proposed approach is tested on bimanual skills but can be applied in any scenario where the prioritization between potentially conflicting tasks needs to be learned. We evaluate the proposed framework in: two different tasks with humanoids requiring the learning of priorities and a loco-manipulation scenario, showing that the approach can be exploited to learn the prioritization of multiple tasks in parallel.Comment: Accepted for publication at the IEEE Transactions on Robotic

Negotiating Large Obstacles with a Humanoid Robot via Multi-Contact Motion Planning

Incremental progress in humanoid robot locomotion over the years has achieved essential capabilities such as navigation over at or uneven terrain, stepping over small obstacles and imbing stairls. However, the locomotion research has mostly been limited to using only bipedal gait and only foot contacts with the environment, using the upper body for balancing without considering additional external contacts. As a result, challenging locomotion tasks like climbing over large obstacles relative to the size of the robot have remained unsolved. In this paper, we address this class of open problems with an approach based on multi-contact motion planning, guided by physical human demonstrations. Our goal is to make humanoid locomotion problem more tractable by taking advantage of objects in the surrounding environment instead of avoiding them. We propose a multi-contact motion planning algorithm for humanoid robot locomotion which exploits the multi-contacts at the upper and lower body limbs. We propose a contact stability measure, which simplies the contact search from demonstration and contact transition motion generation for the multi-contact motion planning algorithm. The algorithm uses the whole-body motions generated via Quadratic Programming (QP) based solver methods. The multi-contact motion planning algorithm is applied for a challenging task of climbing over a relatively larger obstacle compared to the robot. We validate our planning approach with simulations and experiments for climbing over a large wooden obstacle with COMAN, which is a complaint humanoid robot with 23 degrees of freedom (DOF). We also propose a generalization method, the \Policy-Contraction Learning Method" to extend the algorithm for generating new multi-contact plans for our multi-contact motion planner, that can adapt to changes in the environment. The method learns a general policy and the multi-contact behavior from the human demonstrations, for generating new multi-contact plans for the obstacle-negotiation

“iCub, clean the table!” A robot learning from demonstration approach using Deep Neural Networks

Autonomous service robots have become a key research topic in robotics, particularly for household chores. A typical home scenario is highly unconstrained and a service robot needs to adapt constantly to new situations. In this paper, we address the problem of autonomous cleaning tasks in uncontrolled environments. In our approach, a human instructor uses kinestethic demonstrations to teach a robot how to perform different cleaning tasks on a table. Then, we use Task Parametrized Gaussian Mixture Models (TP-GMMs) to encode the demonstrations variability, while providing appropriate generalization abilities. TP-GMMs extend Gaussian Mixture Models with an auxiliary set of reference frames, in order to extrapolate the demonstrations to different task parameters such as movement locations, amplitude or orientations. However, the reference frames (that parametrize TP-GMMs) can be very difficult to extract in practice, as it may require segmenting the cluttered images of the working table-top. Instead, in this work the reference frames are automatically extracted from robot camera images, using a deep neural network that was trained during human demonstrations of a cleaning task. This approach has two main benefits: (i) it takes the human completely out of the loop while performing complex cleaning tasks; and (ii) the network is able to identify the specific task to be performed directly from image data, thus also enabling automatic task selection from a set of previously demonstrated tasks. The system was implemented on the iCub humanoid robot. During the tests, the robot was able to successfully clean a table with two different types of dirt (wiping a marker’s scribble or sweeping clusters of lentils).info:eu-repo/semantics/publishedVersio

Scaled Autonomy for Networked Humanoids

Humanoid robots have been developed with the intention of aiding in environments designed for humans. As such, the control of humanoid morphology and effectiveness of human robot interaction form the two principal research issues for deploying these robots in the real world. In this thesis work, the issue of humanoid control is coupled with human robot interaction under the framework of scaled autonomy, where the human and robot exchange levels of control depending on the environment and task at hand. This scaled autonomy is approached with control algorithms for reactive stabilization of human commands and planned trajectories that encode semantically meaningful motion preferences in a sequential convex optimization framework. The control and planning algorithms have been extensively tested in the field for robustness and system verification. The RoboCup competition provides a benchmark competition for autonomous agents that are trained with a human supervisor. The kid-sized and adult-sized humanoid robots coordinate over a noisy network in a known environment with adversarial opponents, and the software and routines in this work allowed for five consecutive championships. Furthermore, the motion planning and user interfaces developed in the work have been tested in the noisy network of the DARPA Robotics Challenge (DRC) Trials and Finals in an unknown environment. Overall, the ability to extend simplified locomotion models to aid in semi-autonomous manipulation allows untrained humans to operate complex, high dimensional robots. This represents another step in the path to deploying humanoids in the real world, based on the low dimensional motion abstractions and proven performance in real world tasks like RoboCup and the DRC

Learning Robot Control using a Hierarchical SOM-based Encoding

Hierarchical representations and modeling of sensorimotor observations is a fundamental approach for the development of scalable robot control strategies. Previously, we introduced the novel Hierarchical Self-Organizing Map-based Encoding algorithm (HSOME) that is based on a computational model of infant cognition. Each layer is a temporally augmented SOM and every node updates a decaying activation value. The bottom level encodes sensori-motor instances while their temporal associations are hierarchically built on the layers above. In the past, HSOME has shown to support hierarchical encoding of sequential sensor-actuator observations both in abstract domains and real humanoid robots. Two novel features are presented here starting with the novel skill acquisition in the complex domain of learning a double tap tactile gesture between two humanoid robots. During reproduction, the robot can either perform a double tap or prioritize to receive a higher reward by performing a single tap instead. Secondly, HSOME has been extended to recall past observations and reproduce rhythmic patterns in the absence of input relevant to the joints by priming initially the reproduction of specific skills with an input. We also demonstrate in simulation how a complex behavior emerges from the automatic reuse of distinct oscillatory swimming demonstrations of a robotic salamander

Sistema de aquisição de dados por interface háptica

Mestrado em Engenharia MecânicaNeste trabalho e apresentada uma interface háptica com realimentação de força para a teleoperação de um robô humanoide é que aborda um novo conceito destinado à aprendizagem por demonstração em robôs, denominado de ensino telecinestésico. A interface desenvolvida pretende promover o ensino cinestésico num ambiente de tele-robótica enriquecido pela virtualização háptica do ambiente e restrições do robô. Os dados recolhidos através desta poderão então ser usados em aprendizagem por demonstração, uma abordagem poderosa que permite aprender padrões de movimento sem a necessidade de modelos dinâmicos complexos, mas que geralmente é apresentada com demonstrações que não são fornecidas teleoperando os robôs. Várias experiências são referidas onde o ensino cinestésico em aprendizagem robótica foi utilizado com um sucesso considerável, bem como novas metodologias e aplicações com aparelhos hápticos. Este trabalho foi realizado com base na plataforma proprietária de 27 graus-de-liberdade do Projeto Humanoide da Universidade de Aveiro (PHUA), definindo novas methodologias de comando em tele-operação, uma nova abordagem de software e ainda algumas alterações ao hardware. Um simulador de corpo inteiro do robô em MATLAB SimMechanics é apresentado que é capaz de determinar os requisitos dinâmicos de binário de cada junta para uma dada postura ou movimento, exemplificando com um movimento efectuado para subir um degrau. Ir a mostrar algumas das potencialidades mas também algumas das limitações restritivas do software. Para testar esta nova abordagem tele-cinestésica são dados exemplos onde o utilizador pode desenvolver demonstrações interagindo fisicamente com o robô humanoide através de um joystick háptico PHANToM. Esta metodologia ir a mostrar que permite uma interação natural para o ensino e perceção tele-robóticos, onde o utilizador fornece instruções e correções funcionais estando ciente da dinâmica do sistema e das suas capacidades e limitações físicas. Ser a mostrado que a abordagem consegue atingir um bom desempenho mesmo com operadores inexperientes ou não familiarizados com o sistema. Durante a interação háptica, a informação sensorial e as ordens que guiam a uma tarefa específica podem ser gravados e posteriormente utilizados para efeitos de aprendizagem.In this work an haptic interface using force feedback for the teleoperation of a humanoid robot is presented, that approaches a new concept for robot learning by demonstration known as tele-kinesthethic teaching. This interface aims at promoting kinesthethic teaching in telerobotic environments enriched by the haptic virtualization of the robot's environment and restrictions. The data collected through this interface can later be in robot learning by demonstration, a powerful approach for learning motion patterns without complex dynamical models, but which is usually presented using demonstrations that are not provided by teleoperating the robots. Several experiments are referred where kinesthetic teaching for robot learning was used with considerable success, as well as other new methodologies and applications with haptic devices. This work was conducted on the proprietary 27 DOF University of Aveiro Humanoid Project (PHUA) robot, de ning new wiring and software solutions, as well as a new teleoperation command methodology. A MATLAB Sim- Mechanics full body robot simulator is presented that is able to determine dynamic joint torque requirements for a given robot movement or posture, exempli ed with a step climbing application. It will show some of the potentialities but also some restricting limitations of the software. To test this new tele-kinesthetic approach, examples are shown where the user can provide demonstrations by physically interacting with the humanoid robot through a PHANToM haptic joystick. This methodology will show that it enables a natural interface for telerobotic teaching and sensing, in which the user provides functional guidance and corrections while being aware of the dynamics of the system and its physical capabilities and / or constraints. It will also be shown that the approach can have a good performance even with inexperienced or unfamiliarized operators. During haptic interaction, the sensory information and the commands guiding the execution of a speci c task can be recorded and that data log from the human-robot interaction can be later used for learning purposes

Model-free Probabilistic Movement Primitives for physical interaction

Physical interaction in robotics is a complex problem that requires not only accurate reproduction of the kinematic trajectories but also of the forces and torques exhibited during the movement. We base our approach on Movement Primitives (MP), as MPs provide a framework for modelling complex movements and introduce useful operations on the movements, such as generalization to novel situations, time scaling, and others. Usually, MPs are trained with imitation learning, where an expert demonstrates the trajectories. However, MPs used in physical interaction either require additional learning approaches, e.g., reinforcement learning, or are based on handcrafted solutions. Our goal is to learn and generate movements for physical interaction that are learned with imitation learning, from a small set of demonstrated trajectories. The Probabilistic Movement Primitives (ProMPs) framework is a recent MP approach that introduces beneficial properties, such as combination and blending of MPs, and represents the correlations present in the movement. The ProMPs provides a variable stiffness controller that reproduces the movement but it requires a dynamics model of the system. Learning such a model is not a trivial task, and, therefore, we introduce the model-free ProMPs, that are learning jointly the movement and the necessary actions from a few demonstrations. We derive a variable stiffness controller analytically. We further extent the ProMPs to include force and torque signals, necessary for physical interaction. We evaluate our approach in simulated and real robot tasks

Online quantum mixture regression for trajectory learning by demonstration

In this work, we present the online Quantum Mixture Model (oQMM), which combines the merits of quantum mechanics and stochastic optimization. More specifically it allows for quantum effects on the mixture states, which in turn become a superposition of conventional mixture states. We propose an efficient stochastic online learning algorithm based on the online Expectation Maximization (EM), as well as a generation and decay scheme for model components. Our method is suitable for complex robotic applications, where data is abundant or where we wish to iteratively refine our model and conduct predictions during the course of learning. With a synthetic example, we show that the algorithm can achieve higher numerical stability. We also empirically demonstrate the efficacy of our method in well-known regression benchmark datasets. Under a trajectory Learning by Demonstration setting we employ a multi-shot learning application in joint angle space, where we observe higher quality of learning and reproduction. We compare against popular and well-established methods, widely adopted across the robotics community