Scaled Autonomy for Networked Humanoids

Humanoid robots have been developed with the intention of aiding in environments designed for humans. As such, the control of humanoid morphology and effectiveness of human robot interaction form the two principal research issues for deploying these robots in the real world. In this thesis work, the issue of humanoid control is coupled with human robot interaction under the framework of scaled autonomy, where the human and robot exchange levels of control depending on the environment and task at hand. This scaled autonomy is approached with control algorithms for reactive stabilization of human commands and planned trajectories that encode semantically meaningful motion preferences in a sequential convex optimization framework. The control and planning algorithms have been extensively tested in the field for robustness and system verification. The RoboCup competition provides a benchmark competition for autonomous agents that are trained with a human supervisor. The kid-sized and adult-sized humanoid robots coordinate over a noisy network in a known environment with adversarial opponents, and the software and routines in this work allowed for five consecutive championships. Furthermore, the motion planning and user interfaces developed in the work have been tested in the noisy network of the DARPA Robotics Challenge (DRC) Trials and Finals in an unknown environment. Overall, the ability to extend simplified locomotion models to aid in semi-autonomous manipulation allows untrained humans to operate complex, high dimensional robots. This represents another step in the path to deploying humanoids in the real world, based on the low dimensional motion abstractions and proven performance in real world tasks like RoboCup and the DRC

From walking to running: robust and 3D humanoid gait generation via MPC

Humanoid robots are platforms that can succeed in tasks conceived for humans. From locomotion in unstructured environments, to driving cars, or working in industrial plants, these robots have a potential that is yet to be disclosed in systematic every-day-life applications. Such a perspective, however, is opposed by the need of solving complex engineering problems under the hardware and software point of view. In this thesis, we focus on the software side of the problem, and in particular on locomotion control. The operativity of a legged humanoid is subordinate to its capability of realizing a reliable locomotion. In many settings, perturbations may undermine the balance and make the robot fall. Moreover, complex and dynamic motions might be required by the context, as for instance it could be needed to start running or climbing stairs to achieve a certain location in the shortest time. We present gait generation schemes based on Model Predictive Control (MPC) that tackle both the problem of robustness and tridimensional dynamic motions. The proposed control schemes adopt the typical paradigm of centroidal MPC for reference motion generation, enforcing dynamic balance through the Zero Moment Point condition, plus a whole-body controller that maps the generated trajectories to joint commands. Each of the described predictive controllers also feature a so-called stability constraint, preventing the generation of diverging Center of Mass trajectories with respect to the Zero Moment Point. Robustness is addressed by modeling the humanoid as a Linear Inverted Pendulum and devising two types of strategies. For persistent perturbations, a way to use a disturbance observer and a technique for constraint tightening (to ensure robust constraint satisfaction) are presented. In the case of impulsive pushes instead, techniques for footstep and timing adaptation are introduced. The underlying approach is to interpret robustness as a MPC feasibility problem, thus aiming at ensuring the existence of a solution for the constrained optimization problem to be solved at each iteration in spite of the perturbations. This perspective allows to devise simple solutions to complex problems, favoring a reliable real-time implementation. For the tridimensional locomotion, on the other hand, the humanoid is modeled as a Variable Height Inverted Pendulum. Based on it, a two stage MPC is introduced with particular emphasis on the implementation of the stability constraint. The overall result is a gait generation scheme that allows the robot to overcome relatively complex environments constituted by a non-flat terrain, with also the capability of realizing running gaits. The proposed methods are validated in different settings: from conceptual simulations in Matlab to validations in the DART dynamic environment, up to experimental tests on the NAO and the OP3 platforms

Human-Inspired Balancing and Recovery Stepping for Humanoid Robots

Robustly maintaining balance on two legs is an important challenge for humanoid robots. The work presented in this book represents a contribution to this area. It investigates efficient methods for the decision-making from internal sensors about whether and where to step, several improvements to efficient whole-body postural balancing methods, and proposes and evaluates a novel method for efficient recovery step generation, leveraging human examples and simulation-based reinforcement learning

Robot motion planning with contact from global pseudo-inverse map

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Mechanical Engineering, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 101-108).In the robot motion planning problems, environment and its objects are often treated as obstacles to be avoided. However, there are situations where contacting with the environment is not costly. Moreover, in many cases, making contact can actually help a robot to maneuver around to reach a goal state which would not have been possible otherwise. This thesis presents a framework for motion planner that utilizes multiple contacts with the environment and its objects. The planner is targeted to autonomously generate motion, where robot has to make multiple contact with different part of its body in order to achieve a task objective. It is motivated by and has significance in developing a robust humanoid planner that is capable of recovering from a fall down. The recent DRC has been marked with compilation of humanoid robots falling down, but only one robot managed to recover to a standing up position. In a real disaster scenario, the inability to stand up would mean end of the rescue mission for what is extremely expensive machinery. A robust planner capable of recovery is must and this work contributes towards it. The developed planner autonomously generates standing up motion from fall down in the presence of torque limits. The proposed multi-contact motion planner leverages upon following two key components. Existing multi-contact planners require good initial seeds to successfully generate a motion. These are hard to find and often manually encoded. Here, we utilize pre-computed global pseudo-inverse map (inverse kinematic map for each contact-state that has property of global resolution, connected by connectivity functions) to generate multi-contact motion from current configuration to the goal without need for an initial seed. Nevertheless, constructing the global pseudo-inverse map is computationally expensive. In an effort to facilitate the construction, we utilize singular configurations as a heuristic to reduce the search space and justify its use based on the physical analysis. Although computationally expensive, once pre-computed, the global map can be used to generate plans fast online in a multi-query manner.by Changrak Choi.Ph. D

Off-Policy Temporal Difference Learning For Robotics And Autonomous Systems

Reinforcement learning (RL) is a rapidly advancing field with implications in autonomous vehicles, medicine, finance, along with several other applications. Particularly, off-policy temporal difference (TD) learning, a specific type of RL technique, has been widely used in a variety of autonomous tasks. However, there remain significant challenges that must be overcome before it can be successfully applied to various real-world applications. In this thesis, we specifically address several major challenges in off-policy TD learning. In the first part of the thesis, we introduce an efficient method of learning complex stand-up motion of humanoid robots by Q-learning. Standing up after falling is an essential ability for humanoid robots yet it is difficult to learn flexible stand-up motions for various fallen positions due to the complexity of the task. We reduce sample complexity of learning by applying a clustering method and utilizing the bilateral symmetric feature of humanoid robots. The learned policy is demonstrated in both simulation and on a physical robot. The greedy update of Q-learning, however, often causes overoptimism and instability. In the second part of the thesis, we propose a novel Bayesian approach to Q-learning, called ADFQ, which improves the greedy update issues by providing a principled way of updating Q-values based on uncertainty of Q-belief distributions. The algorithm converges to Q-learning as the uncertainty approaches zero, and its efficient computational complexity enables the algorithm to be extended with a neural network. Both ADFQ and its neural network extension outperform their comparing algorithms by improving the estimation bias and converging faster to optimal Q-values. In the last part of the thesis, we apply off-policy TD methods to solve the active information acquisition problem where an autonomous agent is tasked with acquiring information about targets of interests. Off-policy TD learning provides solutions for classical challenges in this problem -- system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. In particular, we introduce a method of learning a unified policy for in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model

Learning Motion Skills for a Humanoid Robot

This thesis investigates the learning of motion skills for humanoid robots. As groundwork, a humanoid robot with integrated fall management was developed as an experimental platform. Then, two different approaches for creating motion skills were investigated. First, one that is based on Cartesian quintic splines with optimized parameters. Second, a reinforcement learning-based approach that utilizes the first approach as a reference motion to guide the learning. Both approaches were tested on the developed robot and on further simulated robots to show their generalization. A special focus was set on the locomotion skill, but a standing-up and kick skill are also discussed. Diese Dissertation beschäftigt sich mit dem Lernen von Bewegungsfähigkeiten für humanoide Roboter. Als Grundlage wurde zunächst ein humanoider Roboter mit integriertem Fall Management entwickelt, welcher als Experimentalplatform dient. Dann wurden zwei verschiedene Ansätze für die Erstellung von Bewegungsfähigkeiten untersucht. Zu erst einer der kartesische quintische Splines mit optimierten Parametern nutzt. Danach wurde ein Ansatz basierend auf bestärkendem Lernen untersucht, welcher den ersten Ansatz als Referenzbewegung benutzt. Beide Ansätze wurden sowohl auf der entwickelten Roboterplatform, als auch auf weiteren simulierten Robotern getestet um die Generalisierbarkeit zu zeigen. Ein besonderer Fokus wurde auf die Fähigkeit des Gehens gelegt, aber auch Aufsteh- und Schussfähigkeiten werden diskutiert

Generation of whole-body motion for humanoid robots with the complete dynamics

Cette thèse propose une solution au problème de la génération de mouvements pour les robots humanoïdes. Le cadre qui est proposé dans cette thèse génère des mouvements corps-complet en utilisant la dynamique inverse avec l'espace des tâches et en satisfaisant toutes les contraintes de contact. La spécification des mouvements se fait à travers objectifs dans l'espace des tâches et la grande redondance du système est gérée avec une pile de tâches où les tâches moins prioritaires sont atteintes seulement si elles n'interfèrent pas avec celles de plus haute priorité. À cette fin, un QP hiérarchique est utilisé, avec l'avantage d'être en mesure de préciser tâches d'égalité ou d'inégalité à tous les niveaux de la hiérarchie. La capacité de traiter plusieurs contacts non-coplanaires est montrée par des mouvements où le robot s'assoit sur une chaise et monte une échelle. Le cadre générique de génération de mouvements est ensuite appliqué à des études de cas à l'aide de HRP-2 et Romeo. Les mouvements complexes et similaires à l'humain sont obtenus en utilisant l'imitation du mouvement humain où le mouvement acquis passe par un processus cinématique et dynamique. Pour faire face à la nature instantanée de la dynamique inverse, un générateur de cycle de marche est utilisé comme entrée pour la pile de tâches qui effectue une correction locale de la position des pieds sur la base des points de contact permettant de marcher sur un terrain accidenté. La vision stéréo est également introduite pour aider dans le processus de marche. Pour une récupération rapide d'équilibre, le capture point est utilisé comme une tâche contrôlée dans une région désirée de l'espace. En outre, la génération de mouvements est présentée pour CHIMP, qui a besoin d'un traitement particulier.This thesis aims at providing a solution to the problem of motion generation for humanoid robots. The proposed framework generates whole-body motion using the complete robot dynamics in the task space satisfying contact constraints. This approach is known as operational-space inverse-dynamics control. The specification of the movements is done through objectives in the task space, and the high redundancy of the system is handled with a prioritized stack of tasks where lower priority tasks are only achieved if they do not interfere with higher priority ones. To this end, a hierarchical quadratic program is used, with the advantage of being able to specify tasks as equalities or inequalities at any level of the hierarchy. Motions where the robot sits down in an armchair and climbs a ladder show the capability to handle multiple non-coplanar contacts. The generic motion generation framework is then applied to some case studies using HRP-2 and Romeo. Complex and human-like movements are achieved using human motion imitation where the acquired motion passes through a kinematic and then dynamic retargeting processes. To deal with the instantaneous nature of inverse dynamics, a walking pattern generator is used as an input for the stack of tasks which makes a local correction of the feet position based on the contact points allowing to walk on non-planar surfaces. Visual feedback is also introduced to aid in the walking process. Alternatively, for a fast balance recovery, the capture point is introduced in the framework as a task and it is controlled within a desired region of space. Also, motion generation is presented for CHIMP which is a robot that needs a particular treatment

Towards energy-efficient limit-cycle walking in biped service robots: design analysis, modeling and experimental study of biped robot actuated by linear motors

Researchers have been studying biped robots for many years, and, while many advances in the field have been accomplished, there still remain the challenge to transfer the existing solutions into real applications. The main issues are related to mobility and autonomy. In mobility, biped robots have evolved greatly, nevertheless they are still far from what a human can do in the work-site. Similarly, autonomy of biped platforms has been tackled on several different grounds, but its core problem still remains, and it is associated to energy issues. Because of these energy issues, lately the main attention has been redirected to the long term autonomy of the biped robotics platforms. For that, much effort has been made to develop new more energy-efficient biped robots. The GIMBiped project in Aalto University was established to tackle the previous issues in energy efficiency and mobility, through the study and implementation of dynamic and energy-efficient bipedal robotic waking. This thesis falls into the first studies needed to achieve the previous goal using the GIMBiped testbed, starting with a detailed analysis of the nonlinear dynamics of the target system, using a modeling and simulation tools. This work also presents an assessment of different control techniques based on Limit Cycle walking, carried out on a two-dimensional kneed bipedal simulator. Furthermore, a numerical continuation analysis of the mechanical parameters of the first GIMBiped prototype was performed, using the same approximated planar kneed biped model. This study is done to analyze the effect that such variations in the mechanical design parameters produce in the stability and energy-efficiency of the system.Finally, experiments were performed in the GIMBiped testbed. These experiments show the results of a hybrid control technique proposed by the author, which combines traditional ZMP-based walking approach with a Limit Cycle trajectory-following control. Furthermore the results of a pure ZMP-based type of control are also presented.

TOWARDS THE GROUNDING OF ABSTRACT CATEGORIES IN COGNITIVE ROBOTS

The grounding of language in humanoid robots is a fundamental problem, especially in social scenarios which involve the interaction of robots with human beings. Indeed, natural language represents the most natural interface for humans to interact and exchange information about concrete entities like KNIFE, HAMMER and abstract concepts such as MAKE, USE. This research domain is very important not only for the advances that it can produce in the design of human-robot communication systems, but also for the implication that it can have on cognitive science. Abstract words are used in daily conversations among people to describe events and situations that occur in the environment. Many scholars have suggested that the distinction between concrete and abstract words is a continuum according to which all entities can be varied in their level of abstractness. The work presented herein aimed to ground abstract concepts, similarly to concrete ones, in perception and action systems. This permitted to investigate how different behavioural and cognitive capabilities can be integrated in a humanoid robot in order to bootstrap the development of higher-order skills such as the acquisition of abstract words. To this end, three neuro-robotics models were implemented. The first neuro-robotics experiment consisted in training a humanoid robot to perform a set of motor primitives (e.g. PUSH, PULL, etc.) that hierarchically combined led to the acquisition of higher-order words (e.g. ACCEPT, REJECT). The implementation of this model, based on a feed-forward artificial neural networks, permitted the assessment of the training methodology adopted for the grounding of language in humanoid robots. In the second experiment, the architecture used for carrying out the first study was reimplemented employing recurrent artificial neural networks that enabled the temporal specification of the action primitives to be executed by the robot. This permitted to increase the combinations of actions that can be taught to the robot for the generation of more complex movements. For the third experiment, a model based on recurrent neural networks that integrated multi-modal inputs (i.e. language, vision and proprioception) was implemented for the grounding of abstract action words (e.g. USE, MAKE). Abstract representations of actions ("one-hot" encoding) used in the other two experiments, were replaced with the joints values recorded from the iCub robot sensors. Experimental results showed that motor primitives have different activation patterns according to the action's sequence in which they are embedded. Furthermore, the performed simulations suggested that the acquisition of concepts related to abstract action words requires the reactivation of similar internal representations activated during the acquisition of the basic concepts, directly grounded in perceptual and sensorimotor knowledge, contained in the hierarchical structure of the words used to ground the abstract action words.This study was financed by the EU project RobotDoC (235065) from the Seventh Framework Programme (FP7), Marie Curie Actions Initial Training Network