Learning for Humanoid Multi-Contact Navigation Planning

Humanoids' abilities to navigate uneven terrain make them well-suited for disaster response efforts, but humanoid motion planning in unstructured environments remains a challenging problem. In this dissertation we focus on planning contact sequences for a humanoid robot navigating in large unstructured environments using multi-contact motion, including both foot and palm contacts. In particular, we address the two following questions: (1) How do we efficiently generate a feasible contact sequence? and (2) How do we efficiently generate contact sequences which lead to dynamically-robust motions? For the first question, we propose a library-based method that retrieves motion plans from a library constructed offline, and adapts them with local trajectory optimization to generate the full motion plan from the start to the goal. This approach outperforms a conventional graph search contact planner when it is difficult to decide which contact is preferable with a simplified robot model and local environment information. We also propose a learning approach to estimate the difficulty to traverse a certain region based on the environment features. By integrating the two approaches, we propose a planning framework that uses graph search planner to find contact sequences around easy regions. When it is necessary to go through a difficult region, the framework switches to use the library-based method around the region to find a feasible contact sequence faster. For the second question, we consider dynamic motions in contact planning. Most humanoid motion generators do not optimize the dynamic robustness of a contact sequence. By querying a learned model to predict the dynamic feasibility and robustness of each contact transition from a centroidal dynamics optimizer, the proposed planner efficiently finds contact sequences which lead to dynamically-robust motions. We also propose a learning-based footstep planner which takes external disturbances into account. The planner considers not only the poses of the planned contact sequence, but also alternative contacts near the planned contact sequence that can be used to recover from external disturbances. Neural networks are trained to efficiently predict multi-contact zero-step and one-step capturability, which allows the planner to generate contact sequences robust to external disturbances efficiently.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162908/1/linyuchi_1.pd

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

Predictive Whole-Body Control of Humanoid Robot Locomotion

Humanoid robots are machines built with an anthropomorphic shape. Despite decades of research into the subject, it is still challenging to tackle the robot locomotion problem from an algorithmic point of view. For example, these machines cannot achieve a constant forward body movement without exploiting contacts with the environment. The reactive forces resulting from the contacts are subject to strong limitations, complicating the design of control laws. As a consequence, the generation of humanoid motions requires to exploit fully the mathematical model of the robot in contact with the environment or to resort to approximations of it. This thesis investigates predictive and optimal control techniques for tackling humanoid robot motion tasks. They generate control input values from the system model and objectives, often transposed as cost function to minimize. In particular, this thesis tackles several aspects of the humanoid robot locomotion problem in a crescendo of complexity. First, we consider the single step push recovery problem. Namely, we aim at maintaining the upright posture with a single step after a strong external disturbance. Second, we generate and stabilize walking motions. In addition, we adopt predictive techniques to perform more dynamic motions, like large step-ups. The above-mentioned applications make use of different simplifications or assumptions to facilitate the tractability of the corresponding motion tasks. Moreover, they consider first the foot placements and only afterward how to maintain balance. We attempt to remove all these simplifications. We model the robot in contact with the environment explicitly, comparing different methods. In addition, we are able to obtain whole-body walking trajectories automatically by only specifying the desired motion velocity and a moving reference on the ground. We exploit the contacts with the walking surface to achieve these objectives while maintaining the robot balanced. Experiments are performed on real and simulated humanoid robots, like the Atlas and the iCub humanoid robots

Mobile Robots Navigation

Mobile robots navigation includes different interrelated activities: (i) perception, as obtaining and interpreting sensory information; (ii) exploration, as the strategy that guides the robot to select the next direction to go; (iii) mapping, involving the construction of a spatial representation by using the sensory information perceived; (iv) localization, as the strategy to estimate the robot position within the spatial map; (v) path planning, as the strategy to find a path towards a goal location being optimal or not; and (vi) path execution, where motor actions are determined and adapted to environmental changes. The book addresses those activities by integrating results from the research work of several authors all over the world. Research cases are documented in 32 chapters organized within 7 categories next described

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Generation of whole-body motion for humanoid robots with the complete dynamics

Cette thèse propose une solution au problème de la génération de mouvements pour les robots humanoïdes. Le cadre qui est proposé dans cette thèse génère des mouvements corps-complet en utilisant la dynamique inverse avec l'espace des tâches et en satisfaisant toutes les contraintes de contact. La spécification des mouvements se fait à travers objectifs dans l'espace des tâches et la grande redondance du système est gérée avec une pile de tâches où les tâches moins prioritaires sont atteintes seulement si elles n'interfèrent pas avec celles de plus haute priorité. À cette fin, un QP hiérarchique est utilisé, avec l'avantage d'être en mesure de préciser tâches d'égalité ou d'inégalité à tous les niveaux de la hiérarchie. La capacité de traiter plusieurs contacts non-coplanaires est montrée par des mouvements où le robot s'assoit sur une chaise et monte une échelle. Le cadre générique de génération de mouvements est ensuite appliqué à des études de cas à l'aide de HRP-2 et Romeo. Les mouvements complexes et similaires à l'humain sont obtenus en utilisant l'imitation du mouvement humain où le mouvement acquis passe par un processus cinématique et dynamique. Pour faire face à la nature instantanée de la dynamique inverse, un générateur de cycle de marche est utilisé comme entrée pour la pile de tâches qui effectue une correction locale de la position des pieds sur la base des points de contact permettant de marcher sur un terrain accidenté. La vision stéréo est également introduite pour aider dans le processus de marche. Pour une récupération rapide d'équilibre, le capture point est utilisé comme une tâche contrôlée dans une région désirée de l'espace. En outre, la génération de mouvements est présentée pour CHIMP, qui a besoin d'un traitement particulier.This thesis aims at providing a solution to the problem of motion generation for humanoid robots. The proposed framework generates whole-body motion using the complete robot dynamics in the task space satisfying contact constraints. This approach is known as operational-space inverse-dynamics control. The specification of the movements is done through objectives in the task space, and the high redundancy of the system is handled with a prioritized stack of tasks where lower priority tasks are only achieved if they do not interfere with higher priority ones. To this end, a hierarchical quadratic program is used, with the advantage of being able to specify tasks as equalities or inequalities at any level of the hierarchy. Motions where the robot sits down in an armchair and climbs a ladder show the capability to handle multiple non-coplanar contacts. The generic motion generation framework is then applied to some case studies using HRP-2 and Romeo. Complex and human-like movements are achieved using human motion imitation where the acquired motion passes through a kinematic and then dynamic retargeting processes. To deal with the instantaneous nature of inverse dynamics, a walking pattern generator is used as an input for the stack of tasks which makes a local correction of the feet position based on the contact points allowing to walk on non-planar surfaces. Visual feedback is also introduced to aid in the walking process. Alternatively, for a fast balance recovery, the capture point is introduced in the framework as a task and it is controlled within a desired region of space. Also, motion generation is presented for CHIMP which is a robot that needs a particular treatment

Use of gaze and gait analysis to assess the effect of footway environmental factors on older pedestrians' accessibility

Walking in the footway environment is an essential activity of daily living and the physical activity associated can also improve an individual’s quality of life. The possibility and ability for the individual to reach opportunities and participate in activities on foot indicate the accessibility of the footway environment. One of the major hazards in the footway environment that impedes the accessibility is falling. While it may happen to anyone, falling is more common in older people and the consequence of falling could seriously deteriorate their quality of life. Falls among older people is one of the major public health problems. Fall-induced injuries, either physical or psychological, can lead to further physical frailty and social isolation. Although falls study on the elderly has been widely discussed, there is little information available on risk factors in the outdoor footway environment. The current guidance of design and condition of footway environment, however, is short of scientific justification. The potential hazard of a single step-height, such as defects and kerbs on the pavement, is commonly encountered as well as negotiated by the individual under different lighting conditions. Therefore, a framework within which the interaction among sensory and physical capability and environmental factors could be well investigated is needed. This thesis investigates the process of planning for negotiating upcoming step-heights on the pavement, and aims at establishing changes in gaze (where people look) and gait (how people walk) behaviour in relation to combinations of environmental factors such as step-heights and lighting levels. About 17 young (aged 25-34) and 17 older (aged 65-74) participants walked on a straight walkway with 16 experimental conditions (8 step-heights at 2 lighting levels, one step-height in each environmental setting), and the visual and walking patterns were collected. The results demonstrated the inconspicuous descending step-heights (due to a lack of visibility) as well as the lower lighting level demanded additional visual attention and additional time to plan footsteps. Step-height of 60 mm was found to be the threshold of sensory as well as physical capability, which should be considered as suggested guideline for pavement design. Both 125 and 30 mm were perceived as VI more dangerous due in part to additional body function requirement and the nature of visual ambiguity respectively. More importantly, young participants demonstrated the agility and adaptability to different environmental setting whereas older people demonstrated more disturbed gait pattern and increased visual attention. Also, older people were more likely to face the risk of falling as they detected the step-heights later, had shorter time for response than young participants. Therefore in footway design, older people’s perception as well as reaction to response to the footway environment should be considered and the pavement should be designed to accommodate their needs. Bringing together the gaze and gait analysis is proved to be important in this thesis and it is essential for further research in understanding the cognitive process in between. The responses to risks of falling could not be understood without knowing the association between the visual perception and gait adjustments. The approach developed in this thesis might be used to further analyse microscopic as well as macroscopic scales of accessibility of the footway environment and provide some insights into how the individual’s sensory, cognitive and physical capability affect the decision making process while walking

Metastable legged-robot locomotion

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2008.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 195-215).A variety of impressive approaches to legged locomotion exist; however, the science of legged robotics is still far from demonstrating a solution which performs with a level of flexibility, reliability and careful foot placement that would enable practical locomotion on the variety of rough and intermittent terrain humans negotiate with ease on a regular basis. In this thesis, we strive toward this particular goal by developing a methodology for designing control algorithms for moving a legged robot across such terrain in a qualitatively satisfying manner, without falling down very often. We feel the definition of a meaningful metric for legged locomotion is a useful goal in and of itself. Specifically, the mean first-passage time (MFPT), also called the mean time to failure (MTTF), is an intuitively practical cost function to optimize for a legged robot, and we present the reader with a systematic, mathematical process for obtaining estimates of this MFPT metric. Of particular significance, our models of walking on stochastically rough terrain generally result in dynamics with a fast mixing time, where initial conditions are largely "forgotten" within 1 to 3 steps. Additionally, we can often find a near-optimal solution for motion planning using only a short time-horizon look-ahead. Although we openly recognize that there are important classes of optimization problems for which long-term planning is required to avoid "running into a dead end" (or off of a cliff!), we demonstrate that many classes of rough terrain can in fact be successfully negotiated with a surprisingly high level of long-term reliability by selecting the short-sighted motion with the greatest probability of success. The methods used throughout have direct relevance to machine learning, providing a physics-based approach to reduce state space dimensionality and mathematical tools to obtain a scalar metric quantifying performance of the resulting reduced-order system.by Katie Byl.Ph.D