Dynamic Walking: Toward Agile and Efficient Bipedal Robots

Dynamic walking on bipedal robots has evolved from an idea in science fiction to a practical reality. This is due to continued progress in three key areas: a mathematical understanding of locomotion, the computational ability to encode this mathematics through optimization, and the hardware capable of realizing this understanding in practice. In this context, this review article outlines the end-to-end process of methods which have proven effective in the literature for achieving dynamic walking on bipedal robots. We begin by introducing mathematical models of locomotion, from reduced order models that capture essential walking behaviors to hybrid dynamical systems that encode the full order continuous dynamics along with discrete footstrike dynamics. These models form the basis for gait generation via (nonlinear) optimization problems. Finally, models and their generated gaits merge in the context of real-time control, wherein walking behaviors are translated to hardware. The concepts presented are illustrated throughout in simulation, and experimental instantiation on multiple walking platforms are highlighted to demonstrate the ability to realize dynamic walking on bipedal robots that is agile and efficient

Reachability Map for Diverse and Energy Efficient Stepping of Humanoids

In legged locomotion, the relationship between different gait behaviors and energy consumption must consider the full-body dynamics and the robot control as a whole, which cannot be captured by simple models. This work studies the totality of robot dynamics and whole-body optimal control as a coupled system to investigate energy consumption during balance recovery. We developed a two-phase nonlinear optimization pipeline for dynamic stepping, which generates reachability maps showing complex energy-stepping relations. We optimize gait parameters to search all reachable locations and quantify the energy cost during dynamic transitions, which allows studying the relationship between energy consumption and stepping locations given different initial conditions. We found that to achieve efficient actuation, the stepping location and timing can have simple approximations close to the underlying optimality, resulting in optimal step positions with a 10.9% lower energy cost than those generated by linear inverted pendulum model. Despite the complexity of this nonlinear process, we found that near-minimal effort stepping locations are within a region of attractions, rather than a narrow solution space suggested by a simple model. This provides new insights into the nonuniqueness of near-optimal solutions in robot motion planning and control, and the diversity of stepping behavior in humans

Superando la brecha de la realidad: Algoritmos de aprendizaje por imitación y por refuerzos para problemas de locomoción robótica bípeda

ilustraciones, diagramas, fotografíasEsta tesis presenta una estrategia de entrenamiento de robots que utiliza técnicas de aprendizaje artificial para optimizar el rendimiento de los robots en tareas complejas. Motivado por los impresionantes logros recientes en el aprendizaje automático, especialmente en juegos y escenarios virtuales, el proyecto tiene como objetivo explorar el potencial de estas técnicas para mejorar las capacidades de los robots más allá de la programación humana tradicional a pesar de las limitaciones impuestas por la brecha de la realidad. El caso de estudio seleccionado para esta investigación es la locomoción bípeda, ya que permite dilucidar los principales desafíos y ventajas de utilizar métodos de aprendizaje artificial para el aprendizaje de robots. La tesis identifica cuatro desafíos principales en este contexto: la variabilidad de los resultados obtenidos de los algoritmos de aprendizaje artificial, el alto costo y riesgo asociado con la realización de experimentos en robots reales, la brecha entre la simulación y el comportamiento del mundo real, y la necesidad de adaptar los patrones de movimiento humanos a los sistemas robóticos. La propuesta consiste en tres módulos principales para abordar estos desafíos: Enfoques de Control No Lineal, Aprendizaje por Imitación y Aprendizaje por Reforzamiento. El módulo de Enfoques de Control No Lineal establece una base al modelar robots y emplear técnicas de control bien establecidas. El módulo de Aprendizaje por Imitación utiliza la imitación para generar políticas iniciales basadas en datos de captura de movimiento de referencia o resultados preliminares de políticas para crear patrones de marcha similares a los humanos y factibles. El módulo de Aprendizaje por Refuerzos complementa el proceso mejorando de manera iterativa las políticas paramétricas, principalmente a través de la simulación pero con el rendimiento en el mundo real como objetivo final. Esta tesis enfatiza la modularidad del enfoque, permitiendo la implementación de los módulos individuales por separado o su combinación para determinar la estrategia más efectiva para diferentes escenarios de entrenamiento de robots. Al utilizar una combinación de técnicas de control establecidas, aprendizaje por imitación y aprendizaje por refuerzos, la estrategia de entrenamiento propuesta busca desbloquear el potencial para que los robots alcancen un rendimiento optimizado en tareas complejas, contribuyendo al avance de la inteligencia artificial en la robótica no solo en sistemas virtuales sino en sistemas reales.The thesis introduces a comprehensive robot training framework that utilizes artificial learning techniques to optimize robot performance in complex tasks. Motivated by recent impressive achievements in machine learning, particularly in games and virtual scenarios, the project aims to explore the potential of these techniques for improving robot capabilities beyond traditional human programming. The case study selected for this investigation is bipedal locomotion, as it allows for elucidating key challenges and advantages of using artificial learning methods for robot learning. The thesis identifies four primary challenges in this context: the variability of results obtained from artificial learning algorithms, the high cost and risk associated with conducting experiments on real robots, the reality gap between simulation and real-world behavior, and the need to adapt human motion patterns to robotic systems. The proposed approach consists of three main modules to address these challenges: Non-linear Control Approaches, Imitation Learning, and Reinforcement Learning. The Non-linear Control module establishes a foundation by modeling robots and employing well-established control techniques. The Imitation Learning module utilizes imitation to generate initial policies based on reference motion capture data or preliminary policy results to create feasible human-like gait patterns. The Reinforcement Learning module complements the process by iteratively improving parametric policies, primarily through simulation but ultimately with real-world performance as the ultimate goal. The thesis emphasizes the modularity of the approach, allowing for the implementation of individual modules separately or their combination to determine the most effective strategy for different robot training scenarios. By employing a combination of established control techniques, imitation learning, and reinforcement learning, the framework seeks to unlock the potential for robots to achieve optimized performances in complex tasks, contributing to the advancement of artificial intelligence in robotics.DoctoradoDoctor en ingeniería mecánica y mecatrónic

Quasi optimal sagittal gait of a biped robot with a new structure of knee joint

The design of humanoid robots has been a tricky challenge for several years. Due to the kinematic complexity of human joints, their movements are notoriously difficult to be reproduced by a mechanism. The human knees allow movements including rolling and sliding, and therefore the design of new bioinspired knees is of utmost importance for the reproduction of anthropomorphic walking in the sagittal plane. In this article, the kinematic characteristics of knees were analyzed and a mechanical solution for reproducing them is proposed. The geometrical, kinematic and dynamic models are built together with an impact model for a biped robot with the new knee kinematic. The walking gait is studied as a problem of parametric optimization under constraints. The trajectories of walking are approximated by mathematical functions for a gait composed of single support phases with impacts. Energy criteria allow comparing the robot provided with the new rolling knee mechanism and a robot equipped with revolute knee joints. The results of the optimizations show that the rolling knee brings a decrease of the sthenic criterion. The comparisons of torques are also observed to show the difference of energy distribution between the actuators. For the same actuator selection, these results prove that the robot with rolling knees can walk longer than the robot with revolute joint knees.ANR R2A

Impact-Aware Multi-Contact Balance Criteria

Intentionally applying impacts while maintaining balance is challenging for legged robots. This study originated from observing experimental data of the humanoid robot HRP-4 intentionally hitting a wall with its right arm while standing on two feet. Strangely, violating the usual zero moment point balance criteria did not systematically result in a fall. To investigate this phenomenon, we propose the zero-step capture region for non-coplanar contacts, defined as the center of mass (CoM) velocity area, and validated it with push-recovery experiments employing the HRP-4 balancing on two non-coplanar contacts. To further enable on-purpose impacts, we compute the set of candidate post-impact CoM velocities accounting for frictional-impact dynamics in three dimensions, and restrict the entire set within the CoM velocity area to maintain balance with the sustained contacts during and after impacts. We illustrate the maximum contact velocity for various HRP-4 stances in simulation, indicating potential for integration into other task-space whole-body controllers or planners. This study is the first to address the challenging problem of applying an intentional impact with a kinematic-controlled humanoid robot on non-coplanar contacts

Optimization of energy efficiency of walking bipedal robots by use of elastic couplings in the form of mechanical springs

This paper presents a method to optimize the en- ergy efficiency of walking bipedal robots by more than 50 % in a speed range from 0.3 to 2.3 m/s using elastic couplings – mechanical springs with movement speed independent pa- rameters. The considered robot consists of a trunk, two stiff legs and two actuators in the hip joints. It is modeled as un- deractuated system to make use of its natural dynamics and feedback controlled with input-output linearization. A nu- merical optimization of the joint angle trajectories as well as the elastic couplings is performed to minimize the average energy expenditure over the whole speed range. The elastic couplings increase the swing leg motion’s natural frequency thus making smaller steps more efficient which reduce the impact loss at the touchdown of the swing leg. The pro- cess of energy turnover is investigated for the robot with and without elastic couplings. Furthermore, the influence of the elastic couplings’ topology, its degree of nonlinearity, the mass distribution, the joint friction, the coefficient of static friction and the selected actuator is analyzed. It is shown that the optimization of the robot’s motion and elastic coupling towards energy efficiency leads to a slightly slower conver- gence rate of the controller, yet no loss of stability and a lower sensitivity with respect to disturbances. The optimal elastic coupling discovered by the numerical optimization is a linear torsion spring between the legs

Methods to improve the coping capacities of whole-body controllers for humanoid robots

Current applications for humanoid robotics require autonomy in an environment specifically adapted to humans, and safe coexistence with people. Whole-body control is promising in this sense, having shown to successfully achieve locomotion and manipulation tasks. However, robustness remains an issue: whole-body controllers can still hardly cope with unexpected disturbances, with changes in working conditions, or with performing a variety of tasks, without human intervention. In this thesis, we explore how whole-body control approaches can be designed to address these issues. Based on whole-body control, contributions have been developed along three main axes: joint limit avoidance, automatic parameter tuning, and generalizing whole-body motions achieved by a controller. We first establish a whole-body torque-controller for the iCub, based on the stack-of-tasks approach and proposed feedback control laws in SE(3). From there, we develop a novel, theoretically guaranteed joint limit avoidance technique for torque-control, through a parametrization of the feasible joint space. This technique allows the robot to remain compliant, while resisting external perturbations that push joints closer to their limits, as demonstrated with experiments in simulation and with the real robot. Then, we focus on the issue of automatically tuning parameters of the controller, in order to improve its behavior across different situations. We show that our approach for learning task priorities, combining domain randomization and carefully selected fitness functions, allows the successful transfer of results between platforms subjected to different working conditions. Following these results, we then propose a controller which allows for generic, complex whole-body motions through real-time teleoperation. This approach is notably verified on the robot to follow generic movements of the teleoperator while in double support, as well as to follow the teleoperator\u2019s upper-body movements while walking with footsteps adapted from the teleoperator\u2019s footsteps. The approaches proposed in this thesis therefore improve the capability of whole-body controllers to cope with external disturbances, different working conditions and generic whole-body motions

Model-Based Optimization for the Analysis of Human Movement and the Design of Rehabilitation Devices

Human motions result from a complex and well-coordinated interaction between the body segments. Walking and the sit-to-stand transfer are amongst the most challenging human motion in terms of coordination and internal loads, respectively. We propose model-based nonlinear optimal control methods to reconstruct and synthesize these motions while considering the dynamics of the motion over the whole time horizon. The redundant and highly nonlinear character of the computed motions encourages to discretize the optimization problem according to direct multiple-shooting methods. The goal is to identify principles which enable us to describe the patterns of these motions. We approach human walking from the perspective of unimpaired subjects and subjects walking with unilateral transfemoral prostheses. Their walking motion is reconstructed from motion capture data using subject-specific threedimensional multibody models. The motion of the models is fitted to the recorded data for a whole stride in a least-squares sense in multi-stage optimal control problems. Analyzing the reconstructed motion for the individual foot placement of the subjects suggests that it relates with the Capturability concept: foot locations are chosen by the subjects which enable a balance between the inherently conflicting goals of effortless progression and quick response to perturbations. In addition, the modulation of the ground collision impact forces at heel strike is found to play a major role in the step-by-step stability strategy. Based on these findings, we propose Capturability as a complementary criterion to the established clinical stability assessment methods. The sit-to-stand motion is particularly demanding for humans with mobility impairments, due to the high joint loads required to lift the body into the standing pose. We synthesize optimal sit-to-stand by solving two-stage optimal control problems. We presume that the sit-to-stand motion is substantially characterized by a preparation phase prior to the actual lift-off. Full body models are established with dynamic model parameters which specifically represent elderly humans from different levels of mobility. For impaired subjects, mobility support is assumed to be provided by generic support actions. The optimization computations result in different patterns which include significant arm motion in both phases. Therefore, the results support our approach to choose a full body representation of the human as well as to consider two stages in the optimal control problem. The computation of optimal assisted sit-to-stand motions of impaired humans offers the opportunity to optimize design parameters for mobility assistance devices providing adequate support. Based on the support actions for the sit-to-stand motions computed for two different levels of impairment, optimal mechanical design parameters for two different sit-to-stand assistance devices are generated. Our approach to separate the human-device interaction at their interface ensures that the optimal support provided to the human by the device is not compromised by any dynamic coupling between them. Solving large-scale nonlinear optimal control problems with multiple stages, we obtain design parameters for the devices which are optimal in terms of the workspace and the mechanical effort required