Natural ZMP trajectories for biped robot reference generation

The control of a biped humanoid is a challenging task due to the hard-to-stabilize dynamics. Walking reference trajectory generation is a key problem. Linear Inverted Pendulum Model (LIPM) and Zero Moment Point (ZMP) Criterion based approaches in stable walking reference generation are reported. In these methods, generally, the ZMP reference during a stepping motion is kept fixed in the middle of the supporting foot sole. This kind of reference generation lacks naturalness, in that, the ZMP in the human walk does not stay fixed, but it moves forward under the supporting foot. This paper proposes a reference generation algorithm based on the LIPM and moving support foot ZMP references. The application of Fourier series approximation simplifies the solution and it generates a smooth ZMP reference. A simple inverse kinematics based joint space controller is used for the tests of the developed reference trajectory through full-dynamics 3D simulation. A 12 DOF biped robot model is used in the simulations. Simulation studies suggest that the moving ZMP references are more energy efficient than the ones with fixed ZMP under the supporting foot. The results are promising for implementations

Bipedal humanoid robot walking reference tuning by the use of evolutionary algorithms

Various aspects of humanoid robotics attracted the attention of researchers in the past four decades. One of the most challenging tasks in this area is the control of bipedal locomotion. The dynamics involved are highly nonlinear and hard to stabilize. A typical fullbody humanoid robot has more than twenty joints and the coupling effects between the links are significant. Reference generation plays a vital role for the success of the walking controller. Stability criteria including the Zero Moment Point (ZMP) criterion are extensively applied for this purpose. However, the stability criteria are usually applied on simplified models like the Linear Inverted Pendulum Model (LIPM) which only partially describes the equations of the motion of the robot. There are also trial and error based techniques and other ad-hoc reference generation techniques as well. This background of complicated dynamics and difficulties in reference generation makes automatic gait (step patterns of legged robots) tuning an interesting area of research. A natural command for a legged robot is the velocity of its locomotion. A number of walk parameters including temporal and spatial variables like stepping period and step size need to be set properly in order to obtain the desired speed. These problems, when considered from kinematics point of view, do not have a unique set of walking parameters as a solution. However, some of the solutions can be more suitable for a stable walk, whereas others may lead to instability and cause robot to fall. This thesis proposes a gait tuning method based on evolutionary methods. A velocity command is given as the input to the system. A ZMP based reference generation method is employed. Walking simulations are performed to assess the fitness of artificial populations. The fitness is measured by the amount of support the simulated bipedal robot received from torsional virtual springs and dampers opposing the changes in body orientation. Cross-over and mutation mechanisms generate new populations. A number of different walking parameters and fitness functions are tested to improve this tuning process. The walking parameters obtained in simulations are applied to the experimental humanoid platform SURALP (Sabanci University ReseArch Labaratory Platform). Experiments verify the merits of the proposed reference tuning method

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

Dynamic Bipedal Locomotion: From Hybrid Zero Dynamics to Control Lyapunov Functions via Experimentally Realizable Methods

Robotic bipedal locomotion has become a rapidly growing field of research as humans increasingly look to augment their natural environments with intelligent machines. In order for these robotic systems to navigate the often unstructured environments of the world and perform tasks, they must first have the capability to dynamically, reliably, and efficiently locomote. Due to the inherently hybrid and underactuated nature of dynamic bipedal walking, the greatest experimental successes in the field have often been achieved by considering all aspects of the problem; with explicit consideration of the interplay between modeling, trajectory planning, and feedback control. The methodology and developments presented in this thesis begin with the modeling and design of dynamic walking gaits on bipedal robots through hybrid zero dynamics (HZD), a mathematical framework that utilizes hybrid system models coupled with nonlinear controllers that results in stable locomotion. This will form the first half of the thesis, and will be used to develop a solid foundation of HZD trajectory optimization tools and algorithms for efficient synthesis of accurate hybrid motion plans for locomotion on two underactuated and compliant 3D bipeds. While HZD and the associated trajectory optimization are an existing framework, the resulting behaviors shown in these preliminary experiments will extend the limits of what HZD has demonstrated is possible thus far in the literature. Specifically, the core results of this thesis demonstrate the first experimental multi-contact humanoid walking with HZD on the DURUS robot and then through the first compliant HZD motion library for walking over a continuum of walking speeds on the Cassie robot. On the theoretical front, a novel formulation of an optimization-based control framework is introduced that couples convergence constraints from control Lyapunov functions (CLF)s with desirable formulations existing in other areas of the bipedal locomotion field that have proven successful in practice, such as inverse dynamics control and quadratic programming approaches. The theoretical analysis and experimental validation of this controller thus forms the second half of this thesis. First, a theoretical analysis is developed which demonstrates several useful properties of the approach for tuning and implementation, and the stability of the controller for HZD locomotion is proven. This is then extended to a relaxed version of the CLF controller, which removes a convergence inequality constraint in lieu of a conservative CLF cost within a quadratic program to achieve tracking. It is then explored how this new CLF formulation can fully leverage the planned HZD walking gaits to achieve the target performance on physical hardware. Towards this goal, an experimental implementation of the CLF controller is derived for the Cassie robot, with the resulting experiments demonstrating the first successful realization of a CLF controller for a 3D biped on hardware in the literature. The accuracy of the robot model and synthesized HZD motion library allow the real-time control implementation to regularize the CLF optimization cost about the nominal walking gait. This drives the controller to choose smooth input torques and anticipated spring torques, as well as regulate an optimal distribution of feasible ground reaction forces on hardware while reliably tracking the planned virtual constraints. These final results demonstrate how each component of this thesis were brought together to form an effective end-to-end implementation of a nonlinear control framework for underactuated locomotion on a bipedal robot through modeling, trajectory optimization, and then ultimately real-time control.</p

Dynamic humanoid locomotion: Hybrid zero dynamics based gait optimization via direct collocation methods

Hybrid zero dynamics (HZD) has emerged as a popular framework for dynamic and underactuated bipedal walking, but has significant implementation difficulties when applied to the high degrees of freedom present in humanoid robots. The primary impediment is the process of gait design–it is difficult for optimizers to converge on a viable set of virtual constraints defining a gait. This dissertation presents a methodology that allows for the fast and reliable generation of efficient multi-domain robotic walking gaits through the framework of HZD, even in the presence of underactuation. To achieve this goal, we unify methods from trajectory optimization with the control framework of multi-domain hybrid zero dynamics. We present a novel optimization formulation in the context of direct collocation methods and HZD where we rigorously generate analytic Jacobians for the constraints. Two collocation methods, local collocation and pseudospectral (global) collocation, are developed within an unified framework, and their performance in different circumstances is comparatively studied. As a result, solving the resulting nonlinear program becomes tractable for large-scale NLP solvers, even for systems as high-dimensional as humanoid robots. We experimentally validate our methodology on the spring-legged prototype humanoid, DURUS, showing that the optimization approach yields dynamic and stable walking gaits for different walking configurations, including unrestricted 3D dynamic walking.Ph.D

Model-Based Optimization for the Analysis of Human Movement and the Design of Rehabilitation Devices

Human motions result from a complex and well-coordinated interaction between the body segments. Walking and the sit-to-stand transfer are amongst the most challenging human motion in terms of coordination and internal loads, respectively. We propose model-based nonlinear optimal control methods to reconstruct and synthesize these motions while considering the dynamics of the motion over the whole time horizon. The redundant and highly nonlinear character of the computed motions encourages to discretize the optimization problem according to direct multiple-shooting methods. The goal is to identify principles which enable us to describe the patterns of these motions. We approach human walking from the perspective of unimpaired subjects and subjects walking with unilateral transfemoral prostheses. Their walking motion is reconstructed from motion capture data using subject-specific threedimensional multibody models. The motion of the models is fitted to the recorded data for a whole stride in a least-squares sense in multi-stage optimal control problems. Analyzing the reconstructed motion for the individual foot placement of the subjects suggests that it relates with the Capturability concept: foot locations are chosen by the subjects which enable a balance between the inherently conflicting goals of effortless progression and quick response to perturbations. In addition, the modulation of the ground collision impact forces at heel strike is found to play a major role in the step-by-step stability strategy. Based on these findings, we propose Capturability as a complementary criterion to the established clinical stability assessment methods. The sit-to-stand motion is particularly demanding for humans with mobility impairments, due to the high joint loads required to lift the body into the standing pose. We synthesize optimal sit-to-stand by solving two-stage optimal control problems. We presume that the sit-to-stand motion is substantially characterized by a preparation phase prior to the actual lift-off. Full body models are established with dynamic model parameters which specifically represent elderly humans from different levels of mobility. For impaired subjects, mobility support is assumed to be provided by generic support actions. The optimization computations result in different patterns which include significant arm motion in both phases. Therefore, the results support our approach to choose a full body representation of the human as well as to consider two stages in the optimal control problem. The computation of optimal assisted sit-to-stand motions of impaired humans offers the opportunity to optimize design parameters for mobility assistance devices providing adequate support. Based on the support actions for the sit-to-stand motions computed for two different levels of impairment, optimal mechanical design parameters for two different sit-to-stand assistance devices are generated. Our approach to separate the human-device interaction at their interface ensures that the optimal support provided to the human by the device is not compromised by any dynamic coupling between them. Solving large-scale nonlinear optimal control problems with multiple stages, we obtain design parameters for the devices which are optimal in terms of the workspace and the mechanical effort required

Realizing Torque Controllers for Underactuated Bipedal Walking Using the Ideal Model Resolved Motion Method

This thesis presents an application of hybrid zero dynamics to realize underactuated bipedal walking on DURUS, a testbed designed and built by SRI International. The main contribution of this work is the ideal model resolved motion method (IMRMM), which is a simple method to convert ideal torque controllers to PD controllers to implement on hardware. Walking was first achieved using the proven method of the hybrid zero dynamics (HZD) reconstruction, followed by the Input-Output Feedback Linearization (IO) and Rapidly Exponentially Stabilizing Control Lyapunov Function Quadratic Programs (CLF-QPs) torque controllers implemented via IMRMM. The simulation and experimental results are presented and compared, and the best resulting specific cost of electrical transport on hardware was computed as 0.63 for the CLF-QP IM-RMM controller, and the record for walking was achieved on a separate occasion with the same CLF-QP IM-RMM controller, which yielded walking for 2 hours and 53 minutes, covering 7 km

Investigation of energy efficiency of hexapod robot locomotion

Disertacijoje nagrinėjamos vaikščiojančių robotų energijos sąnaudų problemos jiems judant lygiu ir nelygiu paviršiumi. Pagrindinis tyrimo objektas yra vaikščiojančio roboto valdymo, aplinkos atpažinimo bei kliūčių išvengimo žinomoje aplinkoje metodas. Energijos sąnaudų minimizavimas leistų praplėsti vaikščiojančių robotų pritaikymą ir veikimo laiką. Pagrindinis darbo tikslas – sukurti energijos sąnaudų minimizavimo metodus vaikščiojantiems robotams ir sukurti aplinkos atpažinimo ir klasifikavimo metodus bei ištirti šešiakojo roboto energijos sąnaudas jiems judant žinomoje aplinkoje. Šie metodai gali būti taikomi vaikščiojantiems daugiakojams robotams. Darbe sprendžiami šie uždaviniai: šešiakojo roboto eisenos parinkimas atsižvelgiant į energijos sąnaudas, paviršiaus kliūčių aptikimo ir perlipimo metodų sudarymas ir jų efektyvumo palyginimas. Taip pat sprendžiami uždaviniai, kurie siejasi su pėdų trajektorijos generavimo metodo kūrimu bei kliūčių dydžio ir tankio įtaka roboto energijos sąnaudoms. Disertaciją sudaro įvadas, trys skyriai, bendrosios išvados, naudotos literatūros ir autoriaus publikacijų disertacijos tema sąrašai. Įvade aptariama tiriamoji problema, darbo aktualumas, aprašomas tyrimų objektas, formuluojamas darbo tikslas bei uždaviniai, aprašoma tyrimų metodika, darbo mokslinis naujumas, darbo rezultatų praktinė reikšmė, ginamieji teiginiai. Įvado pabaigoje pristatomos disertacijos tema autoriaus paskelbtos publikacijos ir pranešimai konferencijose bei disertacijos struktūra. Pirmasis skyrius skirtas literatūros apžvalgai. Jame pateikta mobiliųjų robotų energetinio efektyvumo bei energijos sąnaudų matavimo, skaičiavimo ir optimizavimo metodų analizė. Antrajame skyriuje pateiktas energetiškai efektyvaus judėjimo metodikos sudarymas vaikščiojantiems robotams. Šiame skyriuje pateiktas šešiakojo roboto matematinio ir fizinio modelių sudarymas, nelygaus paviršiaus klasifikavimo modelio sudarymas bei taktilinio kliūčių aptikimo bei perlipimo metodų sudarymas. Skyriaus gale pateikiamos išvados. Trečiajame skyriuje tiriamos energijos sąnaudų priklausomybės nuo roboto eisenos bei judėjimo parametrų, kliūčių aptikimo ir perlipimo tikslumas priklausomai nuo kliūčių skaičiaus roboto kelyje, taip pat kliūčių dydžio ir tankio įtaka roboto energijos sąnaudoms. Disertacijos tema paskelbti 9 straipsniai: keturi – Clarivate Analytics Web of Science duomenų bazės leidiniuose, turinčiuose citavimo rodiklį, trys – Clarivate Analytics Web of Science duomenų bazės „Conference Proceedings“ leidiniuose ir du – kituose recenzuojamuose mokslo leidiniuose. Disertacijos tema perskaityti 7 pranešimai konferencijose Lietuvoje bei kitose šalyse

Towards Robust Bipedal Locomotion:From Simple Models To Full-Body Compliance

Thanks to better actuator technologies and control algorithms, humanoid robots to date can perform a wide range of locomotion activities outside lab environments. These robots face various control challenges like high dimensionality, contact switches during locomotion and a floating-base nature which makes them fall all the time. A rich set of sensory inputs and a high-bandwidth actuation are often needed to ensure fast and effective reactions to unforeseen conditions, e.g., terrain variations, external pushes, slippages, unknown payloads, etc. State of the art technologies today seem to provide such valuable hardware components. However, regarding software, there is plenty of room for improvement. Locomotion planning and control problems are often treated separately in conventional humanoid control algorithms. The control challenges mentioned above are probably the main reason for such separation. Here, planning refers to the process of finding consistent open-loop trajectories, which may take arbitrarily long computations off-line. Control, on the other hand, should be done very fast online to ensure stability. In this thesis, we want to link planning and control problems again and enable for online trajectory modification in a meaningful way. First, we propose a new way of describing robot geometries like molecules which breaks the complexity of conventional models. We use this technique and derive a planning algorithm that is fast enough to be used online for multi-contact motion planning. Similarly, we derive 3LP, a simplified linear three-mass model for bipedal walking, which offers orders of magnitude faster computations than full mechanical models. Next, we focus more on walking and use the 3LP model to formulate online control algorithms based on the foot-stepping strategy. The method is based on model predictive control, however, we also propose a faster controller with time-projection that demonstrates a close performance without numerical optimizations. We also deploy an efficient implementation of inverse dynamics together with advanced sensor fusion and actuator control algorithms to ensure a precise and compliant tracking of the simplified 3LP trajectories. Extensive simulations and hardware experiments on COMAN robot demonstrate effectiveness and strengths of our method. This thesis goes beyond humanoid walking applications. We further use the developed modeling tools to analyze and understand principles of human locomotion. Our 3LP model can describe the exchange of energy between human limbs in walking to some extent. We use this property to propose a metabolic-cost model of human walking which successfully describes trends in various conditions. The intrinsic power of the 3LP model to generate walking gaits in all these conditions makes it a handy solution for walking control and gait analysis, despite being yet a simplified model. To fill the reality gap, finally, we propose a kinematic conversion method that takes 3LP trajectories as input and generates more human-like postures. Using this method, the 3LP model, and the time-projecting controller, we introduce a graphical user interface in the end to simulate periodic and transient human-like walking conditions. We hope to use this combination in future to produce faster and more human-like walking gaits, possibly with more capable humanoid robots