Towards Bipedal Behavior on a Quadrupedal Platform Using Optimal Control

This paper explores the applicability of a Linear Quadratic Regulator (LQR) controller design to the problem of bipedal stance on the Minitaur [1] quadrupedal robot. Restricted to the sagittal plane, this behavior exposes a three degree of freedom (DOF) double inverted pendulum with extensible length that can be projected onto the familiar underactuated revolute-revolute “Acrobot” model by assuming a locked prismatic DOF, and a pinned toe. While previous work has documented the successful use of local LQR control to stabilize a physical Acrobot, simulations reveal that a design very similar to those discussed in the past literature cannot achieve an empirically viable controller for our physical plant. Experiments with a series of increasingly close physical facsimiles leading to the actual Minitaur platform itself corroborate and underscore the physical Minitaur platform corroborate and underscore the implications of the simulation study. We conclude that local LQR-based linearized controller designs are too fragile to stabilize the physical Minitaur platform around its vertically erect equilibrium and end with a brief assessment of a variety of more sophisticated nonlinear control approaches whose pursuit is now in progress

Transfer of support in a dynamic walking robot

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 1995.Includes bibliographical references (p. 39-40).by David William Bailey.M.S

Sequential Motion Planning for Bipedal Somersault via Flywheel SLIP and Momentum Transmission with Task Space Control

In this paper, we present a sequential motion planning and control method for generating somersaults on bipedal robots. The somersault (backflip or frontflip) is considered as a coupling between an axile hopping motion and a rotational motion about the center of mass of the robot; these are encoded by a hopping Spring-loaded Inverted Pendulum (SLIP) model and the rotation of a Flywheel, respectively. We thus present the Flywheel SLIP model for generating the desired motion on the ground phase. In the flight phase, we present a momentum transmission method to adjust the orientation of the lower body based on the conservation of the centroidal momentum. The generated motion plans are realized on the full-dimensional robot via momentum-included task space control. Finally, the proposed method is implemented on a modified version of the bipedal robot Cassie in simulation wherein multiple somersault motions are generated

Motion Planning and Control of Dynamic Humanoid Locomotion

Inspired by human, humanoid robots has the potential to become a general-purpose platform that lives along with human. Due to the technological advances in many field, such as actuation, sensing, control and intelligence, it finally enables humanoid robots to possess human comparable capabilities. However, humanoid locomotion is still a challenging research field. The large number of degree of freedom structure makes the system difficult to coordinate online. The presence of various contact constraints and the hybrid nature of locomotion tasks make the planning a harder problem to solve. Template model anchoring approach has been adopted to bridge the gap between simple model behavior and the whole-body motion of humanoid robot. Control policies are first developed for simple template models like Linear Inverted Pendulum Model (LIPM) or Spring Loaded Inverted Pendulum(SLIP), the result controlled behaviors are then been mapped to the whole-body motion of humanoid robot through optimization-based task-space control strategies. Whole-body humanoid control framework has been verified on various contact situations such as unknown uneven terrain, multi-contact scenarios and moving platform and shows its generality and versatility. For walking motion, existing Model Predictive Control approach based on LIPM has been extended to enable the robot to walk without any reference foot placement anchoring. It is kind of discrete version of \u201cwalking without thinking\u201d. As a result, the robot could achieve versatile locomotion modes such as automatic foot placement with single reference velocity command, reactive stepping under large external disturbances, guided walking with small constant external pushing forces, robust walking on unknown uneven terrain, reactive stepping in place when blocked by external barrier. As an extension of this proposed framework, also to increase the push recovery capability of the humanoid robot, two new configurations have been proposed to enable the robot to perform cross-step motions. For more dynamic hopping and running motion, SLIP model has been chosen as the template model. Different from traditional model-based analytical approach, a data-driven approach has been proposed to encode the dynamics of the this model. A deep neural network is trained offline with a large amount of simulation data based on the SLIP model to learn its dynamics. The trained network is applied online to generate reference foot placements for the humanoid robot. Simulations have been performed to evaluate the effectiveness of the proposed approach in generating bio-inspired and robust running motions. The method proposed based on 2D SLIP model can be generalized to 3D SLIP model and the extension has been briefly mentioned at the end

Towards understanding human locomotion

Die zentrale Frage, die in der vorliegenden Arbeit untersucht wurde, ist, wie man die komplizierte Dynamik des menschlichen Laufens besser verstehen kann. In der wissenschaftlichen Literatur werden zur Beschreibung von Laufbewegungen (Gehen und Rennen) oftmals minimalistische "Template"-Modelle verwendet. Diese sehr einfachen Modelle beschreiben nur einen ausgewählten Teil der Dynamik, meistens die Schwerpunktsbahn. In dieser Arbeit wird nun versucht, mittels Template-Modellen das Verständnis des Laufens voranzubringen. Die Analyse der Schwerpunktsbewegung durch Template-Modelle setzt eine präzise Bestimmung der Schwerpunktsbahn im Experiment voraus. Hierfür wird in Kapitel 2.3 eine neue Methode vorgestellt, welche besonders robust gegen die typischen Messfehler bei Laufexperimenten ist. Die am häfigsten verwendeten Template-Modelle sind das Masse-Feder-Modell und das inverse Pendel, welche zur Beschreibung der Körperschwerpunktsbewegung gedacht sind und das Drehmoment um den Schwerpunkt vernachlässigen. Zur Beschreibung der Stabilisierung der Körperhaltung (und damit der Drehimpulsbilanz) wird in Abschnitt 3.3 das Template-Modell "virtuelles Pendel" für das menschliche Gehen eingeführt und mit experimentellen Daten verglichen. Die Diskussion möglicher Realisierungsmechanismen legt dabei nahe, dass die Aufrichtung des menschlichen Gangs im Laufe der Evolution keine große mechanische Hürde war. In der Literatur wird oft versucht, Eigenschaften der Bewegung wie Stabilität durch Eigenschaften der Template-Modelle zu erklären. Dies wird in modifizierter Form auch in der vorliegen Arbeit getan. Hierzu wird zunächst eine experimentell bestimmte Schwerpunktsbewegung auf das Masse-Feder-Modell übertragen. Anschließend wird die Kontrollvorschrift der Schritt-zu-Schritt-Anpassung der Modellparameter identifiziert sowie eine geeignete Näherung angegeben, um die Stabilität des Modells, welches mit dieser Kontrollvorschrift ausgestattet wird, zu analysieren. Der Vergleich mit einer direkten Bestimmung der Stabilität aus einem Floquet-Modell zeigt qualitativ gute Übereinstimmung. Beide Ansätze führen auf das Ergebnis, dass beim langsamen menschlichen Rennen Störungen innerhalb von zwei Schritten weitgehend abgebaut werden. Zusammenfassend wurde gezeigt, wie Template-Modelle zum Verständnis der Laufbewegung beitragen können. Gerade die Identifikation der individuellen Kontrollvorschrift auf der Abstraktionsebene des Masse-Feder-Modells erlaubt zukünftig neue Wege, aktive Prothesen oder Orthesen in menschenähnlicher Weise zu steuern und ebnet den Weg, menschliches Rennen auf Roboter zu übertragen.Human locomotion is part of our everyday life, however the mechanisms are not well enough understood to be transferred into technical devices like orthoses, protheses or humanoid robots. In biomechanics often minimalistic so-called template models are used to describe locomotion. While these abstract models in principle offer a language to describe both human behavior and technical control input, the relation between human locomotion and locomotion of these templates was long unclear. This thesis focusses on how human locomotion can be described and analyzed using template models. Often, human running is described using the SLIP template. Here, it is shown that SLIP (possibly equipped with any controller) cannot show human-like disturbance reactions, and an appropriate extension of SLIP is proposed. Further, a new template to describe postural stabilization is proposed. Summarizing, this theses shows how simple template models can be used to enhance the understanding of human gait

Trajectory optimization inspired pneumatic locomotion on compliant terrains

Thesis (MEng)--Stellenbosch University, 2022.ENGLISH ABSTRACT: In order to achieve true autonomy, robots have to be able to handle complex and rough terrain generally found outside of the lab. Legged robotics has become the focal point in recent years, aided by the developments in trajectory optimization methods. However, a major problem in legged robotics is dealing with hybrid contacts with different terrain types. The difference in dynamics due to the interaction between the foot and the ground makes it increasingly difficult to design controllers that successfully execute on multiple surfaces. This work investigates trajectory optimization methods for a pneumatically actuated mono-pod on rigid and compliant terrain. Trajectory optimization was utilized to obtain trajectories for acceleration, steady-state and deceleration hopping on compliant terrain, as well as rigid terrain surfaces. For the compliant terrain trajectories a novel method was developed to model the specific characteristics of the compliant terrain. Trajectories were generated using this method for two different compliant terrain types, namely: rough gravel and fine gravel. To mimic the pneumatic actuation in the trajectory optimization problem, a simplified mathematical model was developed to accommodate the bang-bang force of the pneumatic actuator. This model used complementarity constraints and node bucketing techniques to mimic the behaviour of a real pneumatic actuator with damping and delay. Once these methods and models were implemented, trajectories were executed, in open-loop, on a fixed body robotic platform that was designed and built for this thesis. The executions were compared to the trajectory results. The rigid terrain trajectories were executed successfully on a hard surface, but failed on gravel surfaces. The compliant terrain trajectories executed successfully on gravel surfaces, indicating that the method developed to model compliant terrain is a more accurate representation of the gravel surfaces compared to the rigid terrain trajectories. After these results showed that the methods used to describe the compliant terrain proved to be accurate, a free body mono-pod robot and support rig was designed and built. The support rig limited the movement of the mono-pod to the sagittal plane to mimic the limitations of the trajectory optimization model. Acceleration, steady-state and deceleration trajectories were generated for the free body mono-pod on compliant terrain surfaces and a rigid terrain surface. From these trajectories a controller was designed with the main sources of feedback being the height of the robot and the angle of the free moving body of the robot. The free body mono-pod robot used the controller to execute hopping from rest back to rest with three steady-state hops in between. For each terrain type the controller was adjusted based on the generated trajectories. The results show successful execution of the trajectories on all terrain types using the controller. Lastly multi-surface hopping was executed on the mono-pod robot platform. The controller was adjusted to hop from a hard surface to a compliant surface and executed these trajectories successfully.AFRIKAANSE OPSOMMING: Om ware outonomie te bewerkstellig, moet robotte komplekse en rowwe terreine, wat gewoonlik buite die laboratorium voorkom, kan hanteer. Been aangedrewe robotika het die afgelope paar jaar ’n fokuspunt geword in die literatuur, aangehelp deur ontwikkelinge in trajek-optimaliseringsmetodes. ’n Groot probleem in been aangedrewe robotika is egter die hantering van verskillende terreintipes. Die verskil in dinamika as gevolg van die interaksie tussen die voet en die grond maak dit steeds moeiliker om beheerders te ontwerp wat suksesvol op verskeie oppervlaktes uitgevoer kan word. Hierdie werk ondersoek optimaliseringsmetodes van trajekte vir ’n pneumaties aange drewe eenbenige robot op stewige en sagte terrein. Trajek-optimalisering is gebruik om versnelling-, bestendige- en vertraging-trajekte te genereer wat op sagte terreine sowel as stewige terreinoppervlakke spring. Hierdie trajekte is uitgevoer op ’n robot platvorm met ’n vaste liggaam en is vergelyk met die trajek resultate. Die stewige terrein trajek is suksesvol uitgevoer op ’n harde oppervlak, maar het op ’n gruisoppervlak misluk. Die sagte terrein is suksesvol uitgevoer op twee gruisoppervlaktes, wat aandui dat dit ’n meer akkurate voorstelling van die gruisoppervlakke is in vergelyking met die rigiede terrein trajek. Om die pneumatiese kragte in die tajekoptimalisering na te boots, is ’n vereenvoudigde wiskundige model ontwikkel om die drukstootbeweging van die pneumatiese aandrywer te akkommodeer. Hierdie model het komplementariteitsbeperkings en knooppunte gebruik om die gedrag van ’n werklike pneumatiese aandrywer na te boots. Nadat hierdie resultate getoon het dat die metodes wat gebruik is om die terrein te beskryf akkuraat is, is ’n eenbenige robot en ’n ondersteuningsplatvorm vir ’n vrye liggaam robot ontwerp en gebou. Die ondersteuningsplatvorm het die beweging van die eenbenige robot tot die sagittale vlak beperk om die beperkings van die trajekoptimaliseringsmodel na te boots. Versnelling-, bestendige- en vertragings-trajekte is gegenereer vir die eenbenige robot met ’n vrye liggaam op sagte terrein en op stewige terrein. Uit hierdie trajekte is ’n beheerder ontwerp, met die belangrikste bronne van terugvoer die hoogte van die robot en die hoek van die vry bewegende liggaam van die robot. Die eenbenige robot het die beheerder gebruik om van rus na rus te spring, met drie bestendige hoppe tussenin. Vir elke terrein tipe is die beheerder aangepas op grond van die gegenereerde trajekte. Die resultate toon ’n suksesvolle uitvoering van die trajekte op alle terreintipes met behulp van die beheerder. Laastens is ’n twee-oppervlakte-hop uitgevoer op die mono-pod robotplatform. Die beheerder is aangepas om van ’n harde oppervlak na ’n sagte oppervlak te spring en hierdie trajekte was suksesvol uitgevoer.Master

3D turning analysis of a Bipedal Robot

Thesis (MEng)--Stellenbosch University, 2022.ENGLISH ABSTRACT: There is stark contrast between the abilities of legged locomotion found in nature, and locomotion found in lab environments. This performance gap is indicative of a large knowledge gap. Roboticists are required to bridge these gaps to truly invite robots to detach from their support rigs, and actuate within the real world. In this thesis, non-planar contact and discontinuous locomotive dynamics were modeled as a trajectory optimization problem. Consequently, this made understanding the complexities of legged locomotion more tractable. Understanding, and being able to leverage, contact is crucial to successful legged locomotion. Therefore, a comprehensive investigation was conducted into non-planar contact dynamics using a monopod robot. Here, methods of modeling the Coulomb friction cone in contact implicit trajectory optimization were implemented. Literature suggests replacing the friction cone with a polyhedral approximation thereof. However, this method is known to underestimate the resultant friction in non-planar environments. This thesis presents a novel method of modeling the 3D friction cone and compares it to an implementation of the polyhedral approximation. Results from this comparison show that the novel method was significantly more computationally efficient than the polyhedral approximation, without underestimating the friction cone. Dynamic bipedal locomotion remains a struggle for most robotic platforms. Robotics literature provides few examples of robots achieving agile, dynamic locomotion. Therefore, trajectories realizing non-planar dynamic bipedal motion were generated. Experiments were conducted into acceleration, steady-state, deceleration, and rapid turning off the sagittal plane. Optimal trajectories displayed the robot walking at speeds resulting in a Froude number less than 0.5, and running at speeds resulting in a higher Froude number. This is consistent with dynamic gaits found in nature. A sliding-mass velocity profile emerged when conducting long-time-horizon trajectories where the robot accelerated from a rest position and decelerated back to rest after completing multiple steps in a periodic steady-state gait. Additionally, when turning off the sagittal plane, slip occurred at least 93.32% of the duration of contact, and turn overshoot is present in all turn trajectories.AFRIKAANSE OPSOMMING: Daar is skerp kontras tussen die vermoëns van voortbeweging wat in die natuur voorkom, en di`e wat in laboratoriumomgewings voorkom. Hierdie prestasiegaping is aanduidend van ’n groot kennisgaping. Robotiste is verwag om hierdie gapings te oorbrug om robotte uit van hul ondersteuningplatforms uit te kom en in die werklike wˆereld te aandryf. In hierdie tesis word 3D kontak en diskontinue lokomotiefdinamika gemodelleer as ’n trajekoptime ringsprobleem. Gevolglik maak dit die verstandhouding van robotik gebeendebeweging makliker. Die verstaan van kontak, en hoe om dit te gebruik, is noodsaaklik vir suksesvolle voortbeweging. Daarom is ’n omvattende ondersoek uitgevoer, met behulp van ’n monopod robot, om kontakdinamika beter te verstaan. Hier word metodes van modellering van die Coulomb wrywings-keel in kontak-implisiete-trajek-optimering ge¨ımplementeer. Literatuur stel voor dat die wrywingskegel vervang word met ’n veelvlakkige benadering daarvan. Dit is bekend dat hierdie metode die gevolglike wrywing in 3D omgewings onderskat. Hierdie tesis bied ’n nuwe metode om die 3D-wrywingskegel te modelleer, en vergelyk dit met ’n implementering van die veelvlakkige benadering daarvan. Uitslae van hierdie vergelyking toon dat die nuwe metode meer berekeningsdoeltreffend was as die veelvlakkige benadering, sonder om die wrywingskegel te onderskat. Dinamiese tweevoetige voortbeweging bly ’n stryd vir die meeste robotplatforms. Robotikaliteratuur verskaf min voorbeelde van robotte wat dinamiese voortbeweging bereik. Daarom is trajekte gegenereer wat 3D dinamika van tweevoetbeweging realiseer. Eksperimente word uitgevoer na accelerasie, bestendige toestand, verlangsaming en draaie van die sagittale vlak af. Optimale trajekte het die robot laat stap teen ’n spoed wat ’n Froude-getal minder as 0.5 laat kom, en hardloop teen spoed wat ’n ho¨er Froude-getal gehad het. Dit stem ooreens met dinamiese voetvalpatrone wat in die natuur voorkom. ’n Glymassa-spoedprofiel het voor gekom toe die lang-tyd-horison trajekte uitgevoer word: waar die robot van ’n rusposisie versnel het en terug versnel is na rus nadat hy verskeie stappe in periodieke bestendige-toestand voltooi het. Wanneer die robot van die sagittale vlak afdraai, gly hy ten minste 93.32% van die tyd wat kontak plaasgevind, en beurtoorskiet is teenwoordig in alle draaie.Master

Kinematic arrangement optimization of a quadruped robot with genetic algorithms

Quadruped robots are capable of performing a multitude of tasks like walking, running carrying and jumping. As research on quadruped robots grows, so does the variety of the designs available. These designs are often inspired by nature and finalized around technical constraints that are different for each project. A load carrying robot design will take its inspiration from a mule, while a running robot will use a cheetah-like design. However, this technique might be too broad when approaching a designing process for a quadruped robot aimed to accomplish certain tasks with varying degrees of importance. In order to reach an efficient design with precise link lengths and joint positions, for some specific task at hand, a complex series of problems have to be solved. This thesis proposes to use genetic algorithms to handle the designing process. An approach that mimics the evolutionary process of living beings, genetic algorithms can be used to reach quadruped designs which are optimized for a given task. The task-specific nature of this process is expected to result in more efficient designs than simply mimicking 4 animal structures, since animals are evolved to be efficient in a bigger variety of tasks. To explore this, genetic algorithms are used to optimize the kinematic structure of quadruped robots designed for the tasks of vertical jumping and trotting. The robots are optimized for these two tasks separately and then together. Algorithm results are compared to a relatively more conventional quadruped design

Streamlined sim-to-real transfer for deep-reinforcement learning in robotics locomotion

Legged robots possess superior mobility compared to other machines, yet designing controllers for them can be challenging. Classic control methods require engineers to distill their knowledge into controllers, which is time-consuming and limiting when approaching dynamic tasks in unknown environments. Conversely, learning- based methods that gather knowledge from data can potentially unlock the versatility of legged systems. In this thesis, we propose a novel approach called CPG-Actor, which incor- porates feedback into a fully differentiable Central Pattern Generator (CPG) formulation using neural networks and Deep-Reinforcement Learning (RL). This approach achieves approximately twenty times better training performance compared to previous methods and provides insights into the impact of training on the distribution of parameters in both the CPGs and MLP feedback network. Adopting Deep-RL to design controllers comes at the expense of gathering extensive data, typically done in simulation to reduce time. However, controllers trained with data collected in simulation often lose performance when deployed in the real world, referred to as the sim-to-real gap. To address this, we propose a new method called Extended Random Force Injection (ERFI), which randomizes only two parameters to allow for sim-to-real transfer of locomotion controllers. ERFI demonstrated high robustness when varying masses of the base, or attaching a manipulator arm to the robot during testing, and achieved competitive performance comparable to standard randomization techniques. Furthermore, we propose a new method called Roll-Drop to enhance the robustness of Deep-RL policies to observation noise. Roll-Drop introduces dropout during rollout, achieving an 80% success rate when tested with up to 25% noise injected in the observations. Finally, we adopted model-free controllers to enable omni-directional bipedal lo- comotion on point feet with a quadruped robot without any hardware modification or external support. Despite the limitations posed by the quadruped’s hardware, the study considers this a perfect benchmark task to assess the shortcomings of sim- to-real techniques and unlock future avenues for the legged robotics community. Overall, this thesis demonstrates the potential of learning-based methods to design dynamic and robust controllers for legged robots while limiting the effort needed for sim-to-real transfer

Receding-horizon motion planning of quadrupedal robot locomotion

Quadrupedal robots are designed to offer efficient and robust mobility on uneven terrain. This thesis investigates combining numerical optimization and machine learning methods to achieve interpretable kinodynamic planning of natural and agile locomotion. The proposed algorithm, called Receding-Horizon Experience-Controlled Adaptive Legged Locomotion (RHECALL), uses nonlinear programming (NLP) with learned initialization to produce long-horizon, high-fidelity, terrain-aware, whole-body trajectories. RHECALL has been implemented and validated on the ANYbotics ANYmal B and C quadrupeds on complex terrain. The proposed optimal control problem formulation uses the single-rigid-body dynamics (SRBD) model and adopts a direct collocation transcription method which enables the discovery of aperiodic contact sequences. To generate reliable trajectories, we propose fast-to-compute analytical costs that leverage the discretization and terrain-dependent kinematic constraints. To extend the formulation to receding-horizon planning, we propose a segmentation approach with asynchronous centre of mass (COM) and end-effector timings and a heuristic initialization scheme which reuses the previous solution. We integrate real-time 2.5D perception data for online foothold selection. Additionally, we demonstrate that a learned stability criterion can be incorporated into the planning framework. To accelerate the convergence of the NLP solver to locally optimal solutions, we propose data-driven initialization schemes trained using supervised and unsupervised behaviour cloning. We demonstrate the computational advantage of the schemes and the ability to leverage latent space to reconstruct dynamic segments of plans which are several seconds long. Finally, in order to apply RHECALL to quadrupeds with significant leg inertias, we derive the more accurate lump leg single-rigid-body dynamics (LL-SRBD) and centroidal dynamics (CD) models and their first-order partial derivatives. To facilitate intuitive usage of costs, constraints and initializations, we parameterize these models by Euclidean-space variables. We show the models have the ability to shape rotational inertia of the robot which offers potential to further improve agility