The Spherical Inverted Pendulum with Pelvis Width in Polar Coordinates for Humanoid Walking Design

The present communication is a follow up and extension of the paper “The Spherical Inverted Pendulum: Exact Solutions of Gait and Foot Placement Estimation Based on Symbolic Computation” by the same author. The walk design is approached by a 3-D inverted pendulum in a polar coordinate system. The advantage of this model is to easily offer indications of the energy expenditure of an efficient walk. However, the disadvantages that were never recognized by authors previously using this model is that the COG trajectory has to pass through the supporting foot location. This causes an unnecessary and unrealistic waving in the frontal plane during gait. The problem is discussed here and solved by extending the model of the inverted pendulum by introducing the pelvis width and the distance between the hips of the two legs, without adding dynamical complexity

Dynamic Balance and Gait Metrics for Robotic Bipeds

For legged robots to be useful in the real world, they must be able to balance and walk reliably. Both of these abilities improve when a system is more effective at moving itself around relative to its contacts (i.e., its feet). Achieving this type of movement depends both on the controller used to perform the motion and the physical properties of the system. Although much work has been done on the development of dynamic controllers for balance and gait, only limited research exists on how to quantify a system’s physical balance capabilities or how to modify the system to improve those capabilities. From the control perspective, there are three strategies for maintaining balance in bipeds: flexing, leaning, and stepping. Both stepping and leaning strategies typically depend on balance points (critical points used for maintaining or regaining balance) to determine whether or not a step is needed, and if so, where to step. Although several balance point estimators exist, the majority of these methods make undesirable assumptions (e.g., ignoring the impact dynamics, assuming massless legs, planar motion, etc.). From the physical design perspective, one promising approach for analyzing system performance is a set of dynamic ratios called velocity and momentum gains, which are dependent only on the (scale-invariant) dynamic parameters and instantaneous configuration of a system, enabling entire classes of mechanisms to be analyzed at the same time. This thesis makes four key contributions towards improving biped balancing capabilities. First, a dynamic bipedal controller is proposed which uses a 3D balance point estimator both to respond to disturbances and produce reliable stepping. Second, a novel balance point estimator is proposed that facilitates stepping while combining and expanding the features of existing 2D and 3D estimators to produce a generalized 3D formulation. Third, the momentum gain formulation is extended to general 2D and 3D systems, then both gains are compared to centroidal momentum via a spatial formulation and incorporated into a generalized gain definition. Finally, the gains are used as a metric in an optimization framework to design parameterized balancing mechanisms within a given configuration space. Effectively, this enables an optimization of how well a system could balance without the need to pre-specify or co-generate controllers and/or trajectories. To validate the control contributions, simulated bipeds are subjected to external disturbances while standing still and walking. For the gain contributions, the framework is used to compare gain-optimized mechanisms to those based on the cost of transport metric. Through the combination of gain-based physical design optimization and the use of predictive, real-time balance point estimators within dynamic controllers, bipeds and other legged systems will soon be able to achieve reliable balance and gait in the real world

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is not guaranteed, and unexpected hazardous motions can occur. This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots that smoothly transfer to reality, providing only instantaneous proprioceptive observations. By combining original termination conditions and policy smoothness conditioning, we achieve stable learning, sim-to-real transfer and safety using a policy without memory nor explicit history. Reward engineering is then used to give insights into how to keep balance. We demonstrate its performance in reality on the lower-limb medical exoskeleton Atalante

A Stability-Estimator to Unify Humanoid Locomotion: Walking, Stair-Climbing and Ladder-Climbing

The field of Humanoid robotics research has often struggled to find a unique niche that is not better served by other forms of robot. Unlike more traditional industrials robots with a specific purpose, a humanoid robot is not necessarily optimized for any particular task, due to the complexity and balance issues of being bipedal. However, the versatility of a humanoid robot may be ideal for applications such as search and rescue. Disaster sites with chemical, biological, or radiation contamination mean that human rescue workers may face untenable risk. Using a humanoid robot in these dangerous circumstances could make emergency response faster and save human lives. Despite the many successes of existing mobile robots in search and rescue, stair and ladder climbing remains a challenging task due to their form. To execute ladder climbing motions effectively, a humanoid robot requires a reliable estimate of stability. Traditional methods such as Zero Moment Point are not applicable to vertical climbing, and do not account for force limits imposed on end-effectors. This dissertation implements a simple contact wrench space method using a linear combination of contact wrenches. Experiments in simulation showed ZMP equivalence on flat ground. Furthermore, the estimator was able to predict stability with four point contact on a vertical ladder. Finally, an extension of the presented method is proposed based on these findings to address the limitations of the linear combination.Ph.D., Mechanical Engineering and Mechanics -- Drexel University, 201

A Foot Placement Strategy for Robust Bipedal Gait Control

This thesis introduces a new measure of balance for bipedal robotics called the foot placement estimator (FPE). To develop this measure, stability first is defined for a simple biped. A proof of the stability of a simple biped in a controls sense is shown to exist using classical methods for nonlinear systems. With the addition of a contact model, an analytical solution is provided to define the bounds of the region of stability. This provides the basis for the FPE which estimates where the biped must step in order to be stable. By using the FPE in combination with a state machine, complete gait cycles are created without any precalculated trajectories. This includes gait initiation and termination. The bipedal model is then advanced to include more realistic mechanical and environmental models and the FPE approach is verified in a dynamic simulation. From these results, a 5-link, point-foot robot is designed and constructed to provide the final validation that the FPE can be used to provide closed-loop gait control. In addition, this approach is shown to demonstrate significant robustness to external disturbances. Finally, the FPE is shown in experimental results to be an unprecedented estimate of where humans place their feet for walking and jumping, and for stepping in response to an external disturbance

Humanoid Robot Soccer Locomotion and Kick Dynamics: Open Loop Walking, Kicking and Morphing into Special Motions on the Nao Robot

Striker speed and accuracy in the RoboCup (SPL) international robot soccer league is becoming increasingly important as the level of play rises. Competition around the ball is now decided in a matter of seconds. Therefore, eliminating any wasted actions or motions is crucial when attempting to kick the ball. It is common to see a discontinuity between walking and kicking where a robot will return to an initial pose in preparation for the kick action. In this thesis we explore the removal of this behaviour by developing a transition gait that morphs the walk directly into the kick back swing pose. The solution presented here is targeted towards the use of the Aldebaran walk for the Nao robot. The solution we develop involves the design of a central pattern generator to allow for controlled steps with realtime accuracy, and a phase locked loop method to synchronise with the Aldebaran walk so that precise step length control can be activated when required. An open loop trajectory mapping approach is taken to the walk that is stabilized statically through the use of a phase varying joint holding torque technique. We also examine the basic princples of open loop walking, focussing on the commonly overlooked frontal plane motion. The act of kicking itself is explored both analytically and empirically, and solutions are provided that are versatile and powerful. Included as an appendix, the broader matter of striker behaviour (process of goal scoring) is reviewed and we present a velocity control algorithm that is very accurate and efficient in terms of speed of execution

Learning-based methods for planning and control of humanoid robots

Nowadays, humans and robots are more and more likely to coexist as time goes by. The anthropomorphic nature of humanoid robots facilitates physical human-robot interaction, and makes social human-robot interaction more natural. Moreover, it makes humanoids ideal candidates for many applications related to tasks and environments designed for humans. No matter the application, an ubiquitous requirement for the humanoid is to possess proper locomotion skills. Despite long-lasting research, humanoid locomotion is still far from being a trivial task. A common approach to address humanoid locomotion consists in decomposing its complexity by means of a model-based hierarchical control architecture. To cope with computational constraints, simplified models for the humanoid are employed in some of the architectural layers. At the same time, the redundancy of the humanoid with respect to the locomotion task as well as the closeness of such a task to human locomotion suggest a data-driven approach to learn it directly from experience. This thesis investigates the application of learning-based techniques to planning and control of humanoid locomotion. In particular, both deep reinforcement learning and deep supervised learning are considered to address humanoid locomotion tasks in a crescendo of complexity. First, we employ deep reinforcement learning to study the spontaneous emergence of balancing and push recovery strategies for the humanoid, which represent essential prerequisites for more complex locomotion tasks. Then, by making use of motion capture data collected from human subjects, we employ deep supervised learning to shape the robot walking trajectories towards an improved human-likeness. The proposed approaches are validated on real and simulated humanoid robots. Specifically, on two versions of the iCub humanoid: iCub v2.7 and iCub v3