Development of a Locomotion and Balancing Strategy for Humanoid Robots

The locomotion ability and high mobility are the most distinguished features of humanoid robots. Due to the non-linear dynamics of walking, developing and controlling the locomotion of humanoid robots is a challenging task. In this thesis, we study and develop a walking engine for the humanoid robot, NAO, which is the official robotic platform used in the RoboCup Spl. Aldebaran Robotics, the manufacturing company of NAO provides a walking module that has disadvantages, such as being a black box that does not provide control of the gait as well as the robot walk with a bent knee. The latter disadvantage, makes the gait unnatural, energy inefficient and exert large amounts of torque to the knee joint. Thus creating a walking engine that produces a quality and natural gait is essential for humanoid robots in general and is a factor for succeeding in RoboCup competition. Humanoids robots are required to walk fast to be practical for various life tasks. However, its complex structure makes it prone to falling during fast locomotion. On the same hand, the robots are expected to work in constantly changing environments alongside humans and robots, which increase the chance of collisions. Several human-inspired recovery strategies have been studied and adopted to humanoid robots in order to face unexpected and avoidable perturbations. These strategies include hip, ankle, and stepping, however, the use of the arms as a recovery strategy did not enjoy as much attention. The arms can be employed in different motions for fall prevention. The arm rotation strategy can be employed to control the angular momentum of the body and help to regain balance. In this master\u27s thesis, I developed a detailed study of different ways in which the arms can be used to enhance the balance recovery of the NAO humanoid robot while stationary and during locomotion. I model the robot as a linear inverted pendulum plus a flywheel to account for the angular momentum change at the CoM. I considered the role of the arms in changing the body\u27s moment of inertia which help to prevent the robot from falling or to decrease the falling impact. I propose a control algorithm that integrates the arm rotation strategy with the on-board sensors of the NAO. Additionally, I present a simple method to control the amount of recovery from rotating the arms. I also discuss the limitation of the strategy and how it can have a negative impact if it was misused. I present simulations to evaluate the approach in keeping the robot stable against various disturbance sources. The results show the success of the approach in keeping the NAO stable against various perturbations. Finally,I adopt the arm rotation to stabilize the ball kick, which is a common reason for falling in the soccer humanoid RoboCup competitions

Fall Prediction and Controlled Fall for Humanoid Robots

Humanoids which resemble humans in their body structure and degrees of freedom are anticipated to work like them within infrastructures and environments constructed for humans. In such scenarios, even humans who have exceptional manipulation, balancing, and locomotion skills are vulnerable to fall, humanoids being their approximate imitators are no exception to this. Furthermore, their high center of gravity position in relation to their small support polygon makes them more prone to fall, unlike other robots such as quadrupeds. The consequences of these falls are so devastating that it can instantly annihilate both the robot and its surroundings. This has become one of the major stumbling blocks which humanoids have to overcome to operate in real environments. As a result, in this thesis, we have strived to address the imminent fall over of humanoids by developing different control techniques. The fall over problem as such can be divided into three subissues: fall prediction, controlled fall, and its recovery. In the presented work, the first two issues have been addressed, and they are presented in three parts. First, we define what is fall over for humanoids, different sources for it to happen, the effect fall over has both on the robot and to its surroundings, and how to deal with them. Following which, we give a brief introduction to the overall system which includes both the hardware and software components which have been used throughout the work for varied purposes. Second, the first sub-issue is addressed by proposing a generic method to predict the falling over of humanoid robots in a reliable, robust, and agile manner across various terrains, and also amidst arbitrary disturbances. The aforementioned characteristics are strived to attain by proposing a prediction principle inspired by the human balance sensory systems. Accordingly, the fusion of multiple sensors such as inertial measurement unit and gyroscope (IMU), foot pressure sensor (FPS), joint encoders, and stereo vision sensor, which are equivalent to the human\u2019s vestibular, proprioception, and vision systems are considered. We first define a set of feature-based fall indicator variables (FIVs) from the different sensors, and the thresholds for those FIVs are extracted analytically for four major disturbance scenarios. Further, an online threshold interpolation technique and an impulse adaptive counter limit are proposed to manage more generic disturbances. For the generalized prediction process, both the instantaneous and cumulative sum of each FIVs are normalized, and a suitable value is set as the critical limit to predict the fall over. To determine the best combination and the usefulness of multiple sensors, the prediction performance is evaluated on four different types of terrains, in three unique combinations: first, each feature individually with their respective FIVs; second, an intuitive performance based (PF); and finally, Kalman filter based (KF) techniques, which involve the usage of multiple features. For PF and KF techniques, prediction performance evaluations are carried out with and without adding noise. Overall, it is reported that KF performs better than PF and individual sensor features under different conditions. Also, the method\u2019s ability to predict fall overs during the robot\u2019s simple dynamic motion is also tested and verified through simulations. Experimental verification of the proposed prediction method on flat and uneven terrains was carried out with the WALK-MAN humanoid robot. Finally, in reference to the second sub-issue, i.e., the controlled fall, we propose two novel fall control techniques based on energy concepts, which can be applied online to mitigate the impact forces incurred during the falling over of humanoids. Both the techniques are inspired by the break-fall motions, in particular, Ukemi motion practiced by martial arts people. The first technique reduces the total energy using a nonlinear control tool, called energy shaping (ES) and further distributes the reduced energy over multiple contacts by means of energy distribution polygons (EDP). We also include an effective orientation control to safeguard the end-effectors in the event of ground impacts. The performance of the proposed method is numerically evaluated by dynamic simulations under the sudden falling over scenario of the humanoid robot for both lateral and sagittal falls. The effectiveness of the proposed ES and EDP concepts are verified by diverse comparative simulations regarding total energy, distribution, and impact forces. Following the first technique, we proposed another controller to generate an online rolling over motion based on the hypothesis that multi-contact motions can reduce the impact forces even further. To generate efficient rolling motion, critical parameters are defined by the insights drawn from a study on rolling, which are contact positions and attack angles. In addition, energy-injection velocity is proposed as an auxiliary rolling parameter to ensure sequential multiple contacts in rolling. An online rolling controller is synthesized to compute the optimal values of the rolling parameters. The first two parameters are to construct a polyhedron, by selecting suitable contacts around the humanoid\u2019s body. This polyhedron distributes the energy gradually across multiple contacts, thus called energy distribution polyhedron. The last parameter is to inject some additional energy into the system during the fall, to overcome energy drought and tip over successive contacts. The proposed controller, incorporated with energy injection, minimization, and distribution techniques result in a rolling like motion and significantly reduces the impact forces, and it is verified in numerical experiments with a segmented planar robot and a full humanoid model

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

From walking to running: robust and 3D humanoid gait generation via MPC

Humanoid robots are platforms that can succeed in tasks conceived for humans. From locomotion in unstructured environments, to driving cars, or working in industrial plants, these robots have a potential that is yet to be disclosed in systematic every-day-life applications. Such a perspective, however, is opposed by the need of solving complex engineering problems under the hardware and software point of view. In this thesis, we focus on the software side of the problem, and in particular on locomotion control. The operativity of a legged humanoid is subordinate to its capability of realizing a reliable locomotion. In many settings, perturbations may undermine the balance and make the robot fall. Moreover, complex and dynamic motions might be required by the context, as for instance it could be needed to start running or climbing stairs to achieve a certain location in the shortest time. We present gait generation schemes based on Model Predictive Control (MPC) that tackle both the problem of robustness and tridimensional dynamic motions. The proposed control schemes adopt the typical paradigm of centroidal MPC for reference motion generation, enforcing dynamic balance through the Zero Moment Point condition, plus a whole-body controller that maps the generated trajectories to joint commands. Each of the described predictive controllers also feature a so-called stability constraint, preventing the generation of diverging Center of Mass trajectories with respect to the Zero Moment Point. Robustness is addressed by modeling the humanoid as a Linear Inverted Pendulum and devising two types of strategies. For persistent perturbations, a way to use a disturbance observer and a technique for constraint tightening (to ensure robust constraint satisfaction) are presented. In the case of impulsive pushes instead, techniques for footstep and timing adaptation are introduced. The underlying approach is to interpret robustness as a MPC feasibility problem, thus aiming at ensuring the existence of a solution for the constrained optimization problem to be solved at each iteration in spite of the perturbations. This perspective allows to devise simple solutions to complex problems, favoring a reliable real-time implementation. For the tridimensional locomotion, on the other hand, the humanoid is modeled as a Variable Height Inverted Pendulum. Based on it, a two stage MPC is introduced with particular emphasis on the implementation of the stability constraint. The overall result is a gait generation scheme that allows the robot to overcome relatively complex environments constituted by a non-flat terrain, with also the capability of realizing running gaits. The proposed methods are validated in different settings: from conceptual simulations in Matlab to validations in the DART dynamic environment, up to experimental tests on the NAO and the OP3 platforms

Learning Control Policies for Fall Prevention and Safety in Bipedal Locomotion

The ability to recover from an unexpected external perturbation is a fundamental motor skill in bipedal locomotion. An effective response includes the ability to not just recover balance and maintain stability but also to fall in a safe manner when balance recovery is physically infeasible. For robots associated with bipedal locomotion, such as humanoid robots and assistive robotic devices that aid humans in walking, designing controllers which can provide this stability and safety can prevent damage to robots or prevent injury related medical costs. This is a challenging task because it involves generating highly dynamic motion for a high-dimensional, non-linear and under-actuated system with contacts. Despite prior advancements in using model-based and optimization methods, challenges such as requirement of extensive domain knowledge, relatively large computational time and limited robustness to changes in dynamics still make this an open problem. In this thesis, to address these issues we develop learning-based algorithms capable of synthesizing push recovery control policies for two different kinds of robots : Humanoid robots and assistive robotic devices that assist in bipedal locomotion. Our work can be branched into two closely related directions : 1) Learning safe falling and fall prevention strategies for humanoid robots and 2) Learning fall prevention strategies for humans using a robotic assistive devices. To achieve this, we introduce a set of Deep Reinforcement Learning (DRL) algorithms to learn control policies that improve safety while using these robots. To enable efficient learning, we present techniques to incorporate abstract dynamical models, curriculum learning and a novel method of building a graph of policies into the learning framework. We also propose an approach to create virtual human walking agents which exhibit similar gait characteristics to real-world human subjects, using which, we learn an assistive device controller to help virtual human return to steady state walking after an external push is applied. Finally, we extend our work on assistive devices and address the challenge of transferring a push-recovery policy to different individuals. As walking and recovery characteristics differ significantly between individuals, exoskeleton policies have to be fine-tuned for each person which is a tedious, time consuming and potentially unsafe process. We propose to solve this by posing it as a transfer learning problem, where a policy trained for one individual can adapt to another without fine tuning.Ph.D

Using a Combination of PID Control and Kalman Filter to Design of IoT-based Telepresence Self-balancing Robots during COVID-19 Pandemic

COVID-19 is a very dangerous respiratory disease that can spread quickly through the air. Doctors, nurses, and medical personnel need protective clothing and are very careful in treating COVID-19 patients to avoid getting infected with the COVID-19 virus. Hence, a medical telepresence robot, which resembles a humanoid robot, is necessary to treat COVID-19 patients. The proposed self-balancing COVID-19 medical telepresence robot is a medical robot that handles COVID-19 patients, which resembles a stand-alone humanoid soccer robot with two wheels that can maneuver freely in hospital hallways. The proposed robot design has some control problems; it requires steady body positioning and is subjected to disturbance. A control method that functions to find the stability value such that the system response can reach the set-point is required to control the robot's stability and repel disturbances; this is known as disturbance rejection control. This study aimed to control the robot using a combination of Proportional-Integral-Derivative (PID) control and a Kalman filter. Mathematical equations were required to obtain a model of the robot's characteristics. The state-space model was derived from the self-balancing robot's mathematical equation. Since a PID control technique was used to keep the robot balanced, this state-space model was converted into a transfer function model. The second Ziegler-Nichols's rule oscillation method was used to tune the PID parameters. The values of the amplifier constants obtained were Kp=31.002, Ki=5.167, and Kd=125.992128. The robot was designed to be able to maintain its balance for more than one hour by using constant tuning, even when an external disturbance is applied to it. Doi: 10.28991/esj-2021-SP1-016 Full Text: PD

Footstep Adjustment for Biped Push Recovery on Slippery Surfaces

Despite extensive studies on motion stabilization of bipeds, they still suffer from the lack of disturbance coping capability on slippery surfaces. In this paper, a novel controller for stabilizing a bipedal motion in its sagittal plane is developed with regard to the surface friction limitations. By taking into account the physical limitation of the surface in the stabilization trend, a more advanced level of reliability is achieved that provides higher functionalities such as push recovery on low-friction surfaces and prevents the stabilizer from overreacting. The discrete event-based strategy consists of modifying the step length and time period at the beginning of each footstep in order to reestablish stability necessary conditions while taking into account the surface friction limitation as a constraint to prevent slippage. Adjusting footsteps to prevent slippage in confronting external disturbances is perceived as a novel strategy for keeping stability, quite similar to human reaction. The developed methodology consists of rough closed-form solutions utilizing elementary math operations for obtaining the control inputs, allowing to reach a balance between convergence and computational cost, which is quite suitable for real-time operations even with modest computational hardware. Several numerical simulations, including push recovery and switching between different gates on low-friction surfaces, are performed to demonstrate the effectiveness of the proposed controller. In correlation with human-gait experience, the results also reveal some physical aspects favoring stability and the fact of switching between gaits to reduce the risk of falling in confronting different conditions.Comment: for associated simulation video, see https://youtu.be/BWzUgHGdl3

Locomotion and balance control of humanoid robots with dynamic and kinematic constraints

Building a robot capable of servicing and assisting people is one of the ultimate goals in humanoid robotics. To realize this goal, a humanoid robot first needs to be able to perform some fundamental locomotion tasks, such as balancing and walking. However, simply performing such basic tasks in static, open environments is insufficient for a robot to be useful. A humanoid robot should also possess the ability to make use of the object in the environment to generate dynamic motions and improve its mobility. Also, since humanoid robots are expected to work and live closely with humans, having human-like motions is important for them to be human-friendly. This dissertation addresses my work on endowing humanoid robots with the ability to handle dynamic and kinematic constraints while performing the basic tasks in order to achieve more complex locomotion tasks. First, as a representative case of handling dynamic constraints, a biped humanoid robot is required to balance and walk on a cylinder that rolls freely on the ground. This task is difficult even for humans. I introduce a control method for a humanoid robot to execute this challenging task. In order for the robot to be able to actively control cylinder's motion, the dynamics of the cylinder has been taken into account together with the dynamics of the robot in deriving the control method. Its effectiveness has been verified by full-body dynamics simulation and hardware experiments on the Sarcos humanoid robot. Second, as an example of tasks with kinematic constraints, I present a method for real-time control of humanoid robots to track human motions while maintaining balance. It consists of a standard proportional-derivative tracking controller that computes the desired acceleration to track the given reference motion and an optimizer that computes the optimal joint torques and contact forces to realize the desired acceleration, considering the full-body dynamics of the robot and strict constraints on contact forces. By taking advantage of the property that the joint torques do not contribute to the six degrees of freedom of the floating base, I decouple the computation of joint torques and contact forces such that the optimization problem with strict contact force constraints can be solved in real time. In full-body simulation, a humanoid robot is able to imitate various human motions by using this method. Through this work, I demonstrate that considering dynamic and kinematic constraints in the environment in the design of controllers enables humanoid robots to achieve more complex locomotion tasks, such as manipulating a dynamic object or tracking given reference motions, while maintaining balance.Doctor of Philosoph

Predictive Whole-Body Control of Humanoid Robot Locomotion

Humanoid robots are machines built with an anthropomorphic shape. Despite decades of research into the subject, it is still challenging to tackle the robot locomotion problem from an algorithmic point of view. For example, these machines cannot achieve a constant forward body movement without exploiting contacts with the environment. The reactive forces resulting from the contacts are subject to strong limitations, complicating the design of control laws. As a consequence, the generation of humanoid motions requires to exploit fully the mathematical model of the robot in contact with the environment or to resort to approximations of it. This thesis investigates predictive and optimal control techniques for tackling humanoid robot motion tasks. They generate control input values from the system model and objectives, often transposed as cost function to minimize. In particular, this thesis tackles several aspects of the humanoid robot locomotion problem in a crescendo of complexity. First, we consider the single step push recovery problem. Namely, we aim at maintaining the upright posture with a single step after a strong external disturbance. Second, we generate and stabilize walking motions. In addition, we adopt predictive techniques to perform more dynamic motions, like large step-ups. The above-mentioned applications make use of different simplifications or assumptions to facilitate the tractability of the corresponding motion tasks. Moreover, they consider first the foot placements and only afterward how to maintain balance. We attempt to remove all these simplifications. We model the robot in contact with the environment explicitly, comparing different methods. In addition, we are able to obtain whole-body walking trajectories automatically by only specifying the desired motion velocity and a moving reference on the ground. We exploit the contacts with the walking surface to achieve these objectives while maintaining the robot balanced. Experiments are performed on real and simulated humanoid robots, like the Atlas and the iCub humanoid robots