An optimal planning of falling motions of a humanoid robot

Abstract — This paper studies an optimal planning of falling motions of a human-sized humanoid robot to reduce the damage of the robot. We developed a human-sized robot HRP-2FX which has a simplified humanoid robot shape with seven d.o.f. and can emulate motions in the sagittal plane of a humanoid robot. An optimal control is applied to generate the falling motion of HRP-2FX to minimize a performance index, and the optimality has been verified by the experiments on HRP-2FX. I

Learning a Unified Control Policy for Safe Falling

Being able to fall safely is a necessary motor skill for humanoids performing highly dynamic tasks, such as running and jumping. We propose a new method to learn a policy that minimizes the maximal impulse during the fall. The optimization solves for both a discrete contact planning problem and a continuous optimal control problem. Once trained, the policy can compute the optimal next contacting body part (e.g. left foot, right foot, or hands), contact location and timing, and the required joint actuation. We represent the policy as a mixture of actor-critic neural network, which consists of n control policies and the corresponding value functions. Each pair of actor-critic is associated with one of the n possible contacting body parts. During execution, the policy corresponding to the highest value function will be executed while the associated body part will be the next contact with the ground. With this mixture of actor-critic architecture, the discrete contact sequence planning is solved through the selection of the best critics while the continuous control problem is solved by the optimization of actors. We show that our policy can achieve comparable, sometimes even higher, rewards than a recursive search of the action space using dynamic programming, while enjoying 50 to 400 times of speed gain during online execution

QP-based Adaptive-Gains Compliance Control in Humanoid Falls

International audienceWe address the problem of humanoid falling with a decoupled strategy consisting of a pre-impact and a postimpact stage. In the pre-impact stage, geometrical reasoning allows the robot to choose appropriate impact points in the surrounding environment and to adopt a posture to reach them while avoiding impact-singularities and preparing for the postimpact. The surrounding environment can be unstructured and may contain cluttered obstacles. The post-impact stage uses a quadratic program controller that adapts on-line the joint proportional-derivative (PD) gains to make the robot compliant-to absorb impact and post-impact dynamics, which lowers possible damage risks. This is done by a new approach incorporating the stiffness and damping gains directly as decision variables in the QP along with the usually-considered variables of joint accelerations and contact forces. Constraints of the QP prevent the motors from reaching their torque limits during the fall. Several experiments on the humanoid robot HRP-4 in a full-dynamics simulator are presented and discussed

Learning Control Policies for Fall Prevention and Safety in Bipedal Locomotion

The ability to recover from an unexpected external perturbation is a fundamental motor skill in bipedal locomotion. An effective response includes the ability to not just recover balance and maintain stability but also to fall in a safe manner when balance recovery is physically infeasible. For robots associated with bipedal locomotion, such as humanoid robots and assistive robotic devices that aid humans in walking, designing controllers which can provide this stability and safety can prevent damage to robots or prevent injury related medical costs. This is a challenging task because it involves generating highly dynamic motion for a high-dimensional, non-linear and under-actuated system with contacts. Despite prior advancements in using model-based and optimization methods, challenges such as requirement of extensive domain knowledge, relatively large computational time and limited robustness to changes in dynamics still make this an open problem. In this thesis, to address these issues we develop learning-based algorithms capable of synthesizing push recovery control policies for two different kinds of robots : Humanoid robots and assistive robotic devices that assist in bipedal locomotion. Our work can be branched into two closely related directions : 1) Learning safe falling and fall prevention strategies for humanoid robots and 2) Learning fall prevention strategies for humans using a robotic assistive devices. To achieve this, we introduce a set of Deep Reinforcement Learning (DRL) algorithms to learn control policies that improve safety while using these robots. To enable efficient learning, we present techniques to incorporate abstract dynamical models, curriculum learning and a novel method of building a graph of policies into the learning framework. We also propose an approach to create virtual human walking agents which exhibit similar gait characteristics to real-world human subjects, using which, we learn an assistive device controller to help virtual human return to steady state walking after an external push is applied. Finally, we extend our work on assistive devices and address the challenge of transferring a push-recovery policy to different individuals. As walking and recovery characteristics differ significantly between individuals, exoskeleton policies have to be fine-tuned for each person which is a tedious, time consuming and potentially unsafe process. We propose to solve this by posing it as a transfer learning problem, where a policy trained for one individual can adapt to another without fine tuning.Ph.D