Emergence of Human-comparable Balancing Behaviors by Deep Reinforcement Learning

This paper presents a hierarchical framework based on deep reinforcement learning that learns a diversity of policies for humanoid balance control. Conventional zero moment point based controllers perform limited actions during under-actuation, whereas the proposed framework can perform human-like balancing behaviors such as active push-off of ankles. The learning is done through the design of an explainable reward based on physical constraints. The simulated results are presented and analyzed. The successful emergence of human-like behaviors through deep reinforcement learning proves the feasibility of using an AI-based approach for learning humanoid balancing control in a unified framework

Optimal coordination of maximal-effort horizontal and vertical jump motions – a computer simulation study

<p>Abstract</p> <p>Background</p> <p>The purpose of this study was to investigate the coordination strategy of maximal-effort horizontal jumping in comparison with vertical jumping, using the methodology of computer simulation.</p> <p>Methods</p> <p>A skeletal model that has nine rigid body segments and twenty degrees of freedom was developed. Thirty-two Hill-type lower limb muscles were attached to the model. The excitation-contraction dynamics of the contractile element, the tissues around the joints to limit the joint range of motion, as well as the foot-ground interaction were implemented. Simulations were initiated from an identical standing posture for both motions. Optimal pattern of the activation input signal was searched through numerical optimization. For the horizontal jumping, the goal was to maximize the horizontal distance traveled by the body's center of mass. For the vertical jumping, the goal was to maximize the height reached by the body's center of mass.</p> <p>Results</p> <p>As a result, it was found that the hip joint was utilized more vigorously in the horizontal jumping than in the vertical jumping. The muscles that have a function of joint flexion such as the m. iliopsoas, m. rectus femoris and m. tibialis anterior were activated to a greater level during the countermovement in the horizontal jumping with an effect of moving the body's center of mass in the forward direction. Muscular work was transferred to the mechanical energy of the body's center of mass more effectively in the horizontal jump, which resulted in a greater energy gain of the body's center of mass throughout the motion.</p> <p>Conclusion</p> <p>These differences in the optimal coordination strategy seem to be caused from the requirement that the body's center of mass needs to be located above the feet in a vertical jumping, whereas this requirement is not so strict in a horizontal jumping.</p