Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion

Learning policies for bipedal locomotion can be difficult, as experiments are expensive and simulation does not usually transfer well to hardware. To counter this, we need al- gorithms that are sample efficient and inherently safe. Bayesian Optimization is a powerful sample-efficient tool for optimizing non-convex black-box functions. However, its performance can degrade in higher dimensions. We develop a distance metric for bipedal locomotion that enhances the sample-efficiency of Bayesian Optimization and use it to train a 16 dimensional neuromuscular model for planar walking. This distance metric reflects some basic gait features of healthy walking and helps us quickly eliminate a majority of unstable controllers. With our approach we can learn policies for walking in less than 100 trials for a range of challenging settings. In simulation, we show results on two different costs and on various terrains including rough ground and ramps, sloping upwards and downwards. We also perturb our models with unknown inertial disturbances analogous with differences between simulation and hardware. These results are promising, as they indicate that this method can potentially be used to learn control policies on hardware.Comment: To appear in International Conference on Humanoid Robots (Humanoids '2016), IEEE-RAS. (Rika Antonova and Akshara Rai contributed equally

Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped

Controllers in robotics often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. It is typical to use simulation to learn these parameters, but controllers learned in simulation often don't transfer to hardware. This necessitates optimization directly on hardware. However, collecting data on hardware can be expensive. This has led to a recent interest in adapting data-efficient learning techniques to robotics. One popular method is Bayesian Optimization (BO), a sample-efficient black-box optimization scheme, but its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge to reduce dimensionality in a meaningful way, with a focus on bipedal locomotion. In previous work, we proposed a transformation based on knowledge of human walking that projected a 16-dimensional controller to a 1-dimensional space. In simulation, this showed enhanced sample efficiency when optimizing human-inspired neuromuscular walking controllers on a humanoid model. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot -- in simulation and on hardware. We present three different walking controllers; two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.Comment: 8 pages, submitted to IEEE International Conference on Robotics and Automation 201

Systematic Controller Design for Dynamic 3D Bipedal Robot Walking.

Virtual constraints and hybrid zero dynamics (HZD) have emerged as a powerful framework for controlling bipedal robot locomotion, as evidenced by the robust, energetically efficient, and natural-looking walking and running gaits achieved by planar robots such as Rabbit, ERNIE, and MABEL. However, the extension to 3D robots is more subtle, as the choice of virtual constraints has a deciding effect on the stability of a periodic orbit. Furthermore, previous methods did not provide a systematic means of designing virtual constraints to ensure stability. This thesis makes both experimental and theoretical contributions to the control of underactuated 3D bipedal robots. On the experimental side, we present the first realization of dynamic 3D walking using virtual constraints. The experimental success is achieved by augmenting a robust planar walking gait with a novel virtual constraint for the lateral swing hip angle. The resulting controller is tested in the laboratory on a human-scale bipedal robot (MARLO) and demonstrated to stabilize the lateral motion for unassisted 3D walking at approximately 1 m/s. MARLO is one of only two known robots to walk in 3D with stilt-like feet. On the theoretical side, we introduce a method to systematically tune a given choice of virtual constraints in order to stabilize a periodic orbit of a hybrid system. We demonstrate the method to stabilize a walking gait for MARLO, and show that the optimized controller leads to improved lateral control compared to the nominal virtual constraints. We also describe several extensions of the basic method, allowing the use of a restricted Poincaré map and the incorporation of disturbance rejection metrics in the optimization. Together, these methods comprise an important contribution to the theory of HZD.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113370/1/bgbuss_1.pd

Design Principles for Energy-Efficient Legged Locomotion and Implementation on the MIT Cheetah Robot

This paper presents the design principles for highly efficient legged robots, the implementation of the principles in the design of the MIT Cheetah, and the analysis of the high-speed trotting experimental results. The design principles were derived by analyzing three major energy-loss mechanisms in locomotion: heat losses from the actuators, friction losses in transmission, and the interaction losses caused by the interface between the system and the environment. Four design principles that minimize these losses are discussed: employment of high torque-density motors, energy regenerative electronic system, low loss transmission, and a low leg inertia. These principles were implemented in the design of the MIT Cheetah; the major design features are large gap diameter motors, regenerative electric motor drivers, single-stage low gear transmission, dual coaxial motors with composite legs, and the differential actuated spine. The experimental results of fast trotting are presented; the 33-kg robot runs at 22 km/h (6 m/s). The total power consumption from the battery pack was 973 W and resulted in a total cost of transport of 0.5, which rivals running animals' at the same scale. 76% of the total energy consumption is attributed to heat loss from the motor, and the remaining 24% is used in mechanical work, which is dissipated as interaction loss as well as friction losses at the joint and transmission.United States. Defense Advanced Research Projects Agency (M3 Program

The effects of swing-leg retraction on running performance: analysis, simulation, and experiment

Using simple running models, researchers have argued that swing-leg retraction can improve running robot performance. In this paper, we investigate whether this holds for a more realistic simulation model validated against a physical running robot. We find that swing-leg retraction can improve stability and disturbance rejection. Alternatively, swing-leg retraction can simultaneously reduce touchdown forces, slipping likelihood, and impact energy losses. Surprisingly, swing-leg retraction barely affected net energetic efficiency. The retraction rates at which these effects are the greatest are strongly model-dependent, suggesting that robot designers cannot always rely on simplified models to accurately predict such complex behaviors

Optimal relative timing of stance push-off and swing leg retraction

Abstract-Swing leg retraction, the backward rotation of the swing leg prior to heel-strike, is known to have several advantages in legged locomotion. To achieve this motion, a hip torque is required at the end of the swing phase to brake the forward rotation of the leg and/or accelerate its backward motion. In walking, pre-emptive push-off of the stance leg also occurs at the end of the swing, so its relative timing with late-swing retracting torque influences gait energetics. To find the best relative timing between the stance leg's pushoff force and the swing leg retraction torque, we calculate their work-based energetics in a simple bipedal model using impulsive approximations and with the aid of the so-called overlap parameter that quantifies the relative order and the percentage overlap of the push-off and retraction impulses. By minimizing the energetic cost of the gait, we found that it is energetically favorable to start with the push-off force, and postpone braking the leg swing until completely after the push-off (impulsive force/torque). The implication for the more realistic non-impulsive cases is to apply the retraction torque at the very end of the push-off before heel-strike. We show that the results are valid for many other bipedal models, for both periodic and aperiodic gaits, and regardless of the actuator efficiencies for positive and negative work

The Role of Reflexes Versus Central Pattern Generators

Animals execute locomotor behaviors and more with ease. They have evolved these breath-taking abilities over millions of years. Cheetahs can run, dolphins can swim and flies can fly like no artificial technology can. It is often argued that if human technology could mimic nature, then biological-like performance would follow. Unfortunately, the blind copying or mimicking of a part of nature [Ritzmann et al., 2000] does not often lead to the best design for a variety of reasons [Vogel, 1998]. Evolution works on the just good enough principle. Optimal designs are not the necessary end product of evolution. Multiple satisfactory solutions can result in similar performances. Animals do bring to our attention amazing designs, but these designs carry with them the baggage of their history. Moreover, natural design is constrained by factors that may have no relationship to human engineered designs. Animals must be able to grow over time, but still function along the way. Finally, animals are complex and their parts serve multiple functions, not simply the one we happen to examine. In short, in their daunting complexity and integrated function, understanding animal behaviors remains as intractable as their capabilities are tantalizing

Bio-inspired swing leg control for spring-mass robots running on ground with unexpected height disturbance

We proposed three swing leg control policies for spring-mass running robots, inspired by experimental data from our recent collaborative work on ground running birds. Previous investigations suggest that animals may prioritize injury avoidance and/or efficiency as their objective function during running rather than maintaining limit-cycle stability. Therefore, in this study we targeted structural capacity (maximum leg force to avoid damage) and efficiency as the main goals for our control policies, since these objective functions are crucial to reduce motor size and structure weight. Each proposed policy controls the leg angle as a function of time during flight phase such that its objective function during the subsequent stance phase is regulated. The three objective functions that are regulated in the control policies are (i) the leg peak force, (ii) the axial impulse, and (iii) the leg actuator work. It should be noted that each control policy regulates one single objective function. Surprisingly, all three swing leg control policies result in nearly identical subsequent stance phase dynamics. This implies that the implementation of any of the proposed control policies would satisfy both goals (damage avoidance and efficiency) at once. Furthermore, all three control policies require a surprisingly simple leg angle adjustment: leg retraction with constant angular acceleration