Deep Reinforcement Learning for Tensegrity Robot Locomotion

Tensegrity robots, composed of rigid rods connected by elastic cables, have a number of unique properties that make them appealing for use as planetary exploration rovers. However, control of tensegrity robots remains a difficult problem due to their unusual structures and complex dynamics. In this work, we show how locomotion gaits can be learned automatically using a novel extension of mirror descent guided policy search (MDGPS) applied to periodic locomotion movements, and we demonstrate the effectiveness of our approach on tensegrity robot locomotion. We evaluate our method with real-world and simulated experiments on the SUPERball tensegrity robot, showing that the learned policies generalize to changes in system parameters, unreliable sensor measurements, and variation in environmental conditions, including varied terrains and a range of different gravities. Our experiments demonstrate that our method not only learns fast, power-efficient feedback policies for rolling gaits, but that these policies can succeed with only the limited onboard sensing provided by SUPERball's accelerometers. We compare the learned feedback policies to learned open-loop policies and hand-engineered controllers, and demonstrate that the learned policy enables the first continuous, reliable locomotion gait for the real SUPERball robot. Our code and other supplementary materials are available from http://rll.berkeley.edu/drl_tensegrityComment: International Conference on Robotics and Automation (ICRA), 2017. Project website link is http://rll.berkeley.edu/drl_tensegrit

Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics

Properly designing a system to exhibit favorable natural dynamics can greatly simplify designing or learning the control policy. However, it is still unclear what constitutes favorable natural dynamics and how to quantify its effect. Most studies of simple walking and running models have focused on the basins of attraction of passive limit-cycles and the notion of self-stability. We instead emphasize the importance of stepping beyond basins of attraction. We show an approach based on viability theory to quantify robust sets in state-action space. These sets are valid for the family of all robust control policies, which allows us to quantify the robustness inherent to the natural dynamics before designing the control policy or specifying a control objective. We illustrate our formulation using spring-mass models, simple low dimensional models of running systems. We then show an example application by optimizing robustness of a simulated planar monoped, using a gradient-free optimization scheme. Both case studies result in a nonlinear effective stiffness providing more robustness.Comment: 15 pages. This work has been accepted to IEEE Transactions on Robotics (2019

A physical model suggests that hip-localized balance sense in birds improves state estimation in perching: implications for bipedal robots

In addition to a vestibular system, birds uniquely have a balance-sensing organ within the pelvis, called the lumbosacral organ (LSO). The LSO is well developed in terrestrial birds, possibly to facilitate balance control in perching and terrestrial locomotion. No previous studies have quantified the functional benefits of the LSO for balance. We suggest two main benefits of hip-localized balance sense: reduced sensorimotor delay and improved estimation of foot-ground acceleration. We used system identification to test the hypothesis that hip-localized balance sense improves estimates of foot acceleration compared to a head-localized sense, due to closer proximity to the feet. We built a physical model of a standing guinea fowl perched on a platform, and used 3D accelerometers at the hip and head to replicate balance sense by the LSO and vestibular systems. The horizontal platform was attached to the end effector of a 6 DOF robotic arm, allowing us to apply perturbations to the platform analogous to motions of a compliant branch. We also compared state estimation between models with low and high neck stiffness. Cross-correlations revealed that foot-to-hip sensing delays were shorter than foot-to-head, as expected. We used multi-variable output error state-space (MOESP) system identification to estimate foot-ground acceleration as a function of hip- and head-localized sensing, individually and combined. Hip-localized sensors alone provided the best state estimates, which were not improved when fused with head-localized sensors. However, estimates from head-localized sensors improved with higher neck stiffness. Our findings support the hypothesis that hip-localized balance sense improves the speed and accuracy of foot state estimation compared to head-localized sense. The findings also suggest a role of neck muscles for active sensing for balance control: increased neck stiffness through muscle co-contraction can improve the utility of vestibular signals. Our engineering approach provides, to our knowledge, the first quantitative evidence for functional benefits of the LSO balance sense in birds. The findings support notions of control modularity in birds, with preferential vestibular sense for head stability and gaze, and LSO for body balance control,respectively. The findings also suggest advantages for distributed and active sensing for agile locomotion in compliant bipedal robots

A Bioinspired Dynamical Vertical Climbing Robot

This paper describes the inspiration, design, analysis, implementation of and experimentation with the first dynamical vertical climbing robot. Biologists have proposed a pendulous climbing model that abstracts remarkable similarities in dynamic wall scaling behavior exhibited by radically different animal species. We study numerically a version of that pendulous climbing template dynamically re-scaled for applicability to utilitarian payloads with conventional electronics and actuation. This simulation study reveals that the incorporation of passive compliance can compensate for an artifact’s poorer power density and scale disadvantages relative to biology. However the introduction of additional dynamical elements raises new concerns about stability regarding both the power stroke and limb coordination that we allay via mathematical analysis of further simplified models. Combining these numerical and analytical insights into a series of design prototypes, we document the correspondence of the various models to the variously scaled platforms and report that our approximately two kilogram platform climbs dynamically at vertical speeds up to 1.5 bodylengths per second. In particular, the final 2.6 kg final prototype climbs at an average steady state speed of 0.66 m/s against gravity on a carpeted vertical wall, in rough agreement with our various models’ predictions

Convergence of Bayesian Histogram Filters for Location Estimation

We prove convergence of an approximate Bayesian estimator for the (scalar) location estimation problem by recourse to a histogram approximant. We exploit its tractability to present a simple strategy for managing the tradeoff between accuracy and complexity through the cardinality of the underlying partition. Our theoretical results provide explicit (conservative) sufficient conditions under which convergence is guaranteed. Numerical simulations reveal certain extreme cases in which the conditions may be tight, and suggest that this procedure has performance and computational efficiency favorably comparable to particle filters, while affording the aforementioned analytical benefits. We posit that more sophisticated algorithms can make such piecewise-constant representations similarly feasible for very high-dimensional problems. For more information: Kod*La

Energy-Efficient Monopod Running with a Large Payload Based on Open-Loop Parallel Elastic Actuation

Despite the intensive investigations in the past, energetic efficiency is still one of the most important unsolved challenges in legged robot locomotion. This paper presents an unconventional approach to the problem of energetically efficient legged locomotion by applying actuation for spring mass running. This approach makes use of mechanical springs incorporated in parallel with relatively low-torque actuation, which is capable of both accommodating large payload and locomotion with low power input by exploiting self-excited vibration. For a systematic analysis, this paper employs both simulation models and physical platforms. The experiments show that the proposed approach is scalable across different payload between 0 and 150kg, and able to achieve a total cost of transport (TCOT) of 0.10, which is significantly lower than the previous locomotion robots and most of the biological systems in the similar scale, when actuated with the near-to natural frequency with the maximum payload.This study was supported by the Swiss National Science Foundation Grant No. PP00P2123387/1 and the Swiss National Science Foundation through the National Centre of Competence in Research Robotics

Optimal Design Methods for Increasing Power Performance of Multiactuator Robotic Limbs

abstract: In order for assistive mobile robots to operate in the same environment as humans, they must be able to navigate the same obstacles as humans do. Many elements are required to do this: a powerful controller which can understand the obstacle, and power-dense actuators which will be able to achieve the necessary limb accelerations and output energies. Rapid growth in information technology has made complex controllers, and the devices which run them considerably light and cheap. The energy density of batteries, motors, and engines has not grown nearly as fast. This is problematic because biological systems are more agile, and more efficient than robotic systems. This dissertation introduces design methods which may be used optimize a multiactuator robotic limb's natural dynamics in an effort to reduce energy waste. These energy savings decrease the robot's cost of transport, and the weight of the required fuel storage system. To achieve this, an optimal design method, which allows the specialization of robot geometry, is introduced. In addition to optimal geometry design, a gearing optimization is presented which selects a gear ratio which minimizes the electrical power at the motor while considering the constraints of the motor. Furthermore, an efficient algorithm for the optimization of parallel stiffness elements in the robot is introduced. In addition to the optimal design tools introduced, the KiTy SP robotic limb structure is also presented. Which is a novel hybrid parallel-serial actuation method. This novel leg structure has many desirable attributes such as: three dimensional end-effector positioning, low mobile mass, compact form-factor, and a large workspace. We also show that the KiTy SP structure outperforms the classical, biologically-inspired serial limb structure.Dissertation/ThesisDoctoral Dissertation Mechanical Engineering 201