687 research outputs found
Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion
Learning policies for bipedal locomotion can be difficult, as experiments are
expensive and simulation does not usually transfer well to hardware. To counter
this, we need al- gorithms that are sample efficient and inherently safe.
Bayesian Optimization is a powerful sample-efficient tool for optimizing
non-convex black-box functions. However, its performance can degrade in higher
dimensions. We develop a distance metric for bipedal locomotion that enhances
the sample-efficiency of Bayesian Optimization and use it to train a 16
dimensional neuromuscular model for planar walking. This distance metric
reflects some basic gait features of healthy walking and helps us quickly
eliminate a majority of unstable controllers. With our approach we can learn
policies for walking in less than 100 trials for a range of challenging
settings. In simulation, we show results on two different costs and on various
terrains including rough ground and ramps, sloping upwards and downwards. We
also perturb our models with unknown inertial disturbances analogous with
differences between simulation and hardware. These results are promising, as
they indicate that this method can potentially be used to learn control
policies on hardware.Comment: To appear in International Conference on Humanoid Robots (Humanoids
'2016), IEEE-RAS. (Rika Antonova and Akshara Rai contributed equally
Deep Kernels for Optimizing Locomotion Controllers
Sample efficiency is important when optimizing parameters of locomotion
controllers, since hardware experiments are time consuming and expensive.
Bayesian Optimization, a sample-efficient optimization framework, has recently
been widely applied to address this problem, but further improvements in sample
efficiency are needed for practical applicability to real-world robots and
high-dimensional controllers. To address this, prior work has proposed using
domain expertise for constructing custom distance metrics for locomotion. In
this work we show how to learn such a distance metric automatically. We use a
neural network to learn an informed distance metric from data obtained in
high-fidelity simulations. We conduct experiments on two different controllers
and robot architectures. First, we demonstrate improvement in sample efficiency
when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We
then conduct simulation experiments to optimize a 16-dimensional controller for
a 7-link robot model and obtain significant improvements even when optimizing
in perturbed environments. This demonstrates that our approach is able to
enhance sample efficiency for two different controllers, hence is a fitting
candidate for further experiments on hardware in the future.Comment: (Rika Antonova and Akshara Rai contributed equally
Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped
Controllers in robotics often consist of expert-designed heuristics, which
can be hard to tune in higher dimensions. It is typical to use simulation to
learn these parameters, but controllers learned in simulation often don't
transfer to hardware. This necessitates optimization directly on hardware.
However, collecting data on hardware can be expensive. This has led to a recent
interest in adapting data-efficient learning techniques to robotics. One
popular method is Bayesian Optimization (BO), a sample-efficient black-box
optimization scheme, but its performance typically degrades in higher
dimensions. We aim to overcome this problem by incorporating domain knowledge
to reduce dimensionality in a meaningful way, with a focus on bipedal
locomotion. In previous work, we proposed a transformation based on knowledge
of human walking that projected a 16-dimensional controller to a 1-dimensional
space. In simulation, this showed enhanced sample efficiency when optimizing
human-inspired neuromuscular walking controllers on a humanoid model. In this
paper, we present a generalized feature transform applicable to non-humanoid
robot morphologies and evaluate it on the ATRIAS bipedal robot -- in simulation
and on hardware. We present three different walking controllers; two are
evaluated on the real robot. Our results show that this feature transform
captures important aspects of walking and accelerates learning on hardware and
simulation, as compared to traditional BO.Comment: 8 pages, submitted to IEEE International Conference on Robotics and
Automation 201
Deep Reinforcement Learning for Tensegrity Robot Locomotion
Tensegrity robots, composed of rigid rods connected by elastic cables, have a
number of unique properties that make them appealing for use as planetary
exploration rovers. However, control of tensegrity robots remains a difficult
problem due to their unusual structures and complex dynamics. In this work, we
show how locomotion gaits can be learned automatically using a novel extension
of mirror descent guided policy search (MDGPS) applied to periodic locomotion
movements, and we demonstrate the effectiveness of our approach on tensegrity
robot locomotion. We evaluate our method with real-world and simulated
experiments on the SUPERball tensegrity robot, showing that the learned
policies generalize to changes in system parameters, unreliable sensor
measurements, and variation in environmental conditions, including varied
terrains and a range of different gravities. Our experiments demonstrate that
our method not only learns fast, power-efficient feedback policies for rolling
gaits, but that these policies can succeed with only the limited onboard
sensing provided by SUPERball's accelerometers. We compare the learned feedback
policies to learned open-loop policies and hand-engineered controllers, and
demonstrate that the learned policy enables the first continuous, reliable
locomotion gait for the real SUPERball robot. Our code and other supplementary
materials are available from http://rll.berkeley.edu/drl_tensegrityComment: International Conference on Robotics and Automation (ICRA), 2017.
Project website link is http://rll.berkeley.edu/drl_tensegrit
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking
This manuscript presents control of a high-DOF fully actuated lower-limb
exoskeleton for paraplegic individuals. The key novelty is the ability for the
user to walk without the use of crutches or other external means of
stabilization. We harness the power of modern optimization techniques and
supervised machine learning to develop a smooth feedback control policy that
provides robust velocity regulation and perturbation rejection. Preliminary
evaluation of the stability and robustness of the proposed approach is
demonstrated through the Gazebo simulation environment. In addition,
preliminary experimental results with (complete) paraplegic individuals are
included for the previous version of the controller.Comment: Submitted to IEEE Control System Magazine. This version addresses
reviewers' concerns about the robustness of the algorithm and the motivation
for using such exoskeleton
Learning Image-Conditioned Dynamics Models for Control of Under-actuated Legged Millirobots
Millirobots are a promising robotic platform for many applications due to
their small size and low manufacturing costs. Legged millirobots, in
particular, can provide increased mobility in complex environments and improved
scaling of obstacles. However, controlling these small, highly dynamic, and
underactuated legged systems is difficult. Hand-engineered controllers can
sometimes control these legged millirobots, but they have difficulties with
dynamic maneuvers and complex terrains. We present an approach for controlling
a real-world legged millirobot that is based on learned neural network models.
Using less than 17 minutes of data, our method can learn a predictive model of
the robot's dynamics that can enable effective gaits to be synthesized on the
fly for following user-specified waypoints on a given terrain. Furthermore, by
leveraging expressive, high-capacity neural network models, our approach allows
for these predictions to be directly conditioned on camera images, endowing the
robot with the ability to predict how different terrains might affect its
dynamics. This enables sample-efficient and effective learning for locomotion
of a dynamic legged millirobot on various terrains, including gravel, turf,
carpet, and styrofoam. Experiment videos can be found at
https://sites.google.com/view/imageconddy
- …