8,872 research outputs found
Real-to-Sim: Deep Learning with Auto-Tuning to Predict Residual Errors using Sparse Data
Achieving highly accurate kinematic or simulator models that are close to the
real robot can facilitate model-based controls (e.g., model predictive control
or linear-quadradic regulators), model-based trajectory planning (e.g.,
trajectory optimization), and decrease the amount of learning time necessary
for reinforcement learning methods. Thus, the objective of this work is to
learn the residual errors between a kinematic and/or simulator model and the
real robot. This is achieved using auto-tuning and neural networks, where the
parameters of a neural network are updated using an auto-tuning method that
applies equations from an Unscented Kalman Filter (UKF) formulation. Using this
method, we model these residual errors with only small amounts of data - a
necessity as we improve the simulator/kinematic model by learning directly from
hardware operation. We demonstrate our method on robotic hardware (e.g.,
manipulator arm), and show that with the learned residual errors, we can
further close the reality gap between kinematic models, simulations, and the
real robot
Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning
Reinforcement learning (RL) algorithms for real-world robotic applications
need a data-efficient learning process and the ability to handle complex,
unknown dynamical systems. These requirements are handled well by model-based
and model-free RL approaches, respectively. In this work, we aim to combine the
advantages of these two types of methods in a principled manner. By focusing on
time-varying linear-Gaussian policies, we enable a model-based algorithm based
on the linear quadratic regulator (LQR) that can be integrated into the
model-free framework of path integral policy improvement (PI2). We can further
combine our method with guided policy search (GPS) to train arbitrary
parameterized policies such as deep neural networks. Our simulation and
real-world experiments demonstrate that this method can solve challenging
manipulation tasks with comparable or better performance than model-free
methods while maintaining the sample efficiency of model-based methods. A video
presenting our results is available at
https://sites.google.com/site/icml17pilqrComment: Paper accepted to the International Conference on Machine Learning
(ICML) 201
Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics
The most data-efficient algorithms for reinforcement learning in robotics are
model-based policy search algorithms, which alternate between learning a
dynamical model of the robot and optimizing a policy to maximize the expected
return given the model and its uncertainties. Among the few proposed
approaches, the recently introduced Black-DROPS algorithm exploits a black-box
optimization algorithm to achieve both high data-efficiency and good
computation times when several cores are used; nevertheless, like all
model-based policy search approaches, Black-DROPS does not scale to high
dimensional state/action spaces. In this paper, we introduce a new model
learning procedure in Black-DROPS that leverages parameterized black-box priors
to (1) scale up to high-dimensional systems, and (2) be robust to large
inaccuracies of the prior information. We demonstrate the effectiveness of our
approach with the "pendubot" swing-up task in simulation and with a physical
hexapod robot (48D state space, 18D action space) that has to walk forward as
fast as possible. The results show that our new algorithm is more
data-efficient than previous model-based policy search algorithms (with and
without priors) and that it can allow a physical 6-legged robot to learn new
gaits in only 16 to 30 seconds of interaction time.Comment: Accepted at ICRA 2018; 8 pages, 4 figures, 2 algorithms, 1 table;
Video at https://youtu.be/HFkZkhGGzTo ; Spotlight ICRA presentation at
https://youtu.be/_MZYDhfWeL
Fast Model Identification via Physics Engines for Data-Efficient Policy Search
This paper presents a method for identifying mechanical parameters of robots
or objects, such as their mass and friction coefficients. Key features are the
use of off-the-shelf physics engines and the adaptation of a Bayesian
optimization technique towards minimizing the number of real-world experiments
needed for model-based reinforcement learning. The proposed framework
reproduces in a physics engine experiments performed on a real robot and
optimizes the model's mechanical parameters so as to match real-world
trajectories. The optimized model is then used for learning a policy in
simulation, before real-world deployment. It is well understood, however, that
it is hard to exactly reproduce real trajectories in simulation. Moreover, a
near-optimal policy can be frequently found with an imperfect model. Therefore,
this work proposes a strategy for identifying a model that is just good enough
to approximate the value of a locally optimal policy with a certain confidence,
instead of wasting effort on identifying the most accurate model. Evaluations,
performed both in simulation and on a real robotic manipulation task, indicate
that the proposed strategy results in an overall time-efficient, integrated
model identification and learning solution, which significantly improves the
data-efficiency of existing policy search algorithms.Comment: IJCAI 1
Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning
Obstacle avoidance is a fundamental requirement for autonomous robots which
operate in, and interact with, the real world. When perception is limited to
monocular vision avoiding collision becomes significantly more challenging due
to the lack of 3D information. Conventional path planners for obstacle
avoidance require tuning a number of parameters and do not have the ability to
directly benefit from large datasets and continuous use. In this paper, a
dueling architecture based deep double-Q network (D3QN) is proposed for
obstacle avoidance, using only monocular RGB vision. Based on the dueling and
double-Q mechanisms, D3QN can efficiently learn how to avoid obstacles in a
simulator even with very noisy depth information predicted from RGB image.
Extensive experiments show that D3QN enables twofold acceleration on learning
compared with a normal deep Q network and the models trained solely in virtual
environments can be directly transferred to real robots, generalizing well to
various new environments with previously unseen dynamic objects.Comment: Accepted by RSS 2017 workshop New Frontiers for Deep Learning in
Robotic
- …