62 research outputs found
Mutual Alignment Transfer Learning
Training robots for operation in the real world is a complex, time consuming
and potentially expensive task. Despite significant success of reinforcement
learning in games and simulations, research in real robot applications has not
been able to match similar progress. While sample complexity can be reduced by
training policies in simulation, such policies can perform sub-optimally on the
real platform given imperfect calibration of model dynamics. We present an
approach -- supplemental to fine tuning on the real robot -- to further benefit
from parallel access to a simulator during training and reduce sample
requirements on the real robot. The developed approach harnesses auxiliary
rewards to guide the exploration for the real world agent based on the
proficiency of the agent in simulation and vice versa. In this context, we
demonstrate empirically that the reciprocal alignment for both agents provides
further benefit as the agent in simulation can adjust to optimize its behaviour
for states commonly visited by the real-world agent
Fast Model Identification via Physics Engines for Data-Efficient Policy Search
This paper presents a method for identifying mechanical parameters of robots
or objects, such as their mass and friction coefficients. Key features are the
use of off-the-shelf physics engines and the adaptation of a Bayesian
optimization technique towards minimizing the number of real-world experiments
needed for model-based reinforcement learning. The proposed framework
reproduces in a physics engine experiments performed on a real robot and
optimizes the model's mechanical parameters so as to match real-world
trajectories. The optimized model is then used for learning a policy in
simulation, before real-world deployment. It is well understood, however, that
it is hard to exactly reproduce real trajectories in simulation. Moreover, a
near-optimal policy can be frequently found with an imperfect model. Therefore,
this work proposes a strategy for identifying a model that is just good enough
to approximate the value of a locally optimal policy with a certain confidence,
instead of wasting effort on identifying the most accurate model. Evaluations,
performed both in simulation and on a real robotic manipulation task, indicate
that the proposed strategy results in an overall time-efficient, integrated
model identification and learning solution, which significantly improves the
data-efficiency of existing policy search algorithms.Comment: IJCAI 1
Multi-Task Policy Search for Robotics
© 2014 IEEE.Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics. Training individual policies for every single potential task is often impractical, especially for continuous task variations, requiring more principled approaches to share and transfer knowledge among similar tasks. We present a novel approach for learning a nonlinear feedback policy that generalizes across multiple tasks. The key idea is to define a parametrized policy as a function of both the state and the task, which allows learning a single policy that generalizes across multiple known and unknown tasks. Applications of our novel approach to reinforcement and imitation learning in realrobot experiments are shown
- …