4 research outputs found
Offline Meta-Reinforcement Learning for Industrial Insertion
Reinforcement learning (RL) can in principle let robots automatically adapt
to new tasks, but current RL methods require a large number of trials to
accomplish this. In this paper, we tackle rapid adaptation to new tasks through
the framework of meta-learning, which utilizes past tasks to learn to adapt
with a specific focus on industrial insertion tasks. Fast adaptation is crucial
because prohibitively large number of on-robot trials will potentially damage
hardware pieces. Additionally, effective adaptation is also feasible in that
experience among different insertion applications can be largely leveraged by
each other. In this setting, we address two specific challenges when applying
meta-learning. First, conventional meta-RL algorithms require lengthy online
meta-training. We show that this can be replaced with appropriately chosen
offline data, resulting in an offline meta-RL method that only requires
demonstrations and trials from each of the prior tasks, without the need to run
costly meta-RL procedures online. Second, meta-RL methods can fail to
generalize to new tasks that are too different from those seen at meta-training
time, which poses a particular challenge in industrial applications, where high
success rates are critical. We address this by combining contextual
meta-learning with direct online finetuning: if the new task is similar to
those seen in the prior data, then the contextual meta-learner adapts
immediately, and if it is too different, it gradually adapts through
finetuning. We show that our approach is able to quickly adapt to a variety of
different insertion tasks, with a success rate of 100% using only a fraction of
the samples needed for learning the tasks from scratch. Experiment videos and
details are available at
https://sites.google.com/view/offline-metarl-insertion.Comment: ICRA 202
RoboCat: A Self-Improving Foundation Agent for Robotic Manipulation
The ability to leverage heterogeneous robotic experience from different
robots and tasks to quickly master novel skills and embodiments has the
potential to transform robot learning. Inspired by recent advances in
foundation models for vision and language, we propose a foundation agent for
robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned
decision transformer capable of consuming multi-embodiment action-labelled
visual experience. This data spans a large repertoire of motor control skills
from simulated and real robotic arms with varying sets of observations and
actions. With RoboCat, we demonstrate the ability to generalise to new tasks
and robots, both zero-shot as well as through adaptation using only 100--1000
examples for the target task. We also show how a trained model itself can be
used to generate data for subsequent training iterations, thus providing a
basic building block for an autonomous improvement loop. We investigate the
agent's capabilities, with large-scale evaluations both in simulation and on
three different real robot embodiments. We find that as we grow and diversify
its training data, RoboCat not only shows signs of cross-task transfer, but
also becomes more efficient at adapting to new tasks