1,175 research outputs found
ASPiRe:Adaptive Skill Priors for Reinforcement Learning
We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that
leverages prior experience to accelerate reinforcement learning. Unlike
existing methods that learn a single skill prior from a large and diverse
dataset, our framework learns a library of different distinction skill priors
(i.e., behavior priors) from a collection of specialized datasets, and learns
how to combine them to solve a new task. This formulation allows the algorithm
to acquire a set of specialized skill priors that are more reusable for
downstream tasks; however, it also brings up additional challenges of how to
effectively combine these unstructured sets of skill priors to form a new prior
for new tasks. Specifically, it requires the agent not only to identify which
skill prior(s) to use but also how to combine them (either sequentially or
concurrently) to form a new prior. To achieve this goal, ASPiRe includes
Adaptive Weight Module (AWM) that learns to infer an adaptive weight assignment
between different skill priors and uses them to guide policy learning for
downstream tasks via weighted Kullback-Leibler divergences. Our experiments
demonstrate that ASPiRe can significantly accelerate the learning of new
downstream tasks in the presence of multiple priors and show improvement on
competitive baselines.Comment: 36th Conference on Neural Information Processing Systems (NeurIPS
2022
Active model learning and diverse action sampling for task and motion planning
The objective of this work is to augment the basic abilities of a robot by
learning to use new sensorimotor primitives to enable the solution of complex
long-horizon problems. Solving long-horizon problems in complex domains
requires flexible generative planning that can combine primitive abilities in
novel combinations to solve problems as they arise in the world. In order to
plan to combine primitive actions, we must have models of the preconditions and
effects of those actions: under what circumstances will executing this
primitive achieve some particular effect in the world?
We use, and develop novel improvements on, state-of-the-art methods for
active learning and sampling. We use Gaussian process methods for learning the
conditions of operator effectiveness from small numbers of expensive training
examples collected by experimentation on a robot. We develop adaptive sampling
methods for generating diverse elements of continuous sets (such as robot
configurations and object poses) during planning for solving a new task, so
that planning is as efficient as possible. We demonstrate these methods in an
integrated system, combining newly learned models with an efficient
continuous-space robot task and motion planner to learn to solve long horizon
problems more efficiently than was previously possible.Comment: Proceedings of the 2018 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), Madrid, Spain.
https://www.youtube.com/playlist?list=PLoWhBFPMfSzDbc8CYelsbHZa1d3uz-W_
- …