1,226 research outputs found
MARBLER: An Open Platform for Standarized Evaluation of Multi-Robot Reinforcement Learning Algorithms
Multi-agent reinforcement learning (MARL) has enjoyed significant recent
progress, thanks to deep learning. This is naturally starting to benefit
multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However,
existing infrastructure to train and evaluate policies predominantly focus on
challenges in coordinating virtual agents, and ignore characteristics important
to robotic systems. Few platforms support realistic robot dynamics, and fewer
still can evaluate Sim2Real performance of learned behavior. To address these
issues, we contribute MARBLER: Multi-Agent RL Benchmark and Learning
Environment for the Robotarium. MARBLER offers a robust and comprehensive
evaluation platform for MRRL by marrying Georgia Tech's Robotarium (which
enables rapid prototyping on physical MRS) and OpenAI's Gym framework (which
facilitates standardized use of modern learning algorithms). MARBLER offers a
highly controllable environment with realistic dynamics, including barrier
certificate-based obstacle avoidance. It allows anyone across the world to
train and deploy MRRL algorithms on a physical testbed with reproducibility.
Further, we introduce five novel scenarios inspired by common challenges in MRS
and provide support for new custom scenarios. Finally, we use MARBLER to
evaluate popular MARL algorithms and provide insights into their suitability
for MRRL. In summary, MARBLER can be a valuable tool to the MRS research
community by facilitating comprehensive and standardized evaluation of learning
algorithms on realistic simulations and physical hardware. Links to our
open-source framework and the videos of real-world experiments can be found at
https://shubhlohiya.github.io/MARBLER/.Comment: 7 pages, 3 figures, submitted to MRS 2023, for the associated
website, see https://shubhlohiya.github.io/MARBLER
DoShiCo Challenge: Domain Shift in Control Prediction
Training deep neural network policies end-to-end for real-world applications
so far requires big demonstration datasets in the real world or big sets
consisting of a large variety of realistic and closely related 3D CAD models.
These real or virtual data should, moreover, have very similar characteristics
to the conditions expected at test time. These stringent requirements and the
time consuming data collection processes that they entail, are currently the
most important impediment that keeps deep reinforcement learning from being
deployed in real-world applications. Therefore, in this work we advocate an
alternative approach, where instead of avoiding any domain shift by carefully
selecting the training data, the goal is to learn a policy that can cope with
it. To this end, we propose the DoShiCo challenge: to train a model in very
basic synthetic environments, far from realistic, in a way that it can be
applied in more realistic environments as well as take the control decisions on
real-world data. In particular, we focus on the task of collision avoidance for
drones. We created a set of simulated environments that can be used as
benchmark and implemented a baseline method, exploiting depth prediction as an
auxiliary task to help overcome the domain shift. Even though the policy is
trained in very basic environments, it can learn to fly without collisions in a
very different realistic simulated environment. Of course several benchmarks
for reinforcement learning already exist - but they never include a large
domain shift. On the other hand, several benchmarks in computer vision focus on
the domain shift, but they take the form of a static datasets instead of
simulated environments. In this work we claim that it is crucial to take the
two challenges together in one benchmark.Comment: Published at SIMPAR 2018. Please visit the paper webpage for more
information, a movie and code for reproducing results:
https://kkelchte.github.io/doshic
Benchmarking Deep Reinforcement Learning for Continuous Control
Recently, researchers have made significant progress combining the advances
in deep learning for learning feature representations with reinforcement
learning. Some notable examples include training agents to play Atari games
based on raw pixel data and to acquire advanced manipulation skills using raw
sensory inputs. However, it has been difficult to quantify progress in the
domain of continuous control due to the lack of a commonly adopted benchmark.
In this work, we present a benchmark suite of continuous control tasks,
including classic tasks like cart-pole swing-up, tasks with very high state and
action dimensionality such as 3D humanoid locomotion, tasks with partial
observations, and tasks with hierarchical structure. We report novel findings
based on the systematic evaluation of a range of implemented reinforcement
learning algorithms. Both the benchmark and reference implementations are
released at https://github.com/rllab/rllab in order to facilitate experimental
reproducibility and to encourage adoption by other researchers.Comment: 14 pages, ICML 201
- …