831 research outputs found
Parallel Reinforcement Learning Simulation for Visual Quadrotor Navigation
Reinforcement learning (RL) is an agent-based approach for teaching robots to
navigate within the physical world. Gathering data for RL is known to be a
laborious task, and real-world experiments can be risky. Simulators facilitate
the collection of training data in a quicker and more cost-effective manner.
However, RL frequently requires a significant number of simulation steps for an
agent to become skilful at simple tasks. This is a prevalent issue within the
field of RL-based visual quadrotor navigation where state dimensions are
typically very large and dynamic models are complex. Furthermore, rendering
images and obtaining physical properties of the agent can be computationally
expensive. To solve this, we present a simulation framework, built on AirSim,
which provides efficient parallel training. Building on this framework, Ape-X
is modified to incorporate decentralised training of AirSim environments to
make use of numerous networked computers. Through experiments we were able to
achieve a reduction in training time from 3.9 hours to 11 minutes using the
aforementioned framework and a total of 74 agents and two networked computers.
Further details including a github repo and videos about our project,
PRL4AirSim, can be found at https://sites.google.com/view/prl4airsim/homeComment: This work has been submitted to the IEEE International Conference on
Robotics and Automation (ICRA) for possible publication. Copyright may be
transferred without notice, after which this version may no longer be
accessibl
Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer
An important goal of research in Deep Reinforcement Learning in mobile
robotics is to train agents capable of solving complex tasks, which require a
high level of scene understanding and reasoning from an egocentric perspective.
When trained from simulations, optimal environments should satisfy a currently
unobtainable combination of high-fidelity photographic observations, massive
amounts of different environment configurations and fast simulation speeds. In
this paper we argue that research on training agents capable of complex
reasoning can be simplified by decoupling from the requirement of high fidelity
photographic observations. We present a suite of tasks requiring complex
reasoning and exploration in continuous, partially observable 3D environments.
The objective is to provide challenging scenarios and a robust baseline agent
architecture that can be trained on mid-range consumer hardware in under 24h.
Our scenarios combine two key advantages: (i) they are based on a simple but
highly efficient 3D environment (ViZDoom) which allows high speed simulation
(12000fps); (ii) the scenarios provide the user with a range of difficulty
settings, in order to identify the limitations of current state of the art
algorithms and network architectures. We aim to increase accessibility to the
field of Deep-RL by providing baselines for challenging scenarios where new
ideas can be iterated on quickly. We argue that the community should be able to
address challenging problems in reasoning of mobile agents without the need for
a large compute infrastructure
Gym-Ignition: Reproducible Robotic Simulations for Reinforcement Learning
This paper presents Gym-Ignition, a new framework to create reproducible
robotic environments for reinforcement learning research. It interfaces with
the new generation of Gazebo, part of the Ignition Robotics suite, which
provides three main improvements for reinforcement learning applications
compared to the alternatives: 1) the modular architecture enables using the
simulator as a C++ library, simplifying the interconnection with external
software; 2) multiple physics and rendering engines are supported as plugins,
simplifying their selection during the execution; 3) the new distributed
simulation capability allows simulating complex scenarios while sharing the
load on multiple workers and machines. The core of Gym-Ignition is a component
that contains the Ignition Gazebo simulator and exposes a simple interface for
its configuration and execution. We provide a Python package that allows
developers to create robotic environments simulated in Ignition Gazebo.
Environments expose the common OpenAI Gym interface, making them compatible
out-of-the-box with third-party frameworks containing reinforcement learning
algorithms. Simulations can be executed in both headless and GUI mode, the
physics engine can run in accelerated mode, and instances can be parallelized.
Furthermore, the Gym-Ignition software architecture provides abstraction of the
Robot and the Task, making environments agnostic on the specific runtime. This
abstraction allows their execution also in a real-time setting on actual
robotic platforms, even if driven by different middlewares.Comment: Accepted in SII202
Intersection-free Robot Manipulation with Soft-Rigid Coupled Incremental Potential Contact
This paper presents a novel simulation platform, ZeMa, designed for robotic
manipulation tasks concerning soft objects. Such simulation ideally requires
three properties: two-way soft-rigid coupling, intersection-free guarantees,
and frictional contact modeling, with acceptable runtime suitable for deep and
reinforcement learning tasks. Current simulators often satisfy only a subset of
these needs, primarily focusing on distinct rigid-rigid or soft-soft
interactions. The proposed ZeMa prioritizes physical accuracy and integrates
the incremental potential contact method, offering unified dynamics simulation
for both soft and rigid objects. It efficiently manages soft-rigid contact,
operating 75x faster than baseline tools with similar methodologies like
IPC-GraspSim. To demonstrate its applicability, we employ it for parallel grasp
generation, penetrated grasp repair, and reinforcement learning for grasping,
successfully transferring the trained RL policy to real-world scenarios
DIANNE: a modular framework for designing, training and deploying deep neural networks on heterogeneous distributed infrastructure
Deep learning has shown tremendous results on various machine learning tasks, but the nature of the problems being tackled and the size of state-of-the-art deep neural networks often require training and deploying models on distributed infrastructure. DIANNE is a modular framework designed for dynamic (re)distribution of deep learning models and procedures. Besides providing elementary network building blocks as well as various training and evaluation routines, DIANNE focuses on dynamic deployment on heterogeneous distributed infrastructure, abstraction of Internet of Things (loT) sensors, integration with external systems and graphical user interfaces to build and deploy networks, while retaining the performance of similar deep learning frameworks. In this paper the DIANNE framework is proposed as an all-in-one solution for deep learning, enabling data and model parallelism though a modular design, offloading to local compute power, and the ability to abstract between simulation and real environment. (C) 2018 Elsevier Inc. All rights reserved
- …