2,310 research outputs found
Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
Deep Reinforcement Learning (DRL) has been applied successfully to many
robotic applications. However, the large number of trials needed for training
is a key issue. Most of existing techniques developed to improve training
efficiency (e.g. imitation) target on general tasks rather than being tailored
for robot applications, which have their specific context to benefit from. We
propose a novel framework, Assisted Reinforcement Learning, where a classical
controller (e.g. a PID controller) is used as an alternative, switchable policy
to speed up training of DRL for local planning and navigation problems. The
core idea is that the simple control law allows the robot to rapidly learn
sensible primitives, like driving in a straight line, instead of random
exploration. As the actor network becomes more advanced, it can then take over
to perform more complex actions, like obstacle avoidance. Eventually, the
simple controller can be discarded entirely. We show that not only does this
technique train faster, it also is less sensitive to the structure of the DRL
network and consistently outperforms a standard Deep Deterministic Policy
Gradient network. We demonstrate the results in both simulation and real-world
experiments.Comment: Published in ICRA2018. The code is now available at
https://github.com/xie9187/AsDDP
Multi-Robot Path Planning Combining Heuristics and Multi-Agent Reinforcement Learning
Multi-robot path finding in dynamic environments is a highly challenging
classic problem. In the movement process, robots need to avoid collisions with
other moving robots while minimizing their travel distance. Previous methods
for this problem either continuously replan paths using heuristic search
methods to avoid conflicts or choose appropriate collision avoidance strategies
based on learning approaches. The former may result in long travel distances
due to frequent replanning, while the latter may have low learning efficiency
due to low sample exploration and utilization, and causing high training costs
for the model. To address these issues, we propose a path planning method,
MAPPOHR, which combines heuristic search, empirical rules, and multi-agent
reinforcement learning. The method consists of two layers: a real-time planner
based on the multi-agent reinforcement learning algorithm, MAPPO, which embeds
empirical rules in the action output layer and reward functions, and a
heuristic search planner used to create a global guiding path. During movement,
the heuristic search planner replans new paths based on the instructions of the
real-time planner. We tested our method in 10 different conflict scenarios. The
experiments show that the planning performance of MAPPOHR is better than that
of existing learning and heuristic methods. Due to the utilization of empirical
knowledge and heuristic search, the learning efficiency of MAPPOHR is higher
than that of existing learning methods
- …