3,043 research outputs found
Hierarchical Imitation and Reinforcement Learning
We study how to effectively leverage expert feedback to learn sequential
decision-making policies. We focus on problems with sparse rewards and long
time horizons, which typically pose significant challenges in reinforcement
learning. We propose an algorithmic framework, called hierarchical guidance,
that leverages the hierarchical structure of the underlying problem to
integrate different modes of expert interaction. Our framework can incorporate
different combinations of imitation learning (IL) and reinforcement learning
(RL) at different levels, leading to dramatic reductions in both expert effort
and cost of exploration. Using long-horizon benchmarks, including Montezuma's
Revenge, we demonstrate that our approach can learn significantly faster than
hierarchical RL, and be significantly more label-efficient than standard IL. We
also theoretically analyze labeling cost for certain instantiations of our
framework.Comment: Proceedings of the 35th International Conference on Machine Learning
(ICML 2018
A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racing
Autonomous racing with scaled race cars has gained increasing attention as an
effective approach for developing perception, planning and control algorithms
for safe autonomous driving at the limits of the vehicle's handling. To train
agile control policies for autonomous racing, learning-based approaches largely
utilize reinforcement learning, albeit with mixed results. In this study, we
benchmark a variety of imitation learning policies for racing vehicles that are
applied directly or for bootstrapping reinforcement learning both in simulation
and on scaled real-world environments. We show that interactive imitation
learning techniques outperform traditional imitation learning methods and can
greatly improve the performance of reinforcement learning policies by
bootstrapping thanks to its better sample efficiency. Our benchmarks provide a
foundation for future research on autonomous racing using Imitation Learning
and Reinforcement Learning
- …