18,213 research outputs found
Deep Residual Reinforcement Learning
We revisit residual algorithms in both model-free and model-based
reinforcement learning settings. We propose the bidirectional target network
technique to stabilize residual algorithms, yielding a residual version of DDPG
that significantly outperforms vanilla DDPG in the DeepMind Control Suite
benchmark. Moreover, we find the residual algorithm an effective approach to
the distribution mismatch problem in model-based planning. Compared with the
existing TD() method, our residual-based method makes weaker assumptions
about the model and yields a greater performance boost.Comment: AAMAS 202
Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space
Combining model-based and model-free deep reinforcement learning has shown
great promise for improving sample efficiency on complex control tasks while
still retaining high performance. Incorporating imagination is a recent effort
in this direction inspired by human mental simulation of motor behavior. We
propose a learning-adaptive imagination approach which, unlike previous
approaches, takes into account the reliability of the learned dynamics model
used for imagining the future. Our approach learns an ensemble of disjoint
local dynamics models in latent space and derives an intrinsic reward based on
learning progress, motivating the controller to take actions leading to data
that improves the models. The learned models are used to generate imagined
experiences, augmenting the training set of real experiences. We evaluate our
approach on learning vision-based robotic grasping and show that it
significantly improves sample efficiency and achieves near-optimal performance
in a sparse reward environment.Comment: In: Proceedings of the Joint IEEE International Conference on
Development and Learning and on Epigenetic Robotics (ICDL-EpiRob), Oslo,
Norway, Aug. 19-22, 201
- …