1 research outputs found
Quasi-Newton Trust Region Policy Optimization
We propose a trust region method for policy optimization that employs
Quasi-Newton approximation for the Hessian, called Quasi-Newton Trust Region
Policy Optimization QNTRPO. Gradient descent is the de facto algorithm for
reinforcement learning tasks with continuous controls. The algorithm has
achieved state-of-the-art performance when used in reinforcement learning
across a wide range of tasks. However, the algorithm suffers from a number of
drawbacks including: lack of stepsize selection criterion, and slow
convergence. We investigate the use of a trust region method using dogleg step
and a Quasi-Newton approximation for the Hessian for policy optimization. We
demonstrate through numerical experiments over a wide range of challenging
continuous control tasks that our particular choice is efficient in terms of
number of samples and improves performanceComment: 3rd Conference on Robot Learning (CoRL 2019