2 research outputs found
Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
Parameterised actions in reinforcement learning are composed of discrete
actions with continuous action-parameters. This provides a framework for
solving complex domains that require combining high-level actions with flexible
control. The recent P-DQN algorithm extends deep Q-networks to learn over such
action spaces. However, it treats all action-parameters as a single joint input
to the Q-network, invalidating its theoretical foundations. We analyse the
issues with this approach and propose a novel method, multi-pass deep
Q-networks, or MP-DQN, to address them. We empirically demonstrate that MP-DQN
significantly outperforms P-DQN and other previous algorithms in terms of data
efficiency and converged policy performance on the Platform, Robot Soccer Goal,
and Half Field Offense domains.Comment: 8 pages, 4 figure
Deep Imitation Learning for Bimanual Robotic Manipulation
We present a deep imitation learning framework for robotic bimanual
manipulation in a continuous state-action space. A core challenge is to
generalize the manipulation skills to objects in different locations. We
hypothesize that modeling the relational information in the environment can
significantly improve generalization. To achieve this, we propose to (i)
decompose the multi-modal dynamics into elemental movement primitives, (ii)
parameterize each primitive using a recurrent graph neural network to capture
interactions, and (iii) integrate a high-level planner that composes primitives
sequentially and a low-level controller to combine primitive dynamics and
inverse kinematics control. Our model is a deep, hierarchical, modular
architecture. Compared to baselines, our model generalizes better and achieves
higher success rates on several simulated bimanual robotic manipulation tasks.
We open source the code for simulation, data, and models at:
https://github.com/Rose-STL-Lab/HDR-IL