In this paper, we review the question of which action space is best suited
for controlling a real biped robot in combination with Sim2Real training.
Position control has been popular as it has been shown to be more sample
efficient and intuitive to combine with other planning algorithms. However, for
position control gain tuning is required to achieve the best possible policy
performance. We show that instead, using a torque-based action space enables
task-and-robot agnostic learning with less parameter tuning and mitigates the
sim-to-reality gap by taking advantage of torque control's inherent compliance.
Also, we accelerate the torque-based-policy training process by pre-training
the policy to remain upright by compensating for gravity. The paper showcases
the first successful sim-to-real transfer of a torque-based deep reinforcement
learning policy on a real human-sized biped robot. The video is available at
https://youtu.be/CR6pTS39VRE