Modular Reinforcement Learning (RL) decentralizes the control of multi-joint
robots by learning policies for each actuator. Previous work on modular RL has
proven its ability to control morphologically different agents with a shared
actuator policy. However, with the increase in the Degree of Freedom (DoF) of
robots, training a morphology-generalizable modular controller becomes
exponentially difficult. Motivated by the way the human central nervous system
controls numerous muscles, we propose a Synergy-Oriented LeARning (SOLAR)
framework that exploits the redundant nature of DoF in robot control. Actuators
are grouped into synergies by an unsupervised learning method, and a synergy
action is learned to control multiple actuators in synchrony. In this way, we
achieve a low-rank control at the synergy level. We extensively evaluate our
method on a variety of robot morphologies, and the results show its superior
efficiency and generalizability, especially on robots with a large DoF like
Humanoids++ and UNIMALs.Comment: 36th Conference on Neural Information Processing Systems (NeurIPS
2022