Exploration in multi-agent reinforcement learning is a challenging problem,
especially in environments with sparse rewards. We propose a general method for
efficient exploration by sharing experience amongst agents. Our proposed
algorithm, called Shared Experience Actor-Critic (SEAC), applies experience
sharing in an actor-critic framework. We evaluate SEAC in a collection of
sparse-reward multi-agent environments and find that it consistently
outperforms two baselines and two state-of-the-art algorithms by learning in
fewer steps and converging to higher returns. In some harder environments,
experience sharing makes the difference between learning to solve the task and
not learning at all.Comment: 34th Conference on Neural Information Processing Systems (NeurIPS
2020), Vancouver, Canad
