Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning

In multi-agent reinforcement learning, discovering successful collective behaviors is challenging as it requires exploring a joint action space that grows exponentially with the number of agents. While the tractability of independent agent-wise exploration is appealing, this approach fails on tasks that require elaborate group strategies. We argue that coordinating the agents' policies can guide their exploration and we investigate techniques to promote such an inductive bias. We propose two policy regularization methods: TeamReg, which is based on inter-agent action predictability and CoachReg that relies on synchronized behavior selection. We evaluate each approach on four challenging continuous control tasks with sparse rewards that require varying levels of coordination as well as on the discrete action Google Research Football environment. Our experiments show improved performance across many cooperative multi-agent problems. Finally, we analyze the effects of our proposed methods on the policies that our agents learn and show that our methods successfully enforce the qualities that we propose as proxies for coordinated behaviors.Comment: 23 pages, 16 figures. This revised version contains additional results and minor edit

An Inventory of Existing Neuroprivacy Controls

Brain-Computer Interfaces (BCIs) facilitate communication between brains and computers. As these devices become increasingly popular outside of the medical context, research interest in brain privacy risks and countermeasures has bloomed. Several neuroprivacy threats have been identified in the literature, including brain malware, personal data being contained in collected brainwaves and the inadequacy of legal regimes with regards to neural data protection. Dozens of controls have been proposed or implemented for protecting neuroprivacy, although it has not been immediately apparent what the landscape of neuroprivacy controls consists of. This paper inventories the implemented and proposed neuroprivacy risk mitigation techniques from open source repositories, BCI providers and the academic literature. These controls are mapped to the Hoepman privacy strategies and their implementation status is described. Several research directions for ensuring the protection of neuroprivacy are identified