339 research outputs found
DM: Decentralized Multi-Agent Reinforcement Learning for Distribution Matching
Current approaches to multi-agent cooperation rely heavily on centralized
mechanisms or explicit communication protocols to ensure convergence. This
paper studies the problem of distributed multi-agent learning without resorting
to centralized components or explicit communication. It examines the use of
distribution matching to facilitate the coordination of independent agents. In
the proposed scheme, each agent independently minimizes the distribution
mismatch to the corresponding component of a target visitation distribution.
The theoretical analysis shows that under certain conditions, each agent
minimizing its individual distribution mismatch allows the convergence to the
joint policy that generated the target distribution. Further, if the target
distribution is from a joint policy that optimizes a cooperative task, the
optimal policy for a combination of this task reward and the distribution
matching reward is the same joint policy. This insight is used to formulate a
practical algorithm (DM), in which each individual agent matches a target
distribution derived from concurrently sampled trajectories from a joint expert
policy. Experimental validation on the StarCraft domain shows that combining
(1) a task reward, and (2) a distribution matching reward for expert
demonstrations for the same task, allows agents to outperform a naive
distributed baseline. Additional experiments probe the conditions under which
expert demonstrations need to be sampled to obtain the learning benefits
Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments
The Game Theory & Multi-Agent team at DeepMind studies several aspects of
multi-agent learning ranging from computing approximations to fundamental
concepts in game theory to simulating social dilemmas in rich spatial
environments and training 3-d humanoids in difficult team coordination tasks. A
signature aim of our group is to use the resources and expertise made available
to us at DeepMind in deep reinforcement learning to explore multi-agent systems
in complex environments and use these benchmarks to advance our understanding.
Here, we summarise the recent work of our team and present a taxonomy that we
feel highlights many important open challenges in multi-agent research.Comment: Published in AI Communications 202
- …