We consider the problem of learning fair policies in (deep) cooperative
multi-agent reinforcement learning (MARL). We formalize it in a principled way
as the problem of optimizing a welfare function that explicitly encodes two
important aspects of fairness: efficiency and equity. As a solution method, we
propose a novel neural network architecture, which is composed of two
sub-networks specifically designed for taking into account the two aspects of
fairness. In experiments, we demonstrate the importance of the two sub-networks
for fair optimization. Our overall approach is general as it can accommodate
any (sub)differentiable welfare function. Therefore, it is compatible with
various notions of fairness that have been proposed in the literature (e.g.,
lexicographic maximin, generalized Gini social welfare function, proportional
fairness). Our solution method is generic and can be implemented in various
MARL settings: centralized training and decentralized execution, or fully
decentralized. Finally, we experimentally validate our approach in various
domains and show that it can perform much better than previous methods.Comment: International Conference on Machine Learnin
