2 research outputs found
Evaluating the Rainbow DQN Agent in Hanabi with Unseen Partners
Hanabi is a cooperative game that challenges exist-ing AI techniques due to
its focus on modeling the mental states ofother players to interpret and
predict their behavior. While thereare agents that can achieve near-perfect
scores in the game byagreeing on some shared strategy, comparatively little
progresshas been made in ad-hoc cooperation settings, where partnersand
strategies are not known in advance. In this paper, we showthat agents trained
through self-play using the popular RainbowDQN architecture fail to cooperate
well with simple rule-basedagents that were not seen during training and,
conversely, whenthese agents are trained to play with any individual
rule-basedagent, or even a mix of these agents, they fail to achieve
goodself-play scores
Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Agents that interact with other agents often do not know a priori what the
other agents' strategies are, but have to maximise their own online return
while interacting with and learning about others. The optimal adaptive
behaviour under uncertainty over the other agents' strategies w.r.t. some prior
can in principle be computed using the Interactive Bayesian Reinforcement
Learning framework. Unfortunately, doing so is intractable in most settings,
and existing approximation methods are restricted to small tasks. To overcome
this, we propose to meta-learn approximate belief inference and Bayes-optimal
behaviour for a given prior. To model beliefs over other agents, we combine
sequential and hierarchical Variational Auto-Encoders, and meta-train this
inference model alongside the policy. We show empirically that our approach
outperforms existing methods that use a model-free approach, sample from the
approximate posterior, maintain memory-free models of others, or do not fully
utilise the known structure of the environment