16,727 research outputs found
Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning
Humans learn by interacting with their environments and perceiving the
outcomes of their actions. A landmark in artificial intelligence has been the
development of deep reinforcement learning (dRL) algorithms capable of doing
the same in video games, on par with or better than humans. However, it remains
unclear whether the successes of dRL models reflect advances in visual
representation learning, the effectiveness of reinforcement learning algorithms
at discovering better policies, or both. To address this question, we introduce
the Learning Challenge Diagnosticator (LCD), a tool that separately measures
the perceptual and reinforcement learning demands of a task. We use LCD to
discover a novel taxonomy of challenges in the Procgen benchmark, and
demonstrate that these predictions are both highly reliable and can instruct
algorithmic development. More broadly, the LCD reveals multiple failure cases
that can occur when optimizing dRL algorithms over entire video game benchmarks
like Procgen, and provides a pathway towards more efficient progress
Solving Common-Payoff Games with Approximate Policy Iteration
For artificially intelligent learning systems to have widespread
applicability in real-world settings, it is important that they be able to
operate decentrally. Unfortunately, decentralized control is difficult --
computing even an epsilon-optimal joint policy is a NEXP complete problem.
Nevertheless, a recently rediscovered insight -- that a team of agents can
coordinate via common knowledge -- has given rise to algorithms capable of
finding optimal joint policies in small common-payoff games. The Bayesian
action decoder (BAD) leverages this insight and deep reinforcement learning to
scale to games as large as two-player Hanabi. However, the approximations it
uses to do so prevent it from discovering optimal joint policies even in games
small enough to brute force optimal solutions. This work proposes CAPI, a novel
algorithm which, like BAD, combines common knowledge with deep reinforcement
learning. However, unlike BAD, CAPI prioritizes the propensity to discover
optimal joint policies over scalability. While this choice precludes CAPI from
scaling to games as large as Hanabi, empirical results demonstrate that, on the
games to which CAPI does scale, it is capable of discovering optimal joint
policies even when other modern multi-agent reinforcement learning algorithms
are unable to do so. Code is available at https://github.com/ssokota/capi .Comment: AAAI 202
- …