3 research outputs found
Faster no-regret learning dynamics for extensive-form correlated and coarse correlated equilibria
A recent emerging trend in the literature on learning in games has been
concerned with providing faster learning dynamics for correlated and coarse
correlated equilibria in normal-form games. Much less is known about the
significantly more challenging setting of extensive-form games, which can
capture both sequential and simultaneous moves, as well as imperfect
information. In this paper we establish faster no-regret learning dynamics for
\textit{extensive-form correlated equilibria (EFCE)} in multiplayer general-sum
imperfect-information extensive-form games. When all players follow our
accelerated dynamics, the correlated distribution of play is an
-approximate EFCE, where the notation suppresses
parameters polynomial in the description of the game. This significantly
improves over the best prior rate of . To achieve this, we develop
a framework for performing accelerated \emph{Phi-regret minimization} via
predictions. One of our key technical contributions -- that enables us to
employ our generic template -- is to characterize the stability of fixed points
associated with \emph{trigger deviation functions} through a refined
perturbation analysis of a structured Markov chain. Furthermore, for the
simpler solution concept of extensive-form \emph{coarse} correlated equilibrium
(EFCCE) we give a new succinct closed-form characterization of the associated
fixed points, bypassing the expensive computation of stationary distributions
required for EFCE. Our results place EFCCE closer to \emph{normal-form coarse
correlated equilibria} in terms of the per-iteration complexity, although the
former prescribes a much more compelling notion of correlation. Finally,
experiments conducted on standard benchmarks corroborate our theoretical
findings.Comment: Preliminary parts of this paper will appear at the AAAI-22 Workshop
on Reinforcement Learning in Games. This version also contains results from
an earlier preprint published by a subset of the authors (arXiv:2109.08138