12 research outputs found
Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games
The predominant paradigm in evolutionary game theory and more generally
online learning in games is based on a clear distinction between a population
of dynamic agents that interact given a fixed, static game. In this paper, we
move away from the artificial divide between dynamic agents and static games,
to introduce and analyze a large class of competitive settings where both the
agents and the games they play evolve strategically over time. We focus on
arguably the most archetypal game-theoretic setting -- zero-sum games (as well
as network generalizations) -- and the most studied evolutionary learning
dynamic -- replicator, the continuous-time analogue of multiplicative weights.
Populations of agents compete against each other in a zero-sum competition that
itself evolves adversarially to the current population mixture. Remarkably,
despite the chaotic coevolution of agents and games, we prove that the system
exhibits a number of regularities. First, the system has conservation laws of
an information-theoretic flavor that couple the behavior of all agents and
games. Secondly, the system is Poincar\'{e} recurrent, with effectively all
possible initializations of agents and games lying on recurrent orbits that
come arbitrarily close to their initial conditions infinitely often. Thirdly,
the time-average agent behavior and utility converge to the Nash equilibrium
values of the time-average game. Finally, we provide a polynomial time
algorithm to efficiently predict this time-average behavior for any such
coevolving network game.Comment: To appear in AAAI 202
Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
Owing to their stability and convergence speed, extragradient methods have
become a staple for solving large-scale saddle-point problems in machine
learning. The basic premise of these algorithms is the use of an extrapolation
step before performing an update; thanks to this exploration step,
extra-gradient methods overcome many of the non-convergence issues that plague
gradient descent/ascent schemes. On the other hand, as we show in this paper,
running vanilla extragradient with stochastic gradients may jeopardize its
convergence, even in simple bilinear models. To overcome this failure, we
investigate a double stepsize extragradient algorithm where the exploration
step evolves at a more aggressive time-scale compared to the update step. We
show that this modification allows the method to converge even with stochastic
gradients, and we derive sharp convergence rates under an error bound
condition.Comment: In Advances in Neural Information Processing Systems 33 (NeurIPS
2020); 29 pages, 5 figure
Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds
Numerous applications in machine learning and data analytics can be
formulated as equilibrium computation over Riemannian manifolds. Despite the
extensive investigation of their Euclidean counterparts, the performance of
Riemannian gradient-based algorithms remain opaque and poorly understood. We
revisit the original scheme of Riemannian gradient descent (RGD) and analyze it
under a geodesic monotonicity assumption, which includes the well-studied
geodesically convex-concave min-max optimization problem as a special case. Our
main contribution is to show that, despite the phenomenon of distance
distortion, the RGD scheme, with a step size that is agnostic to the manifold's
curvature, achieves a curvature-independent and linear last-iterate convergence
rate in the geodesically strongly monotone setting. To the best of our
knowledge, the possibility of curvature-independent rates and/or last-iterate
convergence in the Riemannian setting has not been considered before
Adaptive extra-gradient methods for min-max optimization and games
International audienceWe present a new family of min-max optimization algorithms that automatically exploit the geometry of the gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones. Thanks to this adaptation mechanism, the proposed method automatically detects whether the problem is smooth or not, without requiring any prior tuning by the optimizer. As a result, the algorithm simultaneously achieves order-optimal convergence rates, i.e., it converges to an ε-optimal solution within O(1/ε) iterations in smooth problems, and within O(1/ε 2) iterations in non-smooth ones.Importantly, these guarantees do not require any of the standard boundedness or Lipschitz continuity conditions that are typically assumed in the literature; in particular, they apply even to problems with singularities (such as resource allocation problems and the like). This adaptation is achieved through the use of a geometric apparatus based on Finsler metrics and a suitably chosen mirror-prox template that allows us to derive sharp convergence rates for the methods at hand