Zero-sum Polymatrix Markov Games: Equilibrium Collapse and Efficient Computation of Nash Equilibria

The works of (Daskalakis et al., 2009, 2022; Jin et al., 2022; Deng et al., 2023) indicate that computing Nash equilibria in multi-player Markov games is a computationally hard task. This fact raises the question of whether or not computational intractability can be circumvented if one focuses on specific classes of Markov games. One such example is two-player zero-sum Markov games, in which efficient ways to compute a Nash equilibrium are known. Inspired by zero-sum polymatrix normal-form games (Cai et al., 2016), we define a class of zero-sum multi-agent Markov games in which there are only pairwise interactions described by a graph that changes per state. For this class of Markov games, we show that an $\epsilon$-approximate Nash equilibrium can be found efficiently. To do so, we generalize the techniques of (Cai et al., 2016), by showing that the set of coarse-correlated equilibria collapses to the set of Nash equilibria. Afterwards, it is possible to use any algorithm in the literature that computes approximate coarse-correlated equilibria Markovian policies to get an approximate Nash equilibrium.Comment: Added missing proofs for the infinite-horizo

Approximating the set of Nash equilibria for convex games

In Feinstein and Rudloff (2023), it was shown that the set of Nash equilibria for any non-cooperative $N$ player game coincides with the set of Pareto optimal points of a certain vector optimization problem with non-convex ordering cone. To avoid dealing with a non-convex ordering cone, an equivalent characterization of the set of Nash equilibria as the intersection of the Pareto optimal points of $N$ multi-objective problems (i.e.\ with the natural ordering cone) is proven. So far, algorithms to compute the exact set of Pareto optimal points of a multi-objective problem exist only for the class of linear problems, which reduces the possibility of finding the true set of Nash equilibria by those algorithms to linear games only. In this paper, we will consider the larger class of convex games. As, typically, only approximate solutions can be computed for convex vector optimization problems, we first show, in total analogy to the result above, that the set of $\epsilon$-approximate Nash equilibria can be characterized by the intersection of $\epsilon$-approximate Pareto optimal points for $N$ convex multi-objective problems. Then, we propose an algorithm based on results from vector optimization and convex projections that allows for the computation of a set that, on one hand, contains the set of all true Nash equilibria, and is, on the other hand, contained in the set of $\epsilon$-approximate Nash equilibria. In addition to the joint convexity of the cost function for each player, this algorithm works provided the players are restricted by either shared polyhedral constraints or independent convex constraints

A probabilistic weak formulation of mean field games and applications

Mean field games are studied by means of the weak formulation of stochastic optimal control. This approach allows the mean field interactions to enter through both state and control processes and take a form which is general enough to include rank and nearest-neighbor effects. Moreover, the data may depend discontinuously on the state variable, and more generally its entire history. Existence and uniqueness results are proven, along with a procedure for identifying and constructing distributed strategies which provide approximate Nash equlibria for finite-player games. Our results are applied to a new class of multi-agent price impact models and a class of flocking models for which we prove existence of equilibria

New Algorithms for Approximate Nash Equilibria in Bimatrix Games

We consider the problem of computing additively approximate Nash equilibria in noncooperative two-player games. We provide a new polynomial time algorithm that achieves an approximation guarantee of 0.36392. We first provide a simpler algorithm, that achieves a 0.38197-approximation, which is exactly the same factor as the algorithm of Daskalakis, Mehta and Papadimitriou.This algorithm is then tuned, improving the approximation error to 0.36392. Our method is relatively fast and simple, as it requires solving only one linear program and it is based on using the solution of an auxiliary zero-sum game as a starting point. Finally we also exhibit a simple reduction that allows us to compute approximate equilibria for multi-player games by using algorithms for two-player games

Approximate Pure Nash Equilibria in Weighted Congestion Games: Existence, Efficient Computation, and Structure

We consider structural and algorithmic questions related to the Nash dynamics of weighted congestion games. In weighted congestion games with linear latency functions, the existence of (pure Nash) equilibria is guaranteed by potential function arguments. Unfortunately, this proof of existence is inefficient and computing equilibria is such games is a {\sf PLS}-hard problem. The situation gets worse when superlinear latency functions come into play; in this case, the Nash dynamics of the game may contain cycles and equilibria may not even exist. Given these obstacles, we consider approximate equilibria as alternative solution concepts. Do such equilibria exist? And if so, can we compute them efficiently? We provide positive answers to both questions for weighted congestion games with polynomial latency functions by exploiting an "approximation" of such games by a new class of potential games that we call $\Psi$-games. This allows us to show that these games have $d!$-approximate equilibria, where $d$ is the maximum degree of the latency functions. Our main technical contribution is an efficient algorithm for computing O(1)-approximate equilibria when $d$ is a constant. For games with linear latency functions, the approximation guarantee is $\frac{3+\sqrt{5}}{2}+O(\gamma)$ for arbitrarily small $\gamma>0$; for latency functions with maximum degree $d\geq 2$, it is $d^{2d+o(d)}$. The running time is polynomial in the number of bits in the representation of the game and $1/\gamma$. As a byproduct of our techniques, we also show the following structural statement for weighted congestion games with polynomial latency functions of maximum degree $d\geq 2$: polynomially-long sequences of best-response moves from any initial state to a $d^{O(d^2)}$-approximate equilibrium exist and can be efficiently identified in such games as long as $d$ is constant.Comment: 31 page

Computing Approximate Nash Equilibria in Polymatrix Games

In an $\epsilon$-Nash equilibrium, a player can gain at most $\epsilon$ by unilaterally changing his behaviour. For two-player (bimatrix) games with payoffs in $[0,1]$, the best-known$\epsilon$ achievable in polynomial time is 0.3393. In general, for $n$-player games an $\epsilon$-Nash equilibrium can be computed in polynomial time for an $\epsilon$ that is an increasing function of $n$ but does not depend on the number of strategies of the players. For three-player and four-player games the corresponding values of $\epsilon$ are 0.6022 and 0.7153, respectively. Polymatrix games are a restriction of general $n$-player games where a player's payoff is the sum of payoffs from a number of bimatrix games. There exists a very small but constant $\epsilon$ such that computing an $\epsilon$-Nash equilibrium of a polymatrix game is \PPAD-hard. Our main result is that a $(0.5+\delta)$-Nash equilibrium of an $n$-player polymatrix game can be computed in time polynomial in the input size and $\frac{1}{\delta}$. Inspired by the algorithm of Tsaknakis and Spirakis, our algorithm uses gradient descent on the maximum regret of the players. We also show that this algorithm can be applied to efficiently find a $(0.5+\delta)$-Nash equilibrium in a two-player Bayesian game