125 research outputs found
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Doubly Optimal No-Regret Learning in Monotone Games
We consider online learning in multi-player smooth monotone games. Existing
algorithms have limitations such as (1) being only applicable to strongly
monotone games; (2) lacking the no-regret guarantee; (3) having only asymptotic
or slow last-iterate convergence rate to a Nash
equilibrium. While the rate is tight for a large class
of algorithms including the well-studied extragradient algorithm and optimistic
gradient algorithm, it is not optimal for all gradient-based algorithms.
We propose the accelerated optimistic gradient (AOG) algorithm, the first
doubly optimal no-regret learning algorithm for smooth monotone games. Namely,
our algorithm achieves both (i) the optimal regret in the
adversarial setting under smooth and convex loss functions and (ii) the optimal
last-iterate convergence rate to a Nash equilibrium in
multi-player smooth monotone games. As a byproduct of the accelerated
last-iterate convergence rate, we further show that each player suffers only an
individual worst-case dynamic regret, providing an exponential
improvement over the previous state-of-the-art bound.Comment: Published at ICML 2023. V2 incorporates reviewers' feedbac
Algorithmic Cheap Talk
The literature on strategic communication originated with the influential
cheap talk model, which precedes the Bayesian persuasion model by three
decades. This model describes an interaction between two agents: sender and
receiver. The sender knows some state of the world which the receiver does not
know, and tries to influence the receiver's action by communicating a cheap
talk message to the receiver.
This paper initiates the algorithmic study of cheap talk in a finite
environment (i.e., a finite number of states and receiver's possible actions).
We first prove that approximating the sender-optimal or the welfare-maximizing
cheap talk equilibrium up to a certain additive constant or multiplicative
factor is NP-hard. Fortunately, we identify three naturally-restricted cases
that admit efficient algorithms for finding a sender-optimal equilibrium. These
include a state-independent sender's utility structure, a constant number of
states or a receiver having only two actions
Graphical One-Sided Markets with Exchange Costs
This paper proposes a new one-sided matching market model in which every
agent has a cost function that is allowed to take a negative value. Our model
aims to capture the situation where some agents can profit by exchanging their
obtained goods with other agents. We formulate such a model based on a
graphical one-sided matching market, introduced by Massand and Simon [Massand
and Simon, IJCAI 2019]. We investigate the existence of stable outcomes for
such a market. We prove that there is an instance that has no core-stable
allocation. On the other hand, we guarantee the existence of two-stable
allocations even where exchange costs exist. However, it is PLS-hard to find a
two-stable allocation for a market with exchange costs even if the maximum
degree of the graph is five
Zero-sum Polymatrix Markov Games: Equilibrium Collapse and Efficient Computation of Nash Equilibria
The works of (Daskalakis et al., 2009, 2022; Jin et al., 2022; Deng et al.,
2023) indicate that computing Nash equilibria in multi-player Markov games is a
computationally hard task. This fact raises the question of whether or not
computational intractability can be circumvented if one focuses on specific
classes of Markov games. One such example is two-player zero-sum Markov games,
in which efficient ways to compute a Nash equilibrium are known. Inspired by
zero-sum polymatrix normal-form games (Cai et al., 2016), we define a class of
zero-sum multi-agent Markov games in which there are only pairwise interactions
described by a graph that changes per state. For this class of Markov games, we
show that an -approximate Nash equilibrium can be found efficiently.
To do so, we generalize the techniques of (Cai et al., 2016), by showing that
the set of coarse-correlated equilibria collapses to the set of Nash
equilibria. Afterwards, it is possible to use any algorithm in the literature
that computes approximate coarse-correlated equilibria Markovian policies to
get an approximate Nash equilibrium.Comment: Added missing proofs for the infinite-horizo
On the Interplay between Social Welfare and Tractability of Equilibria
Computational tractability and social welfare (aka. efficiency) of equilibria
are two fundamental but in general orthogonal considerations in algorithmic
game theory. Nevertheless, we show that when (approximate) full efficiency can
be guaranteed via a smoothness argument \`a la Roughgarden, Nash equilibria are
approachable under a family of no-regret learning algorithms, thereby enabling
fast and decentralized computation. We leverage this connection to obtain new
convergence results in large games -- wherein the number of players
-- under the well-documented property of full efficiency via smoothness in the
limit. Surprisingly, our framework unifies equilibrium computation in disparate
classes of problems including games with vanishing strategic sensitivity and
two-player zero-sum games, illuminating en route an immediate but overlooked
equivalence between smoothness and a well-studied condition in the optimization
literature known as the Minty property. Finally, we establish that a family of
no-regret dynamics attains a welfare bound that improves over the smoothness
framework while at the same time guaranteeing convergence to the set of coarse
correlated equilibria. We show this by employing the clairvoyant mirror descent
algortihm recently introduced by Piliouras et al.Comment: To appear at NeurIPS 202
No-Regret Learning and Equilibrium Computation in Quantum Games
As quantum processors advance, the emergence of large-scale decentralized
systems involving interacting quantum-enabled agents is on the horizon. Recent
research efforts have explored quantum versions of Nash and correlated
equilibria as solution concepts of strategic quantum interactions, but these
approaches did not directly connect to decentralized adaptive setups where
agents possess limited information. This paper delves into the dynamics of
quantum-enabled agents within decentralized systems that employ no-regret
algorithms to update their behaviors over time. Specifically, we investigate
two-player quantum zero-sum games and polymatrix quantum zero-sum games,
showing that no-regret algorithms converge to separable quantum Nash equilibria
in time-average. In the case of general multi-player quantum games, our work
leads to a novel solution concept, (separable) quantum coarse correlated
equilibria (QCCE), as the convergent outcome of the time-averaged behavior
no-regret algorithms, offering a natural solution concept for decentralized
quantum systems. Finally, we show that computing QCCEs can be formulated as a
semidefinite program and establish the existence of entangled (i.e.,
non-separable) QCCEs, which cannot be approached via the current paradigm of
no-regret learning
On the Convergence of No-Regret Learning Dynamics in Time-Varying Games
Most of the literature on learning in games has focused on the restrictive
setting where the underlying repeated game does not change over time. Much less
is known about the convergence of no-regret learning algorithms in dynamic
multiagent settings. In this paper, we characterize the convergence of
optimistic gradient descent (OGD) in time-varying games. Our framework yields
sharp convergence bounds for the equilibrium gap of OGD in zero-sum games
parameterized on natural variation measures of the sequence of games, subsuming
known results for static games. Furthermore, we establish improved second-order
variation bounds under strong convexity-concavity, as long as each game is
repeated multiple times. Our results also apply to time-varying general-sum
multi-player games via a bilinear formulation of correlated equilibria, which
has novel implications for meta-learning and for obtaining refined
variation-dependent regret bounds, addressing questions left open in prior
papers. Finally, we leverage our framework to also provide new insights on
dynamic regret guarantees in static games.Comment: To appear at NeurIPS 2023; V3 incorporates reviewers' feedback and
minor correction
Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization
We propose the first, to our knowledge, loss function for approximate Nash
equilibria of normal-form games that is amenable to unbiased Monte Carlo
estimation. This construction allows us to deploy standard non-convex
stochastic optimization techniques for approximating Nash equilibria, resulting
in novel algorithms with provable guarantees. We complement our theoretical
analysis with experiments demonstrating that stochastic gradient descent can
outperform previous state-of-the-art approaches
- …