14 research outputs found
Solving Structured Hierarchical Games Using Differential Backward Induction
Many real-world systems possess a hierarchical structure where a strategic
plan is forwarded and implemented in a top-down manner. Examples include
business activities in large companies or policy making for reducing the spread
during pandemics. We introduce a novel class of games that we call structured
hierarchical games (SHGs) to capture these strategic interactions. In an SHG,
each player is represented as a vertex in a multi-layer decision tree and
controls a real-valued action vector reacting to orders from its predecessors
and influencing its descendants' behaviors strategically based on its own
subjective utility. SHGs generalize extensive form games as well as Stackelberg
games. For general SHGs with (possibly) nonconvex payoffs and high-dimensional
action spaces, we propose a new solution concept which we call local subgame
perfect equilibrium. By exploiting the hierarchical structure and strategic
dependencies in payoffs, we derive a back propagation-style gradient-based
algorithm which we call Differential Backward Induction to compute an
equilibrium. We theoretically characterize the convergence properties of DBI
and empirically demonstrate a large overlap between the stable points reached
by DBI and equilibrium solutions. Finally, we demonstrate the effectiveness of
our algorithm in finding \emph{globally} stable solutions and its scalability
for a recently introduced class of SHGs for pandemic policy making
Convex-Concave Min-Max Stackelberg Games
Min-max optimization problems (i.e., min-max games) have been attracting a
great deal of attention because of their applicability to a wide range of
machine learning problems. Although significant progress has been made
recently, the literature to date has focused on games with independent strategy
sets; little is known about solving games with dependent strategy sets, which
can be characterized as min-max Stackelberg games. We introduce two first-order
methods that solve a large class of convex-concave min-max Stackelberg games,
and show that our methods converge in polynomial time. Min-max Stackelberg
games were first studied by Wald, under the posthumous name of Wald's maximin
model, a variant of which is the main paradigm used in robust optimization,
which means that our methods can likewise solve many convex robust optimization
problems. We observe that the computation of competitive equilibria in Fisher
markets also comprises a min-max Stackelberg game. Further, we demonstrate the
efficacy and efficiency of our algorithms in practice by computing competitive
equilibria in Fisher markets with varying utility structures. Our experiments
suggest potential ways to extend our theoretical results, by demonstrating how
different smoothness properties can affect the convergence rate of our
algorithms.Comment: 25 pages, 4 tables, 1 figure, Forthcoming in NeurIPS 202
LEAD: Least-Action Dynamics for Min-Max Optimization
Adversarial formulations such as generative adversarial networks (GANs) have
rekindled interest in two-player min-max games. A central obstacle in the
optimization of such games is the rotational dynamics that hinder their
convergence. Existing methods typically employ intuitive, carefully
hand-designed mechanisms for controlling such rotations. In this paper, we take
a novel approach to address this issue by casting min-max optimization as a
physical system. We leverage tools from physics to introduce LEAD (Least-Action
Dynamics), a second-order optimizer for min-max games. Next, using Lyapunov
stability theory and spectral analysis, we study LEAD's convergence properties
in continuous and discrete-time settings for bilinear games to demonstrate
linear convergence to the Nash equilibrium. Finally, we empirically evaluate
our method on synthetic setups and CIFAR-10 image generation to demonstrate
improvements over baseline methods
Zero-Sum Stochastic Stackelberg Games
Zero-sum stochastic games have found important applications in a variety of
fields, from machine learning to economics. Work on this model has primarily
focused on the computation of Nash equilibrium due to its effectiveness in
solving adversarial board and video games. Unfortunately, a Nash equilibrium is
not guaranteed to exist in zero-sum stochastic games when the payoffs at each
state are not convex-concave in the players' actions. A Stackelberg
equilibrium, however, is guaranteed to exist. Consequently, in this paper, we
study zero-sum stochastic Stackelberg games. Going beyond known existence
results for (non-stationary) Stackelberg equilibria, we prove the existence of
recursive (i.e., Markov perfect) Stackelberg equilibria (recSE) in these games,
provide necessary and sufficient conditions for a policy profile to be a recSE,
and show that recSE can be computed in (weakly) polynomial time via value
iteration. Finally, we show that zero-sum stochastic Stackelberg games can
model the problem of pricing and allocating goods across agents and time. More
specifically, we propose a zero-sum stochastic Stackelberg game whose recSE
correspond to the recursive competitive equilibria of a large class of
stochastic Fisher markets. We close with a series of experiments that showcase
how our methodology can be used to solve the consumption-savings problem in
stochastic Fisher markets.Comment: 29 pages 2 figures, Appeared in NeurIPS'2
Lower Complexity Bounds of Finite-Sum Optimization Problems: The Results and Construction
The contribution of this paper includes two aspects. First, we study the
lower bound complexity for the minimax optimization problem whose objective
function is the average of individual smooth component functions. We
consider Proximal Incremental First-order (PIFO) algorithms which have access
to gradient and proximal oracle for each individual component. We develop a
novel approach for constructing adversarial problems, which partitions the
tridiagonal matrix of classical examples into groups. This construction is
friendly to the analysis of incremental gradient and proximal oracle. With this
approach, we demonstrate the lower bounds of first-order algorithms for finding
an -suboptimal point and an -stationary point in
different settings. Second, we also derive the lower bounds of minimization
optimization with PIFO algorithms from our approach, which can cover the
results in \citep{woodworth2016tight} and improve the results in
\citep{zhou2019lower}