35,555 research outputs found
Game-theoretical control with continuous action sets
Motivated by the recent applications of game-theoretical learning techniques
to the design of distributed control systems, we study a class of control
problems that can be formulated as potential games with continuous action sets,
and we propose an actor-critic reinforcement learning algorithm that provably
converges to equilibrium in this class of problems. The method employed is to
analyse the learning process under study through a mean-field dynamical system
that evolves in an infinite-dimensional function space (the space of
probability distributions over the players' continuous controls). To do so, we
extend the theory of finite-dimensional two-timescale stochastic approximation
to an infinite-dimensional, Banach space setting, and we prove that the
continuous dynamics of the process converge to equilibrium in the case of
potential games. These results combine to give a provably-convergent learning
algorithm in which players do not need to keep track of the controls selected
by the other agents.Comment: 19 page
Learning the Structure and Parameters of Large-Population Graphical Games from Behavioral Data
We consider learning, from strictly behavioral data, the structure and
parameters of linear influence games (LIGs), a class of parametric graphical
games introduced by Irfan and Ortiz (2014). LIGs facilitate causal strategic
inference (CSI): Making inferences from causal interventions on stable behavior
in strategic settings. Applications include the identification of the most
influential individuals in large (social) networks. Such tasks can also support
policy-making analysis. Motivated by the computational work on LIGs, we cast
the learning problem as maximum-likelihood estimation (MLE) of a generative
model defined by pure-strategy Nash equilibria (PSNE). Our simple formulation
uncovers the fundamental interplay between goodness-of-fit and model
complexity: good models capture equilibrium behavior within the data while
controlling the true number of equilibria, including those unobserved. We
provide a generalization bound establishing the sample complexity for MLE in
our framework. We propose several algorithms including convex loss minimization
(CLM) and sigmoidal approximations. We prove that the number of exact PSNE in
LIGs is small, with high probability; thus, CLM is sound. We illustrate our
approach on synthetic data and real-world U.S. congressional voting records. We
briefly discuss our learning framework's generality and potential applicability
to general graphical games.Comment: Journal of Machine Learning Research. (accepted, pending
publication.) Last conference version: submitted March 30, 2012 to UAI 2012.
First conference version: entitled, Learning Influence Games, initially
submitted on June 1, 2010 to NIPS 201
Equilibrium Computation and Robust Optimization in Zero Sum Games with Submodular Structure
We define a class of zero-sum games with combinatorial structure, where the
best response problem of one player is to maximize a submodular function. For
example, this class includes security games played on networks, as well as the
problem of robustly optimizing a submodular function over the worst case from a
set of scenarios. The challenge in computing equilibria is that both players'
strategy spaces can be exponentially large. Accordingly, previous algorithms
have worst-case exponential runtime and indeed fail to scale up on practical
instances. We provide a pseudopolynomial-time algorithm which obtains a
guaranteed -approximate mixed strategy for the maximizing player.
Our algorithm only requires access to a weakened version of a best response
oracle for the minimizing player which runs in polynomial time. Experimental
results for network security games and a robust budget allocation problem
confirm that our algorithm delivers near-optimal solutions and scales to much
larger instances than was previously possible.Comment: 20 pages, 8 figures. A shorter version of this paper appears at AAAI
201
- …