946 research outputs found
Deep Q-Learning for Nash Equilibria: Nash-DQN
Model-free learning for multi-agent stochastic games is an active area of
research. Existing reinforcement learning algorithms, however, are often
restricted to zero-sum games, and are applicable only in small state-action
spaces or other simplified settings. Here, we develop a new data efficient
Deep-Q-learning methodology for model-free learning of Nash equilibria for
general-sum stochastic games. The algorithm uses a local linear-quadratic
expansion of the stochastic game, which leads to analytically solvable optimal
actions. The expansion is parametrized by deep neural networks to give it
sufficient flexibility to learn the environment without the need to experience
all state-action pairs. We study symmetry properties of the algorithm stemming
from label-invariant stochastic games and as a proof of concept, apply our
algorithm to learning optimal trading strategies in competitive electronic
markets.Comment: 16 pages, 4 figure
Mean field game model of corruption
A simple model of corruption that takes into account the effect of the
interaction of a large number of agents by both rational decision making and
myopic behavior is developed. Its stationary version turns out to be a rare
example of an exactly solvable model of mean-field-game type. The results show
clearly how the presence of interaction (including social norms) influences the
spread of corruption
Mean Field Equilibrium in Dynamic Games with Complementarities
We study a class of stochastic dynamic games that exhibit strategic
complementarities between players; formally, in the games we consider, the
payoff of a player has increasing differences between her own state and the
empirical distribution of the states of other players. Such games can be used
to model a diverse set of applications, including network security models,
recommender systems, and dynamic search in markets. Stochastic games are
generally difficult to analyze, and these difficulties are only exacerbated
when the number of players is large (as might be the case in the preceding
examples).
We consider an approximation methodology called mean field equilibrium to
study these games. In such an equilibrium, each player reacts to only the long
run average state of other players. We find necessary conditions for the
existence of a mean field equilibrium in such games. Furthermore, as a simple
consequence of this existence theorem, we obtain several natural monotonicity
properties. We show that there exist a "largest" and a "smallest" equilibrium
among all those where the equilibrium strategy used by a player is
nondecreasing, and we also show that players converge to each of these
equilibria via natural myopic learning dynamics; as we argue, these dynamics
are more reasonable than the standard best response dynamics. We also provide
sensitivity results, where we quantify how the equilibria of such games move in
response to changes in parameters of the game (e.g., the introduction of
incentives to players).Comment: 56 pages, 5 figure
Fitted Q-Learning in Mean-field Games
In the literature, existence of equilibria for discrete-time mean field games
has been in general established via Kakutani's Fixed Point Theorem. However,
this fixed point theorem does not entail any iterative scheme for computing
equilibria. In this paper, we first propose a Q-iteration algorithm to compute
equilibria for mean-field games with known model using Banach Fixed Point
Theorem. Then, we generalize this algorithm to model-free setting using fitted
Q-iteration algorithm and establish the probabilistic convergence of the
proposed iteration. Then, using the output of this learning algorithm, we
construct an approximate Nash equilibrium for finite-agent stochastic game with
mean-field interaction between agents.Comment: 22 page
- …