Stochastic Games : recent results

Nous présentons des résultats récents sur les jeux stochastiques finis. Ce texte est à paraître dans le Handbook of Game Theory, vol 3., eds. R.J. Aumann et S. HartJeux stochastiques

Hypergraph conditions for the solvability of the ergodic equation for zero-sum games

The ergodic equation is a basic tool in the study of mean-payoff stochastic games. Its solvability entails that the mean payoff is independent of the initial state. Moreover, optimal stationary strategies are readily obtained from its solution. In this paper, we give a general sufficient condition for the solvability of the ergodic equation, for a game with finite state space but arbitrary action spaces. This condition involves a pair of directed hypergraphs depending only on the ``growth at infinity'' of the Shapley operator of the game. This refines a recent result of the authors which only applied to games with bounded payments, as well as earlier nonlinear fixed point results for order preserving maps, involving graph conditions.Comment: 6 pages, 1 figure, to appear in Proc. 54th IEEE Conference on Decision and Control (CDC 2015

Two-player games : a reduction

The general idea of the proof is to define a class of sets, the solvable sets, which can safely be thought of as absorbing states.stochastic games; recursive games

Limit Your Consumption! Finding Bounds in Average-energy Games

Energy games are infinite two-player games played in weighted arenas with quantitative objectives that restrict the consumption of a resource modeled by the weights, e.g., a battery that is charged and drained. Typically, upper and/or lower bounds on the battery capacity are part of the problem description. Here, we consider the problem of determining upper bounds on the average accumulated energy or on the capacity while satisfying a given lower bound, i.e., we do not determine whether a given bound is sufficient to meet the specification, but if there exists a sufficient bound to meet it. In the classical setting with positive and negative weights, we show that the problem of determining the existence of a sufficient bound on the long-run average accumulated energy can be solved in doubly-exponential time. Then, we consider recharge games: here, all weights are negative, but there are recharge edges that recharge the energy to some fixed capacity. We show that bounding the long-run average energy in such games is complete for exponential time. Then, we consider the existential version of the problem, which turns out to be solvable in polynomial time: here, we ask whether there is a recharge capacity that allows the system player to win the game. We conclude by studying tradeoffs between the memory needed to implement strategies and the bounds they realize. We give an example showing that memory can be traded for bounds and vice versa. Also, we show that increasing the capacity allows to lower the average accumulated energy.Comment: In Proceedings QAPL'16, arXiv:1610.0769

A Probabilistic Approach to Mean Field Games with Major and Minor Players

We propose a new approach to mean field games with major and minor players. Our formulation involves a two player game where the optimization of the representative minor player is standard while the major player faces an optimization over conditional McKean-Vlasov stochastic differential equations. The definition of this limiting game is justified by proving that its solution provides approximate Nash equilibriums for large finite player games. This proof depends upon the generalization of standard results on the propagation of chaos to conditional dynamics. Because it is on independent interest, we prove this generalization in full detail. Using a conditional form of the Pontryagin stochastic maximum principle (proven in the appendix), we reduce the solution of the mean field game to a forward-backward system of stochastic differential equations of the conditional McKean-Vlasov type, which we solve in the Linear Quadratic setting. We use this class of models to show that Nash equilibriums in our formulation can be different from those of the formulations contemplated so far in the literature