Stationary Anonymous Sequential Games with Undiscounted Rewards

International audienceStationary anonymous sequential games with undiscounted rewards are a special class of games that combines features from both population games (in nitely many players) with stochastic games.We extend the theory for these games to the cases of total expected reward as well as to the expected average reward. We show that equilibria in the anonymous sequential game correspond to the limits of equilibria of related nite population games as the number of players grows to in nity. We provide examples to illustrate our results

Applications of Stationary Anonymous Sequential Games to Multiple Access Control in Wireless Communications

International audienceWe consider in this paper dynamic Multiple Access (MAC) games between a random number of players competing over collision channels. Each of several mobiles involved in an interaction determines whether to transmit at a high or at a low power. High power decreases the lifetime of the battery but results in smaller collision probability. We formulate this game as an anonymous sequential game with undiscounted reward which we recently introduced and which combines features from both population games (infinitely many players) and stochastic games. We briefly present this class of games and basic equilibrium existence results for the total expected reward as well as for the expected average reward. We then apply the theory in the MAC game

Fitted Q-Learning in Mean-field Games

In the literature, existence of equilibria for discrete-time mean field games has been in general established via Kakutani's Fixed Point Theorem. However, this fixed point theorem does not entail any iterative scheme for computing equilibria. In this paper, we first propose a Q-iteration algorithm to compute equilibria for mean-field games with known model using Banach Fixed Point Theorem. Then, we generalize this algorithm to model-free setting using fitted Q-iteration algorithm and establish the probabilistic convergence of the proposed iteration. Then, using the output of this learning algorithm, we construct an approximate Nash equilibrium for finite-agent stochastic game with mean-field interaction between agents.Comment: 22 page

State-Policy Dynamics in Evolutionary Games

International audienceStandard Evolutionary Game Theory framework is a useful tool to study large interacting systems and to understand the strategic behavior of individuals in such complex systems. Adding an individual state to model local feature of each player in this context, allows one to study a wider range of problems in various application areas as networking, biology, etc. In this paper, we introduce such an extension of evolutionary game framework and particularly, we focus on the dynamical aspects of this system. Precisely, we study the coupled dynamics of the policies and the individual states inside a population of interacting individuals. We first define a general model by coupling replicator dynamics and continuous-time Markov Decision Processes and we then consider a particular case of a two policies and two states evolutionary game. We first obtain a system of combined dynamics and we show that the rest-points of this system are equilibria profiles of our evolutionary game with individual state dynamics. Second, by assuming two different time scales between states and policies dynamics, we can compute explicitly the equilibria. Then, by transforming our evolutionary game with individual states into a standard evolutionary game, we obtain an equilibrium profile which is equivalent , in terms of occupation measures and expected fitness to the previous one. All our results are illustrated with numerical analysis

Stationary anonymous sequential games with undiscounted rewards

Stationary anonymous sequential games with undiscounted rewards are a special class of games that combines features from both population games (infinitely many players) with stochastic games. We extend the theory for these games to the cases of total expected cost as well as to the expected average cost. We show that equilibria in the anonymous sequential game correspond to the limit of equilibria of related finite population games as the number of players grow to infinity. We provide many examples to illustrate our results