20 research outputs found
Fitted Q-Learning in Mean-field Games
In the literature, existence of equilibria for discrete-time mean field games
has been in general established via Kakutani's Fixed Point Theorem. However,
this fixed point theorem does not entail any iterative scheme for computing
equilibria. In this paper, we first propose a Q-iteration algorithm to compute
equilibria for mean-field games with known model using Banach Fixed Point
Theorem. Then, we generalize this algorithm to model-free setting using fitted
Q-iteration algorithm and establish the probabilistic convergence of the
proposed iteration. Then, using the output of this learning algorithm, we
construct an approximate Nash equilibrium for finite-agent stochastic game with
mean-field interaction between agents.Comment: 22 page
Mean Field Markov Decision Processes
We consider mean-field control problems in discrete time with discounted
reward, infinite time horizon and compact state and action space. The existence
of optimal policies is shown and the limiting mean-field problem is derived
when the number of individuals tends to infinity. Moreover, we consider the
average reward problem and show that the optimal policy in this mean-field
limit is -optimal for the discounted problem if the number of
individuals is large and the discount factor close to one. This result is very
helpful, because it turns out that in the special case when the reward does
only depend on the distribution of the individuals, we obtain a very
interesting subclass of problems where an average reward optimal policy can be
obtained by first computing an optimal measure from a static optimization
problem and then achieving it with Markov Chain Monte Carlo methods. We give
two applications: Avoiding congestion an a graph and optimal positioning on a
market place which we solve explicitly