In the literature, existence of equilibria for discrete-time mean field games
has been in general established via Kakutani's Fixed Point Theorem. However,
this fixed point theorem does not entail any iterative scheme for computing
equilibria. In this paper, we first propose a Q-iteration algorithm to compute
equilibria for mean-field games with known model using Banach Fixed Point
Theorem. Then, we generalize this algorithm to model-free setting using fitted
Q-iteration algorithm and establish the probabilistic convergence of the
proposed iteration. Then, using the output of this learning algorithm, we
construct an approximate Nash equilibrium for finite-agent stochastic game with
mean-field interaction between agents.Comment: 22 page