A Mean Field Approach for Optimization in Particles Systems and Applications

This paper investigates the limit behavior of Markov Decision Processes (MDPs) made of independent particles evolving in a common environment, when the number of particles goes to infinity. In the finite horizon case or with a discounted cost and an infinite horizon, we show that when the number of particles becomes large, the optimal cost of the system converges almost surely to the optimal cost of a discrete deterministic system (the ``optimal mean field''). Convergence also holds for optimal policies. We further provide insights on the speed of convergence by proving several central limits theorems for the cost and the state of the Markov decision process with explicit formulas for the variance of the limit Gaussian laws. Then, our framework is applied to a brokering problem in grid computing. The optimal policy for the limit deterministic system is computed explicitly. Several simulations with growing numbers of processors are reported. They compare the performance of the optimal policy of the limit system used in the finite case with classical policies (such as Join the Shortest Queue) by measuring its asymptotic gain as well as the threshold above which it starts outperforming classical policies

Accuracy of simulations for stochastic dynamic models

This paper provides a general framework for the simulation of stochastic dynamic models. Our analysis rests upon a continuity property of invariant distributions and a generalized law of large numbers. We then establish that the simulated moments from numerical approximations converge to their exact values as the approximation errors of the computed solutions converge to zero. These asymptotic results are of further interest in the comparative study of dynamic solutions, model estimation, and derivation of error bounds for the simulated moments

Recursive equilibria in an Aiyagari style economy with permanent income shocks

In this paper, we prove the existence of a recursive competitive equilibrium (RCE) for an Aiyagari style economy with permanent income shocks and perpetual youth structure. We show that there exist equilibria where borrowing constraints are never binding. This allows us to establish a non-trivial lower bound on the equilibrium interest rate. To solve the individual’s problem, we present a new approach that uses lattices of consumption functions to deal with the non-compact state space and the unbounded utility function. The approach uses only the first order conditions of the problem (Euler equations). The proof is constructive and it serves as a theoretical foundation for the convergence of a policy function iteration procedure.Permanent income shocks; incomplete markets; dynamic general equilibrium; heterogeneous agents

A Weak Dynamic Programming Principle for Combined Optimal Stopping and Stochastic Control with Ef\mathcal{E}^f- expectations

We study a combined optimal control/stopping problem under a nonlinear expectation ${\cal E}^f$ induced by a BSDE with jumps, in a Markovian framework. The terminal reward function is only supposed to be Borelian. The value function $u$ associated with this problem is generally irregular. We first establish a {\em sub- (resp. super-) optimality principle of dynamic programming} involving its {\em upper- (resp. lower-) semicontinuous envelope} $u^*$ (resp. $u_*$). This result, called {\em weak} dynamic programming principle (DPP), extends that obtained in \cite{BT} in the case of a classical expectation to the case of an ${\cal E}^f$-expectation and Borelian terminal reward function. Using this {\em weak} DPP, we then prove that $u^*$ (resp. $u_*$) is a {\em viscosity sub- (resp. super-) solution} of a nonlinear Hamilton-Jacobi-Bellman variational inequality