The ODE method for stability of skip-free Markov chains with applications to MCMC

Fluid limit techniques have become a central tool to analyze queueing networks over the last decade, with applications to performance analysis, simulation and optimization. In this paper, some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space and the initial condition by a large constant. The resulting fluid limit is the solution of an ordinary differential equation (ODE) in ``most'' of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similarly to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov chain Monte Carlo methods.Comment: Published in at http://dx.doi.org/10.1214/07-AAP471 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Geometric bounds for stationary distributions of infinite Markov chains via Lyapunov functions

Title from cover. "July, 1998."Includes bibliographical references (p. 18-20).Supported in part by a grant from the NSF. DMI-9610486 Supported in part by a grant from the ARO. DAAL-03-92-G-0115Dimitris Bertsimas, David Gamarnik, and John N. Tsitsiklis