2 research outputs found
The ODE method for stability of skip-free Markov chains with applications to MCMC
Fluid limit techniques have become a central tool to analyze queueing
networks over the last decade, with applications to performance analysis,
simulation and optimization. In this paper, some of these techniques are
extended to a general class of skip-free Markov chains. As in the case of
queueing models, a fluid approximation is obtained by scaling time, space and
the initial condition by a large constant. The resulting fluid limit is the
solution of an ordinary differential equation (ODE) in ``most'' of the state
space. Stability and finer ergodic properties for the stochastic model then
follow from stability of the set of fluid limits. Moreover, similarly to the
queueing context where fluid models are routinely used to design control
policies, the structure of the limiting ODE in this general setting provides an
understanding of the dynamics of the Markov chain. These results are
illustrated through application to Markov chain Monte Carlo methods.Comment: Published in at http://dx.doi.org/10.1214/07-AAP471 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org