Quasi-stationary distributions

This paper contains a survey of results related to quasi-stationary distributions, which arise in the setting of stochastic dynamical systems that eventually evanesce, and which may be useful in describing the long-term behaviour of such systems before evanescence. We are concerned mainly with continuous-time Markov chains over a finite or countably infinite state space, since these processes most often arise in applications, but will make reference to results for other processes where appropriate. Next to giving an historical account of the subject, we review the most important results on the existence and identification of quasi-stationary distributions for general Markov chains, and give special attention to birth-death processes and related models. Results on the question of whether a quasi-stationary distribution, given its existence, is indeed a good descriptor of the long-term behaviour of a system before evanescence, are reviewed as well. The paper is concluded with a summary of recent developments in numerical and approximation methods

Theoretical analysis of a Stochastic Approximation approach for computing Quasi-Stationary distributions

This paper studies a method, which has been proposed in the Physics literature by [8, 7, 10], for estimating the quasi-stationary distribution. In contrast to existing methods in eigenvector estimation, the method eliminates the need for explicit transition matrix manipulation to extract the principal eigenvector. Our paper analyzes the algorithm by casting it as a stochastic approximation algorithm (Robbins-Monro) [23, 16]. In doing so, we prove its convergence and obtain its rate of convergence. Based on this insight, we also give an example where the rate of convergence is very slow. This problem can be alleviated by using an improved version of the algorithm that is given in this paper. Numerical experiments are described that demonstrate the effectiveness of this improved method

Quasi-stationary distributions for reducible absorbing Markov chains in discrete time

We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \to \infty,$ conditional on survival up to $n$. It is known that, when $S$ is irreducible, the limiting conditional distribution of the chain equals the (unique) quasi-stationary distribution of the chain, while the latter is the (unique) $\rho$-invariant distribution for the one-step transition probability matrix of the (sub)Markov chain on $S,$ $\rho$ being the Perron-Frobenius eigenvalue of this matrix. Addressing similar issues in a setting in which $S$ may be reducible, we identify all quasi-stationary distributions and obtain a necessary and sufficient condition for one of them to be the unique $\rho$-invariant distribution. We also reveal conditions under which the limiting conditional distribution equals the $\rho$-invariant distribution if it is unique. We conclude with some examples

Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes

New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can be applied to processes with asymptotically coupled and uncoupled finite phase spaces.Comment: 83 page

Quasi-stationary distribution for multi-dimensional birth and death processes conditioned to survival of all coordinates

This article studies the quasi-stationary behaviour of multidimensional birth and death processes, modeling the interaction between several species, absorbed when one of the coordinates hits 0. We study models where the absorption rate is not uniformly bounded, contrary to most of the previous works. To handle this natural situation, we develop original Lyapunov function arguments that might apply in other situations with unbounded killing rates. We obtain the exponential convergence in total variation of the conditional distributions to a unique stationary distribution, uniformly with respect to the initial distribution. Our results cover general birth and death models with stronger intra-specific than inter-specific competition, and cases with neutral competition with explicit conditions on the dimension of the process.Comment: 18 page