Birth-death processes with killing: orthogonal polynomials and quasi-stationary distributions

The Karlin-McGregor representation for the transition probabilities of a birth-death process with an absorbing bottom state involves a sequence of orthogonal polynomials and the corresponding measure. This representation can be generalized to a setting in which a transition to the absorbing state ({\em killing}) is possible from any state rather than just one state. The purpose of this paper is to investigate to what extent properties of birth-death processes, in particular with regard to the existence of quasi-stationary distributions, remain valid in the generalized setting. It turns out that the elegant structure of the theory of quasi-stationarity for birth-death processes remains intact as long as killing is possible from only finitely many states, but breaks down otherwise

Weak convergence of conditioned birth-death processes in discrete time

We consider a discrete-time birth-death process on the nonnegative integers with -1 as an absorbing state and study the limiting behaviour as $n \to \infty$ of the process conditioned on nonabsorption until time $n$. By proving that a condition recently proposed by Martinez and Vares is vacuously true, we establish that the conditioned process is always weakly convergent when all self-transition probabilities are zero. In the aperiodic case we obtain a necessary and sufficient condition for weak convergence

Quasi-stationary distributions for birth-death processes with killing

The Karlin-McGregor representation for the transition probabilities of a birth-death process with an absorbing bottom state involves a sequence of orthogonal polynomials and the corresponding measure. This representation can be generalized to a setting in which a transition to the absorbing state (killing) is possible from any state rather than just one state. The purpose of this paper is to investigate to what extent properties of birth-death processes, in particular with regard to the existence of quasi-stationary distributions, remain valid in the generalized setting. It turns out that the elegant structure of the theory of quasi-stationarity for birth-death processes remains largely intact as long as killing is possible from only finitely many states. In particular, the existence of a quasi-stationary distribution is ensured in this case if absorption is certain and the state probabilities tend to zero exponentially fast

Quasi-stationary distributions for a class of discrete-time Markov chains

This paper is concerned with the circumstances under which a discrete-time absorbing Markov chain has a quasi-stationary distribution. We showed in a previous paper that a pure birth-death process with an absorbing bottom state has a quasi-stationary distribution -- actually an infinite family of quasi-stationary distributions -- if and only if absorption is certain and the chain is geometrically transient. If we widen the setting by allowing absorption in one step ({\it killing}) from any state, the two conditions are still necessary, but no longer sufficient. We show that the birth-death-type of behaviour prevails as long as the number of states in which killing can occur is finite. But if there are infinitely many such states, and if the chain is geometrically transient and absorption certain, then there may be 0, 1, or infinitely many quasi-stationary distributions. Examples of each type of behaviour are presented. We also survey and supplement the theory of quasi-stationary distributions for discrete-time Markov chains in general

An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory

We present an update formula that allows the expression of the deviation matrix of a continuous-time Markov process with denumerable state space having generator matrix Q* through a continuous-time Markov process with generator matrix Q. We show that under suitable stability conditions the algorithm converges at a geometric rate. By applying the concept to three different examples, namely, the M/M/1 queue with vacations, the M/G/1 queue, and a tandem network, we illustrate the broad applicability of our approach. For a problem in admission control, we apply our approximation algorithm toMarkov decision theory for computing the optimal control policy. Numerical examples are presented to highlight the efficiency of the proposed algorithm. © 2010 INFORMS

The maximum number of infected individuals in SIS epidemic models: Computational techniques and quasi-stationary distributions

We study the maximumn umber of infected individuals observed during an epidemic for a Susceptible–Infected–Susceptible (SIS) model which corresponds to a birth–death process with an absorbing state. We develop computational schemes for the corresponding distributions in a transient regime and till absorption. Moreover, we study the distribution of the current number of infectedindividuals given that the maximum number during the epidemic has not exceeded a given threshold. In this sense, some quasi-stationary distributions of a related process are also discussed