71 research outputs found

    Birth-death processes with killing: orthogonal polynomials and quasi-stationary distributions

    Get PDF
    The Karlin-McGregor representation for the transition probabilities of a birth-death process with an absorbing bottom state involves a sequence of orthogonal polynomials and the corresponding measure. This representation can be generalized to a setting in which a transition to the absorbing state ({\em killing}) is possible from any state rather than just one state. The purpose of this paper is to investigate to what extent properties of birth-death processes, in particular with regard to the existence of quasi-stationary distributions, remain valid in the generalized setting. It turns out that the elegant structure of the theory of quasi-stationarity for birth-death processes remains intact as long as killing is possible from only finitely many states, but breaks down otherwise

    Quasi-stationary distributions for a class of discrete-time Markov chains

    Get PDF
    This paper is concerned with the circumstances under which a discrete-time absorbing Markov chain has a quasi-stationary distribution. We showed in a previous paper that a pure birth-death process with an absorbing bottom state has a quasi-stationary distribution -- actually an infinite family of quasi-stationary distributions -- if and only if absorption is certain and the chain is geometrically transient. If we widen the setting by allowing absorption in one step ({\it killing}) from any state, the two conditions are still necessary, but no longer sufficient. We show that the birth-death-type of behaviour prevails as long as the number of states in which killing can occur is finite. But if there are infinitely many such states, and if the chain is geometrically transient and absorption certain, then there may be 0, 1, or infinitely many quasi-stationary distributions. Examples of each type of behaviour are presented. We also survey and supplement the theory of quasi-stationary distributions for discrete-time Markov chains in general

    Orthogonal polynomials on â„ś<sup>+</sup> and birth-death processes with killing

    Get PDF

    An approximation approach for the deviation matrix of continuous-time Markov processes with application to Markov decision theory

    Get PDF
    We present an update formula that allows the expression of the deviation matrix of a continuous-time Markov process with denumerable state space having generator matrix Q* through a continuous-time Markov process with generator matrix Q. We show that under suitable stability conditions the algorithm converges at a geometric rate. By applying the concept to three different examples, namely, the M/M/1 queue with vacations, the M/G/1 queue, and a tandem network, we illustrate the broad applicability of our approach. For a problem in admission control, we apply our approximation algorithm toMarkov decision theory for computing the optimal control policy. Numerical examples are presented to highlight the efficiency of the proposed algorithm. © 2010 INFORMS

    Quasistationarity Of Continuous-Time Markov Chains With Positive Drift

    No full text
    We shall study continuous-time Markov chains on the nonnegative integers which are both irreducible and transient, and which exhibit discernible stationarity before drift to infinity &quot;sets in&quot;. We will show how this `quasi&apos; stationary behaviour can be modelled using a limiting conditional distribution: specifically, the limiting state probabilities conditional on not having left 0 for the last time. By way of a dual chain, obtained by killing the original process on last exit from 0, we invoke the theory of quasistationarity for absorbing Markov chains. We prove that the conditioned state probabilities of the original chain are equal to the state probabilities of its dual conditioned on non-absorption, thus allowing us to establish the simultaneous existence, and then equivalence, of their limiting conditional distributions. Although a limiting conditional distribution for the dual chain is always a quasistationary distribution in the usual sense, a similar statement is not possible fo..

    Condition monitoring: a new perspective

    No full text
    • …
    corecore