1,043 research outputs found
A survey of generalized inverses and their use in stochastic modelling
In many stochastic models, in particular Markov chains in discrete or continuous time and Markov
renewal processes, a Markov chain is present either directly or indirectly through some form of
embedding. The analysis of many problems of interest associated with these models, eg. stationary
distributions, moments of first passage time distributions and moments of occupation time random
variables, often concerns the solution of a system of linear equations involving I – P, where P is the
transition matrix of a finite, irreducible, discrete time Markov chain.
Generalized inverses play an important role in the solution of such singular sets of equations. In this
paper we survey the application of generalized inverses to the aforementioned problems. The
presentation will include results concerning the analysis of perturbed systems and the characterization of
types of generalized inverses associated with Markovian kernels
Simple procedures for finding mean first passage times in Markov chains
The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I – P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell’s fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse I – P + eπT where eT = (1, 1, …,1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures
Bounds on expected coupling times in Markov chains
In the author’s paper “Coupling and Mixing Times in Markov Chains” (RLIMS, 11, 1-
22, 2007) it was shown that it is very difficult to find explicit expressions for the
expected time to coupling in a general Markov chain. In this paper simple upper and
lower bounds are given for the expected time to coupling in a discrete time finite
Markov chain. Extensions to the bounds under additional restrictive conditions are also
given with detailed comparisons provided for two and three state chains
Stationary distributions and mean first passage times of perturbed Markov chains
Stationary distributions of perturbed finite irreducible discrete time Markov chains are intimately
connected with the behaviour of associated mean first passage times. This interconnection is explored
through the use of generalized matrix inverses. Some interesting qualitative results regarding the nature
of the relative and absolute changes to the stationary probabilities are obtained together with some
improved bounds
Coupling and mixing times in a Markov Chains [sic]
The derivation of the expected time to coupling in a Markov chain and its relation to the
expected time to mixing (as introduced by the author in “Mixing times with applications
to perturbed Markov chains” Linear Algebra Appl. (417, 108-123 (2006)) are explored.
The two-state cases and three-state cases are examined in detail
Markovian queues with correlated arrival processes
In an attempt to examine the effect of dependencies in the arrival process on the steady state
queue length process in single server queueing models with exponential service time distribution,
four different models for the arrival process, each with marginally distributed exponential interarrivals
to the queueing system, are considered. Two of these models are based upon the upper
and lower bounding joint distribution functions given by the Fréchet bounds for bivariate
distributions with specified marginals, the third is based on Downton’s bivariate exponential
distribution and fourthly the usual M/M/1 model. The aim of the paper is to compare conditions
for stability and explore the queueing behaviour of the different models
- …