Skip to main content
Article thumbnail
Location of Repository

Reduced System Algorithms for Markov Chains

By Ram Lal and U. Narayan Bhat


A reduced system is a smaller system derived in the process of analyzing a larger system. In solving for steady state probabilities of a Markov chain, generally the solution can be found by first solving a reduced system of equations which is obtained by appropriately partitioning the transition probability (or rate) matrix. Following Lal (Lal, R. 1981. A unified study of algorithms for steady state probabilities in Makov chains. Ph.D. Dissertation, Department of Operations Research and Engineering Management, School of Engineering and Applied Sciences, Southern Methodist University, Dallas, TX 75275.), a Markov chain can be categorized as standard or nonstandard depending on the location of an invertible submatrix necessary for an efficient solution in a transition probability (or rate) matrix. In this paper, algorithms for the determination of steady state probabilities are developed by using (i) a backward recursion which is efficient for standard systems and (ii) a forward recursion which is efficient for nonstandard systems. It is also shown that the backward recursion can be used for finding the first passage time distribution and its mean and variance.reduced system, steady state probabilities, queueing theory

DOI identifier: 10.1287/mnsc.34.10.1202
OAI identifier:
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • (external link)
  • Suggested articles

    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.