Accurate calculations of stationary distributions and mean first passage times in Markov renewal processes and Markov chains

This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986) procedure. The technique is numerically stable in that it doesn't involve subtractions. Algebraic expressions for the special cases of one, two, three and four states are derived. A consequence of the procedure is that the stationary distribution of the embedded Markov chain does not need to be derived in advance but can be found accurately from the derived mean first passage times. MatLab is utilized to carry out the computations, using some test problems from the literature

The accurate computation of key properties of Markov and semi-Markov Processes

Based upon the Grassman, Taksar and Heyman algorithm [1] and the equivalent Sheskin State Reduction algorithm [2] for finding the stationary distribution of a finite irreducible Markov chain, Kohlas [3] developed a procedure for fi nding the mean fi rst passage times (MFPTs) (or absorption probabilities) in semi-Markov processes. The method is numerically stable as it doesn't involve subtraction. It works well for focussing on the MFPTs from any state to a fixed state but it is not ideally suited for a global expression for the MFPT matrix. We present a refinement of the Kohlas algorithm which we specialise to the case of Markov chains to find expressions for the MFPT matrix. A consequence of our procedure is that the stationary distribution does not need to be derived in advance but is found from the MFPTs. This also leads to an expression for the group inverse of I - P where P is the transition matrix of the embedded Markov chain. A comparison, using some test problems from the literature, with other techniques using generalised matrix inverses is also presented. References: 1] Grassman W.K., Taksar M.I., and Heyman D.P., Regenerative analysis and steady state distributions for Markov chains, Oper. Res. 33, (1985), 1107-1116. [2] Sheskin T.J., A Markov partitioning algorithm for computing steady state probabilities, Oper. Res. 33 (1985), 228-235. [3] Kohlas J. Numerical computation of mean fi rst passage times and absorption probabilities in Markov and semi-Markov models, Zeit fur Oper Res, 30, (1986), 197-207

A comparison of computational techniques of the key properties of Markov Chains

The presenter has recently been exploring the accurate computation of the stationary distribution for finite Markov chains based upon the Grassman, Taksar and Heyman (GTH) algorithm ([1]) with further extensions of this procedure, based upon the ideas of Kohlas ([2]), for finding the mean first passage time matrix. The methods are numerically stable as they do not involve subtraction. In addition, a number of perturbation techniques, where the rows of the transition matrix are sequentially updated, are also considered for computing these quantities. These techniques, together with some standard techniques using matrix inverses and generalized inverses, are compared for accuracy, using some test problems from the literature. References: [1} Grassman W.K., Taksar M.I., and Heyman D.P., Regenerative analysis and steady state distributions for Markov chains, Oper. Res. 33, (1985), 1107-1116. [2] Kohlas J. Numerical computation of mean first passage times and absorption probabilities in Markov and semi-Markov models, Zeit fur Oper Res, 30, (1986), 197-207

Measuring the impact of unconventional monetary policy on the US business cycle

The paper estimates a dynamic macroeconometric model for the US economy that captures two important features commonly observed in the study of the US business cycle, namely the strong co-movement of key macroeconomic quantities, and the distinction between expansionary and recessionary phases. The model extends the factor-augmented vector autoregressive model of Bernanke et al. (2005) by combining Markov switching with factor augmentation, modeling the Markov switching probabilities endogenously, and adopting a full Bayesian estimation approach which uses shrinkage priors for several parts of the parameter space. Exploiting a large data set for the US economy ranging from 1971:Q1 to 2014:Q2, the model is applied to measure not only the dynamic effects of unconventional monetary policy within distinct stages of the business cycle, but also the dynamic response of the recession probabilities, based on conducting counterfactual simulations. The results obtained provide new insights on the effect of monetary policy under changing business cycle phases, and highlight the importance of discriminating between expansionary and recessionary phases of the business cycle when analyzing the impact of monetary policy on the macroeconomy.Series: Working Papers in Regional Scienc

Red Light Green Light Method for Solving Large Markov Chains

Discrete-time discrete-state finite Markov chains are versatile mathematical models for a wide range of real-life stochastic processes. One of most common tasks in studies of Markov chains is computation of the stationary distribution. Without loss of generality, and drawing our motivation from applications to large networks, we interpret this problem as one of computing the stationary distribution of a random walk on a graph. We propose a new controlled, easily distributed algorithm for this task, briefly summarized as follows: at the beginning, each node receives a fixed amount of cash (positive or negative), and at each iteration, some nodes receive `green light' to distribute their wealth or debt proportionally to the transition probabilities of the Markov chain; the stationary probability of a node is computed as a ratio of the cash distributed by this a node to the total cash distributed by all nodes together. Our method includes as special cases a wide range of known, very different, and previously disconnected methods including power iterations, Gauss-Southwell, and online distributed algorithms. We prove exponential convergence of our method, demonstrate its high efficiency, and derive scheduling strategies for the green-light, that achieve convergence rate faster than state-of-the-art algorithms

Contributions to modeling and computer efficient estimation for Gaussian space -time processes

This thesis research provides several contributions to computer efficient methodology for estimation with space-time data. First we propose a parsimonious class of computer-efficient Gaussian spatial interaction models that includes as special cases CAR and SAR-like models. This extended class is capable of modeling smooth spatial random fields. We show that, for rectangular lattices, this class is equivalent to higher-order Markov random fields. Thus we capture the computational advantage of iterative updating of Markov random fields, while at the same time provide the possibility of simple interpretation of smooth spatial structure. This class of spatial models is defined via a spatial structure removing orthogonal transformation, which we propose for any spatial interaction model as a means to improve computation time. Such a transformation is a one-time preprocessing step in iterative estimation, such as in MCMC. For very large data on a rectangular lattice we can achieve further computational savings by circulant embedding which enables use of FFT for calculations. We examine how the model as well as the embedding can be incorporated in hierarchical models for space time data with spatially varying temporal trend components. We describe an application in arctic hydrology where gridded runoff fields are investigated for local trends

Aggregate matrix-analytic techniques and their applications

The complexity of computer systems affects the complexity of modeling techniques that can be used for their performance analysis. In this dissertation, we develop a set of techniques that are based on tractable analytic models and enable efficient performance analysis of computer systems. Our approach is three pronged: first, we propose new techniques to parameterize measurement data with Markovian-based stochastic processes that can be further used as input into queueing systems; second, we propose new methods to efficiently solve complex queueing models; and third, we use the proposed methods to evaluate the performance of clustered Web servers and propose new load balancing policies based on this analysis.;We devise two new techniques for fitting measurement data that exhibit high variability into Phase-type (PH) distributions. These techniques apply known fitting algorithms in a divide-and-conquer fashion. We evaluate the accuracy of our methods from both the statistics and the queueing systems perspective. In addition, we propose a new methodology for fitting measurement data that exhibit long-range dependence into Markovian Arrival Processes (MAPs).;We propose a new methodology, ETAQA, for the exact solution of M/G/1-type processes, (GI/M/1-type processes, and their intersection, i.e., quasi birth-death (QBD) processes. ETAQA computes an aggregate steady state probability distribution and a set of measures of interest. E TAQA is numerically stable and computationally superior to alternative solution methods. Apart from ETAQA, we propose a new methodology for the exact solution of a class of GI/G/1-type processes based on aggregation/decomposition.;Finally, we demonstrate the applicability of the proposed techniques by evaluating load balancing policies in clustered Web servers. We address the high variability in the service process of Web servers by dedicating the servers of a cluster to requests of similar sizes and propose new, content-aware load balancing policies. Detailed analysis shows that the proposed policies achieve high user-perceived performance and, by continuously adapting their scheduling parameters to the current workload characteristics, provide good performance under conditions of transient overload