Block SOR preconditioned projection methods for Kronecker structured Markovian representations

Kronecker structured representations are used to cope with the state space explosion problem in Markovian modeling and analysis. Currently, an open research problem is that of devising strong preconditioners to be used with projection methods for the computation of the stationary vector of Markov chains (MCs) underlying such representations. This paper proposes a block successive overrelaxation (BSOR) preconditioner for hierarchical Markovian models (HMMs1) that are composed of multiple low-level models and a high-level model that defines the interaction among low-level models. The Kronecker structure of an HMM yields nested block partitionings in its underlying continuous-time MC which may be used in the BSOR preconditioner. The computation of the BSOR preconditioned residual in each iteration of a preconditioned projection method becomes the problem of solving multiple nonsingular linear systems whose coefficient matrices are the diagonal blocks of the chosen partitioning. The proposed BSOR preconditioner solves these systems using sparse LU or real Schur factors of diagonal blocks. The fill-in of sparse LU factorized diagonal blocks is reduced using the column approximate minimum degree (COLAMD) ordering. A set of numerical experiments is presented to show the merits of the proposed BSOR preconditioner. © 2005 Society for Industrial and Applied Mathematics

Block SOR Preconditional Projection Methods for Kronecker Structured Markovian Representations

Kronecker structured representations are used to cope with the state space explosion problem in Markovian modeling and analysis. Currently an open research problem is that of devising strong preconditioners to be used with projection methods for the computation of the stationary vector of Markov chains (MCs) underlying such representations. This paper proposes a block SOR (BSOR) preconditioner for hierarchical Markovian Models (HMMs) that are composed of multiple low level models and a high level model that defines the interaction among low level models. The Kronecker structure of an HMM yields nested block partitionings in its underlying continuous-time MC which may be used in the BSOR preconditioner. The computation of the BSOR preconditioned residual in each iteration of a preconditioned projection method becoms the problem of solving multiple nonsingular linear systems whose coefficient matrices are the diagonal blocks of the chosen partitioning. The proposed BSOR preconditioner solvers these systems using sparse LU or real Schur factors of diagonal blocks. The fill-in of sparse LU factorized diagonal blocks is reduced using the column approximate minimum degree algorithm (COLAMD). A set of numerical experiments are presented to show the merits of the proposed BSOR preconditioner

Comparison of multilevel methods for kronecker-based Markovian representations

The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates

Decompositional analysis of Kronecker structured Markov chains

This contribution proposes a decompositional iterative method with low memory requirements for the steadystate analysis ofKronecker structured Markov chains. The Markovian system is formed by a composition of subsystems using the Kronecker sum operator for local transitions and the Kronecker product operator for synchronized transitions. Even though the interactions among subsystems, which are captured by synchronized transitions, need not be weak, numerical experiments indicate that the solver benefits considerably from weak interactions among subsystems, and is to be recommended specifically in this case. © 2008, Kent State University

Block SOR for Kronecker structured representations

Cataloged from PDF version of article.The Kronecker structure of a hierarchical Markovian model (HMM) induces nested block partitionings in the transition matrix of its underlying Markov chain. This paper shows how sparse real Schur factors of certain diagonal blocks of a given partitioning induced by the Kronecker structure can be constructed from smaller component matrices and their real Schur factors. Furthermore, it shows how the column approximate minimum degree (COLAMD) ordering algorithm can be used to reduce fill-in of the remaining diagonal blocks that are sparse LU factorized. Combining these ideas, the paper proposes three-level block successive over-relaxation (BSOR) as a competitive steady state solver for HMMs. Finally, on a set of numerical experiments it demonstrates how these ideas reduce storage required by the factors of the diagonal blocks and improve solution time compared to an all LU factorization implementation of the BSOR solver. © 2004 Elsevier Inc. All rights reserved

On compact solution vectors in Kronecker-based Markovian analysis

State based analysis of stochastic models for performance and dependability often requires the computation of the stationary distribution of a multidimensional continuous-time Markov chain (CTMC). The infinitesimal generator underlying a multidimensional CTMC with a large reachable state space can be represented compactly in the form of a block matrix in which each nonzero block is expressed as a sum of Kronecker products of smaller matrices. However, solution vectors used in the analysis of such Kronecker-based Markovian representations require memory proportional to the size of the reachable state space. This implies that memory allocated to solution vectors becomes a bottleneck as the size of the reachable state space increases. Here, it is shown that the hierarchical Tucker decomposition (HTD) can be used with adaptive truncation strategies to store the solution vectors during Kronecker-based Markovian analysis compactly and still carry out the basic operations including vector–matrix multiplication in Kronecker form within Power, Jacobi, and Generalized Minimal Residual methods. Numerical experiments on multidimensional problems of varying sizes indicate that larger memory savings are obtained with the HTD approach as the number of dimensions increases. © 2017 Elsevier B.V

On the convergence of a class of multilevel methods for large sparse Markov chains

This paper investigates the theory behind the steady state analysis of large sparse Markov chains with a recently proposed class of multilevel methods using concepts from algebraic multigrid and iterative aggregation- disaggregation. The motivation is to better understand the convergence characteristics of the class of multilevel methods and to have a clearer formulation that will aid their implementation. In doing this, restriction (or aggregation) and prolongation (or disaggregation) operators of multigrid are used, and the Kronecker-based approach for hierarchical Markovian models is employed, since it suggests a natural and compact definition of grids (or levels). However, the formalism used to describe the class of multilevel methods for large sparse Markov chains has no influence on the theoretical results derived. © 2007 Society for Industrial and Applied Mathematics

Formalisms for specifying Markovian population models

In this survey, we compare several languages for specifying Markovian population models such as queuing networks and chemical reaction networks. All these languages — matrix descriptions, stochastic Petri nets, stoichiometric equations, stochastic process algebras, and guarded command models — describe continuous-time Markov chains, but they differ according to important properties, such as compositionality, expressiveness and succinctness, executability, and ease of use. Moreover, they provide different support for checking the well-formedness of a model and for analyzing a model

On vector-kronecker product multiplication with rectangular factors

The infinitesimal generator matrix underlying a multidimensional Markov chain can be represented compactly by using sums of Kronecker products of small rectangular matrices. For such compact representations, analysis methods based on vector-Kronecker product multiplication need to be employed. When the factors in the Kronecker product terms are relatively dense, vector- Kronecker product multiplication can be performed efficiently by the shuffle algorithm. When the factors are relatively sparse, it may be more efficient to obtain nonzero elements of the generator matrix in Kronecker form on the fly and multiply them with corresponding elements of the vector. This work proposes a modification to the shuffle algorithm that multiplies relevant elements of the vector with submatrices of factors in which zero rows and columns are omitted. This approach avoids unnecessary floating-point operations that evaluate to zero during the course of the multiplication and possibly reduces the amount of memory used. Numerical experiments on a large number of models indicate that in many cases the modified shuffle algorithm performs a smaller number of floating-point operations than the shuffle algorithm and the algorithm that generates nonzeros on the fly, sometimes with a minimum number of floating-point operations and as little of memory possible. © 2015 Society for Industrial and Applied Mathematics

Markov chain models of instantaneously coupled intracellular calcium channels

Localized calcium elevations known as calcium puffs or sparks are cellular signals arising from cooperative activity of clusters of inositol 1,4,5-trisphosphate receptors (IP3Rs) or ryanodine receptors (RyRs) located at calcium release sites on the endoplasmic or sarcoplasmic reticulum membrane. When Markov chain models of these intracellular calcium-regulated calcium channels are coupled via a mathematical representation of the calcium microdomain, simulated calcium release sites may exhibit the phenomenon of stochastic calcium excitability where the IP3Rs or RyRs open and close in a concerted fashion. Although the biophysical theory relating the kinetics of single channels to the collective phenomena of puffs and sparks is only beginning to be developed, Markov chain models of coupled intracellular channels give insight into the dynamics of calcium puffs and sparks.;Interestingly, under some conditions simulated puffs and sparks can be observed even when the single channel model used does not include slow calcium inactivation or any long-lived closed state. In this case termination of the localized calcium elevation occurs when all of the intracellular channels at a release site simultaneously close through a process called stochastic attrition. This dissertation investigates the statistical properties of stochastic attrition viewed as an absorption time on a terminating Markov chain that represents a calcium release site composed of two-state channels that are activated by calcium. Assuming that the local calcium concentration experienced by a channel depends only on the number of open channels at the calcium release site, the probability distribution function for the time until stochastic attrition occurs is derived and an analytical formula for the expectation of this random variable is presented. Also explored is how the contribution of stochastic attrition to the termination of calcium puffs and sparks depends on the number of channels at a release site, the source amplitude of the channels, the background calcium concentration, channel kinetics, and the cooperativity of calcium binding.;This dissertation also studies whether single channel models with calcium inactivation are less sensitive to the details of release site ultrastructure than models that lack a slow calcium-inactivation process. Release site dynamics obtained from simulated calcium release sites composed of instantaneously coupled calcium-regulated calcium channels whose random spatial locations were chosen from a uniform distribution on a disc of specified radius are compared to simulations with channels arranged on hexagonal lattices. Analysis of puff/spark statistics confirms that puffs and sparks are less sensitive to the spatial organization of release sites when the single channel model includes a slow inactivation process. The validity of several different mean-field reductions that do not explicitly account for the details of release site ultrastructure is also investigated.;Calcium release site models are stochastic automata networks that involve many functional transitions, that is, the transition probabilities of each channel depend on the local calcium concentration and thus the state of the other channels. A Kronecker structured representation for calcium release site models is presented and benchmark stationary distribution calculations using both exact and approximate iterative numerical solution techniques that leverage this structure are performed. When it is possible to obtain an exact solution, response measures such as the number of channels in a particular state converge more quickly using the iterative numerical methods than occupation measures calculated via Monte Carlo simulation. When an exact solution is not feasible, iterative approximate methods based on the Power method may be used, with performance similar to Monte Carlo estimates