Recursively accelerated multilevel aggregation for markov chains

Abstract. A recursive acceleration method is proposed for multiplicative multilevel aggregation algorithms that calculate the stationary probability vector of large, sparse, and irreducible Markov chains. Pairs of consecutive iterates at all branches and levels of a multigrid W cycle with simple, nonoverlapping aggregation are recombined to produce improved iterates at those levels. This is achieved by solving quadratic programming problems with inequality constraints: the linear combination of the two iterates is sought that has a minimal two-norm residual, under the constraint that all vector components are nonnegative. It is shown how the two-dimensional quadratic programming problems can be solved explicitly in an efficient way. The method is further enhanced by windowed top-level acceleration of the W cycles using the same constrained quadratic programming approach. Recursive acceleration is an attractive alternative to smoothing the restriction and interpolation operators, since the operator complexity is better controlled and the probabilistic interpretation of coarse-level operators is maintained on all levels. Numerical results are presented showing that the resulting recursively accelerated multilevel aggregation cycles for Markov chains, combined with top-level acceleration, converge significantly faster than W cycles and lead to close-to-linear computational complexity for challenging test problems

Adaptive relaxation for the steady-state analysis of Markov chains

We consider a variant of the well-known Gauss-Seidel method for the solution of Markov chains in steady state. Whereas the standard algorithm visits each state exactly once per iteration in a predetermined order, the alternative approach uses a dynamic strategy. A set of states to be visited is maintained which can grow and shrink as the computation progresses. In this manner, we hope to concentrate the computational work in those areas of the chain in which maximum improvement in the solution can be achieved. We consider the adaptive approach both as a solver in its own right and as a relaxation method within the multi-level algorithm. Experimental results show significant computational savings in both cases

Lumpable continuous-time stochastic automata networks

Cataloged from PDF version of article.The generator matrix of a continuous-time stochastic automata network (SAN) is a sum of tensor products of smaller matrices, which may have entries that are functions of the global state space. This paper specifies easy to check conditions for a class of ordinarily lumpable partitionings of the generator of a continuous-time SAN in which aggregation is performed automaton by automaton. When there exists a lumpable partitioning induced by the tensor representation of the generator, it is shown that an efficient aggregation-iterative disaggregation algorithm may be employed to compute the steady-state distribution. The results of experiments with two SAN models show that the proposed algorithm performs better than the highly competitive block Gauss-Seidel in terms of both the number of iterations and the time to converge to the solution. © 2002 Elsevier Science B.V. All rights reserved

Multilevel Aggregation Methods for Small-World Graphs with Application to Random-Walk Ranking

We describe multilevel aggregation in the specific context of using Markov chains to rank the nodes of graphs. More generally, aggregation is a graph coarsening technique that has a wide range of possible uses regarding information retrieval applications. Aggregation successfully generates efficient multilevel methods for solving nonsingular linear systems and various eigenproblems from discretized partial differential equations, which tend to involve mesh-like graphs. Our primary goal is to extend the applicability of aggregation to similar problems on small-world graphs, with a secondary goal of developing these methods for eventual applicability towards many other tasks such as using the information in the hierarchies for node clustering or pattern recognition. The nature of small-world graphs makes it difficult for many coarsening approaches to obtain useful hierarchies that have complexity on the order of the number of edges in the original graph while retaining the relevant properties of the original graph. Here, for a set of synthetic graphs with the small-world property, we show how multilevel hierarchies formed with non-overlapping strength-based aggregation have optimal or near optimal complexity. We also provide an example of how these hierarchies are employed to accelerate convergence of methods that calculate the stationary probability vector of large, sparse, irreducible, slowly-mixing Markov chains on such small-world graphs. The stationary probability vector of a Markov chain allows one to rank the nodes in a graph based on the likelihood that a long random walk visits each node. These ranking approaches have a wide range of applications including information retrieval and web ranking, performance modeling of computer and communication systems, analysis of social networks, dependability and security analysis, and analysis of biological systems

Block SOR for Kronecker structured representations

Cataloged from PDF version of article.The Kronecker structure of a hierarchical Markovian model (HMM) induces nested block partitionings in the transition matrix of its underlying Markov chain. This paper shows how sparse real Schur factors of certain diagonal blocks of a given partitioning induced by the Kronecker structure can be constructed from smaller component matrices and their real Schur factors. Furthermore, it shows how the column approximate minimum degree (COLAMD) ordering algorithm can be used to reduce fill-in of the remaining diagonal blocks that are sparse LU factorized. Combining these ideas, the paper proposes three-level block successive over-relaxation (BSOR) as a competitive steady state solver for HMMs. Finally, on a set of numerical experiments it demonstrates how these ideas reduce storage required by the factors of the diagonal blocks and improve solution time compared to an all LU factorization implementation of the BSOR solver. © 2004 Elsevier Inc. All rights reserved

Proportional lumpability and proportional bisimilarity

3noIn this paper, we deal with the lumpability approach to cope with the state space explosion problem inherent to the computation of the stationary performance indices of large stochastic models. The lumpability method is based on a state aggregation technique and applies to Markov chains exhibiting some structural regularity. Moreover, it allows one to efficiently compute the exact values of the stationary performance indices when the model is actually lumpable. The notion of quasi-lumpability is based on the idea that a Markov chain can be altered by relatively small perturbations of the transition rates in such a way that the new resulting Markov chain is lumpable. In this case, only upper and lower bounds on the performance indices can be derived. Here, we introduce a novel notion of quasi-lumpability, named proportional lumpability, which extends the original definition of lumpability but, differently from the general definition of quasi-lumpability, it allows one to derive exact stationary performance indices for the original process. We then introduce the notion of proportional bisimilarity for the terms of the performance process algebra PEPA. Proportional bisimilarity induces a proportional lumpability on the underlying continuous-time Markov chains. Finally, we prove some compositionality results and show the applicability of our theory through examples.openopenMarin A.; Piazza C.; Rossi S.Marin, A.; Piazza, C.; Rossi, S

Weak Markovian Bisimulation Congruences and Exact CTMC-Level Aggregations for Concurrent Processes

We have recently defined a weak Markovian bisimulation equivalence in an integrated-time setting, which reduces sequences of exponentially timed internal actions to individual exponentially timed internal actions having the same average duration and execution probability as the corresponding sequences. This weak Markovian bisimulation equivalence is a congruence for sequential processes with abstraction and turns out to induce an exact CTMC-level aggregation at steady state for all the considered processes. However, it is not a congruence with respect to parallel composition. In this paper, we show how to generalize the equivalence in a way that a reasonable tradeoff among abstraction, compositionality, and exactness is achieved for concurrent processes. We will see that, by enhancing the abstraction capability in the presence of concurrent computations, it is possible to retrieve the congruence property with respect to parallel composition, with the resulting CTMC-level aggregation being exact at steady state only for a certain subset of the considered processes.Comment: In Proceedings QAPL 2012, arXiv:1207.055