Iterative disaggregation for a class of lumpable discrete-time stochastic automata networks

Cataloged from PDF version of article.Stochastic automata networks (SANs) have been developed and used in the last 15 years as a modeling formalism for large systems that can be decomposed into loosely connected components. In this work, we concentrate on the not so much emphasized discrete-time SANs. First, we remodel and extend an SAN that arises in wireless communications. Second, for an SAN with functional transitions, we derive conditions for a special case of ordinary lumpability in which aggregation is done automaton by automaton. Finally, for this class of lumpable discrete-time SANs we devise an efficient aggregation–iterative disaggregation algorithm and demonstrate its performance on the SAN model of interest. © 2002 Elsevier Science B.V. All rights reserved

Comparison of multilevel methods for kronecker-based Markovian representations

The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates

On the convergence of a class of multilevel methods for large sparse Markov chains

This paper investigates the theory behind the steady state analysis of large sparse Markov chains with a recently proposed class of multilevel methods using concepts from algebraic multigrid and iterative aggregation- disaggregation. The motivation is to better understand the convergence characteristics of the class of multilevel methods and to have a clearer formulation that will aid their implementation. In doing this, restriction (or aggregation) and prolongation (or disaggregation) operators of multigrid are used, and the Kronecker-based approach for hierarchical Markovian models is employed, since it suggests a natural and compact definition of grids (or levels). However, the formalism used to describe the class of multilevel methods for large sparse Markov chains has no influence on the theoretical results derived. © 2007 Society for Industrial and Applied Mathematics

Krylov subspace methods and their generalizations for solving singular linear operator equations with applications to continuous time Markov chains

Viele Resultate über MR- und OR-Verfahren zur Lösung linearer Gleichungssysteme bleiben (in leicht modifizierter Form) gültig, wenn der betrachtete Operator nicht invertierbar ist. Neben dem für reguläre Probleme charakteristischen Abbruchverhalten, kann bei einem singulären Gleichungssystem auch ein so genannter singulärer Zusammenbruch auftreten. Für beide Fälle werden verschiedene Charakterisierungen angegeben. Die Unterrauminverse, eine spezielle verallgemeinerte Inverse, beschreibt die Näherungen eines MR-Unterraumkorrektur-Verfahrens. Für Krylov-Unterräume spielt die Drazin-Inverse eine Schlüsselrolle. Bei Krylov-Unterraum-Verfahren kann a-priori entschieden werden, ob ein regulärer oder ein singulärer Abbruch auftritt. Wir können zeigen, dass ein Krylov-Verfahren genau dann für beliebige Startwerte eine Lösung des linearen Gleichungssystems liefert, wenn der Index der Matrix nicht größer als eins und das Gleichungssystem konsistent ist. Die Berechnung stationärer Zustandsverteilungen zeitstetiger Markov-Ketten mit endlichem Zustandsraum stellt eine praktische Aufgabe dar, welche die Lösung eines singulären linearen Gleichungssystems erfordert. Die Eigenschaften der Übergangs-Halbgruppe folgen aus einfachen Annahmen auf rein analytischem und matrixalgebrischen Wege. Insbesondere ist die erzeugende Matrix eine singuläre M-Matrix mit Index 1. Ist die Markov-Kette irreduzibel, so ist die stationäre Zustandsverteilung eindeutig bestimmt

Markov chain models of instantaneously coupled intracellular calcium channels

Localized calcium elevations known as calcium puffs or sparks are cellular signals arising from cooperative activity of clusters of inositol 1,4,5-trisphosphate receptors (IP3Rs) or ryanodine receptors (RyRs) located at calcium release sites on the endoplasmic or sarcoplasmic reticulum membrane. When Markov chain models of these intracellular calcium-regulated calcium channels are coupled via a mathematical representation of the calcium microdomain, simulated calcium release sites may exhibit the phenomenon of stochastic calcium excitability where the IP3Rs or RyRs open and close in a concerted fashion. Although the biophysical theory relating the kinetics of single channels to the collective phenomena of puffs and sparks is only beginning to be developed, Markov chain models of coupled intracellular channels give insight into the dynamics of calcium puffs and sparks.;Interestingly, under some conditions simulated puffs and sparks can be observed even when the single channel model used does not include slow calcium inactivation or any long-lived closed state. In this case termination of the localized calcium elevation occurs when all of the intracellular channels at a release site simultaneously close through a process called stochastic attrition. This dissertation investigates the statistical properties of stochastic attrition viewed as an absorption time on a terminating Markov chain that represents a calcium release site composed of two-state channels that are activated by calcium. Assuming that the local calcium concentration experienced by a channel depends only on the number of open channels at the calcium release site, the probability distribution function for the time until stochastic attrition occurs is derived and an analytical formula for the expectation of this random variable is presented. Also explored is how the contribution of stochastic attrition to the termination of calcium puffs and sparks depends on the number of channels at a release site, the source amplitude of the channels, the background calcium concentration, channel kinetics, and the cooperativity of calcium binding.;This dissertation also studies whether single channel models with calcium inactivation are less sensitive to the details of release site ultrastructure than models that lack a slow calcium-inactivation process. Release site dynamics obtained from simulated calcium release sites composed of instantaneously coupled calcium-regulated calcium channels whose random spatial locations were chosen from a uniform distribution on a disc of specified radius are compared to simulations with channels arranged on hexagonal lattices. Analysis of puff/spark statistics confirms that puffs and sparks are less sensitive to the spatial organization of release sites when the single channel model includes a slow inactivation process. The validity of several different mean-field reductions that do not explicitly account for the details of release site ultrastructure is also investigated.;Calcium release site models are stochastic automata networks that involve many functional transitions, that is, the transition probabilities of each channel depend on the local calcium concentration and thus the state of the other channels. A Kronecker structured representation for calcium release site models is presented and benchmark stationary distribution calculations using both exact and approximate iterative numerical solution techniques that leverage this structure are performed. When it is possible to obtain an exact solution, response measures such as the number of channels in a particular state converge more quickly using the iterative numerical methods than occupation measures calculated via Monte Carlo simulation. When an exact solution is not feasible, iterative approximate methods based on the Power method may be used, with performance similar to Monte Carlo estimates

Parallele numerische Verfahren zur quantitativen Analyse logistischer Systeme

In der vorliegenden Arbeit wird ein numerisches Verfahren zur Lösung sehr großer Markov- Ketten vorgestellt und seine prinzipielle Eignung und Performance experimentell untersucht. Das Verfahren basiert auf hierarchischen und asynchronen Iterationen, nutzt eine hierarchische Kronecker-Darstellung zur Darstellung der Markov-Kette und ist auf einer parallelen Rechenarchitektur mit verteiltem Speicher implementiert. Die Arbeit dokumentiert die Lösung von Markov-Ketten mit bis zu 900 Millionen Zuständen, die aus dem Anwendungsfeld der Logistik resultieren

Low-rank tensor methods for large Markov chains and forward feature selection methods

In the first part of this thesis, we present and compare several approaches for the determination of the steady-state of large-scale Markov chains with an underlying low-rank tensor structure. Such structure is, in our context of interest, associated with the existence of interacting processes. The state space grows exponentially with the number of processes. This type of problems arises, for instance, in queueing theory, in chemical reaction networks, or in telecommunications. As the number of degrees of freedom of the problem grows exponentially with the number of processes, the so-called \textit{curse of dimensionality} severely impairs the use of standard methods for the numerical analysis of such Markov chains. We drastically reduce the number of degrees of freedom by assuming a low-rank tensor structure of the solution. We develop different approaches, all considering a formulation of the problem where all involved structures are considered in their low-rank representations in \textit{tensor train} format. The first approaches that we will consider are associated with iterative solvers, in particular focusing on solving a minimization problem that is equivalent to the original problem of finding the desired steady state. We later also consider tensorized multigrid techniques as main solvers, using different operators for restriction and interpolation. For instance, aggregation/disaggregation operators, which have been extensively used in this field, are applied. In the second part of this thesis, we focus on methods for feature selection. More concretely, since, among the various classes of methods, sequential feature selection methods based on mutual information have become very popular and are widely used in practice, we focus on this particular type of methods. This type of problems arises, for instance, in microarray analysis, in clinical prediction, or in text categorization. Comparative evaluations of these methods have been limited by being based on specific datasets and classifiers. We develop a theoretical framework that allows evaluating the methods based on their theoretical properties. Our framework is based on the properties of the target objective function that the methods try to approximate, and on a novel categorization of features, according to their contribution to the explanation of the class; we derive upper and lower bounds for the target objective function and relate these bounds with the feature types. Then, we characterize the types of approximations made by the methods, and analyse how these approximations cope with the good properties of the target objective function. We also develop a distributional setting designed to illustrate the various deficiencies of the methods, and provide several examples of wrong feature selections. In the context of this setting, we use the minimum Bayes risk as performance measure of the methods

Algebraic Multigrid for Markov Chains and Tensor Decomposition

The majority of this thesis is concerned with the development of efficient and robust numerical methods based on adaptive algebraic multigrid to compute the stationary distribution of Markov chains. It is shown that classical algebraic multigrid techniques can be applied in an exact interpolation scheme framework to compute the stationary distribution of irreducible, homogeneous Markov chains. A quantitative analysis shows that algebraically smooth multiplicative error is locally constant along strong connections in a scaled system operator, which suggests that classical algebraic multigrid coarsening and interpolation can be applied to the class of nonsymmetric irreducible singular M-matrices with zero column sums. Acceleration schemes based on fine-level iterant recombination, and over-correction of the coarse-grid correction are developed to improve the rate of convergence and scalability of simple adaptive aggregation multigrid methods for Markov chains. Numerical tests over a wide range of challenging nonsymmetric test problems demonstrate the effectiveness of the proposed multilevel method and the acceleration schemes. This thesis also investigates the application of adaptive algebraic multigrid techniques for computing the canonical decomposition of higher-order tensors. The canonical decomposition is formulated as a least squares optimization problem, for which local minimizers are computed by solving the first-order optimality equations. The proposed multilevel method consists of two phases: an adaptive setup phase that uses a multiplicative correction scheme in conjunction with bootstrap algebraic multigrid interpolation to build the necessary operators on each level, and a solve phase that uses additive correction cycles based on the full approximation scheme to efficiently obtain an accurate solution. The alternating least squares method, which is a standard one-level iterative method for computing the canonical decomposition, is used as the relaxation scheme. Numerical tests show that for certain test problems arising from the discretization of high-dimensional partial differential equations on regular lattices the proposed multilevel method significantly outperforms the standard alternating least squares method when a high level of accuracy is required