On Markov State Models for Metastable Processes

We consider Markov processes on large state spaces and want to find low-dimensional structure-preserving approximations of the process in the sense that the longest timescales of the dynamics of the original process are reproduced well. Recent years have seen the advance of so-called Markov state models (MSM) for processes on very large state spaces exhibiting metastable dynamics. It has been demonstrated that MSMs are especially useful for modelling the interesting slow dynamics of biomolecules (cf. Noe et al, PNAS(106) 2009) and materials. From the mathematical perspective, MSMs result from Galerkin projection of the transfer operator underlying the original process onto some low-dimensional subspace which leads to an approximation of the dominant eigenvalues of the transfer operators and thus of the longest timescales of the original dynamics. Until now, most articles on MSMs have been based on full subdivisions of state space, i.e., Galerkin projections onto subspaces spanned by indicator functions. We show how to generalize MSMs to alternative low-dimensional subspaces with superior approximation properties, and how to analyse the approximation quality (dominant eigenvalues, propagation of functions) of the resulting MSMs. To this end, we give an overview of the construction of MSMs, the associated stochastics and functional-analysis background, and its algorithmic consequences. Furthermore, we illustrate the mathematical construction with numerical examples

Estimating the eigenvalue error of Markov State Models

We consider a continuous-time, ergodic Markov process on a large continuous or discrete state space. The process is assumed to exhibit a number of metastable sets. Markov state models (MSM) are designed to represent the effective dynamics of such a process by a Markov chain that jumps between the metastable sets with the transition rates of the original process. MSM are used for a number of applications, including molecular dynamics (cf. Noe et al, PNAS(106) 2009)[1], since more than a decade. The rigorous and fully general (no zero temperature limit or comparable restrictions) analysis of their approximation quality, however, has only been started recently. Our first article on this topics (Sarich et al, MMS(8) 2010)[2] introduces an error bound for the difference in propagation of probability densities between the MSM and the original process on long time scales. Herein we provide upper bounds for the error in the eigenvalues between the MSM and the original process which means that we analyse how well the longest timescales in the original process are approximated by the MSM. Our findings are illustrated by numerical experiments

Modularity revisited: A novel dynamics-based concept for decomposing complex networks

Finding modules (or clusters) in large, complex networks is a challenging task, in particular if one is not interested in a full decomposition of the whole network into modules. We consider modular networks that also contain nodes that do not belong to one of modules but to several or to none at all. A new method for analyzing such networks is presented. It is based on spectral analysis of random walks on modular networks. In contrast to other spectral clustering approaches, we use different transition rules of the random walk. This leads to much more prominent gaps in the spectrum of the adapted random walk and allows for easy identification of the network's modular structure, and also identifying the nodes belonging to these modules. We also give a characterization of that set of nodes that do not belong to any module, which we call transition region. Finally, by analyzing the transition region, we describe an algorithm that identifies so called hub-nodes inside the transition region that are important connections between modules or between a module and the rest of the network. The resulting algorithms scale linearly with network size (if the network connectivity is sparse) and thus can also be applied to very large networks

Cycle-flow–based module detection in directed recurrence networks

We present a new cycle-flow–based method for finding fuzzy partitions of weighted directed networks coming from time series data. We show that this method overcomes essential problems of most existing clustering approaches, which tend to ignore important directional information by considering only one-step, one-directional node connections. Our method introduces a novel measure of communication between nodes using multi-step, bidirectional transitions encoded by a cycle decomposition of the probability flow. Symmetric properties of this measure enable us to construct an undirected graph that captures the information flow of the original graph seen by the data and apply clustering methods designed for undirected graphs. Finally, we demonstrate our algorithm by analyzing earthquake time series data, which naturally induce (time-)directed networks

Modularity of Directed Networks: Cycle Decomposition Approach

The problem of decomposing networks into modules (or clusters) has gained much attention in recent years, as it can account for a coarse-grained description of complex systems, often revealing functional subunits of these systems. A variety of module detection algorithms have been proposed, mostly oriented towards finding hard partitionings of undirected networks. Despite the increasing number of fuzzy clustering methods for directed networks, many of these approaches tend to neglect important directional information. In this paper, we present a novel random walk based approach for finding fuzzy partitions of directed, weighted networks, where edge directions play a crucial role in defining how well nodes in a module are interconnected. We will show that cycle decomposition of a random walk process connects the notion of network modules and information transport in a network, leading to a new, symmetric measure of node communication. Finally, we will use this measure to introduce a communication graph, for which we will show that although being undirected it inherits important directional information of modular structures from the original network

Finding dominant structures of nonreversible Markov processes

Finding metastable sets as dominant structures of Markov processes has been shown to be especially useful in modeling interesting slow dynamics of various real world complex processes. Furthermore, coarse graining of such processes based on their dominant structures leads to better understanding and dimension reduction of observed systems. However, in many cases, e.g. for nonreversible Markov processes, dominant structures are often not formed by metastable sets but by important cycles or mixture of both. This paper aims at understanding and identifying these different types of dominant structures for reversible as well as nonreversible ergodic Markov processes. Our algorithmic approach generalizes spectral based methods for reversible process by using Schur decomposition techniques which can tackle also nonreversible cases. We illustrate the mathematical construction of our new approach by numerical experiments

Finding dominant structures of nonreversible Markov processes

Finding metastable sets as dominant structures of Markov processes has been shown to be especially useful in modeling interesting slow dynamics of various real world complex processes. Furthermore, coarse graining of such processes based on their dominant structures leads to better understanding and dimension reduction of observed systems. However, in many cases, e.g. for nonreversible Markov processes, dominant structures are often not formed by metastable sets but by important cycles or mixture of both. This paper aims at understanding and identifying these different types of dominant structures for reversible as well as nonreversible ergodic Markov processes. Our algorithmic approach generalizes spectral based methods for reversible process by using Schur decomposition techniques which can tackle also nonreversible cases. We illustrate the mathematical construction of our new approach by numerical experiments

Reactive flows and unproductive cycles in irreversible Markov chains

We present a comprehensive theory for analysis and understanding of transition events between an initial set A and a target set B for general ergodic finite-state space Markov chains or jump processes, including random walks on networks as they occur, e.g., in Markov State Modelling in molecular dynamics. The theory allows us to decompose the probability flow generated by transition events between the sets A and B into the productive part that directly flows from A to B through reaction pathways and the unproductive part that runs in loops and is supported on cycles of the underlying network. It applies to random walks on directed networks and nonreversible Markov processes and can be seen as an extension of Transition Path Theory. Information on reaction pathways and unproductive cycles results from the stochastic cycle decomposition of the underlying network which also allows to compute their corresponding weight, thus characterizing completely which structure is used how often in transition events. The new theory is illustrated by an application to a Markov State Model resulting from weakly damped Langevin dynamics where the unproductive cycles are associated with periodic orbits of the underlying Hamiltonian dynamics

Random Walks on Complex Modular Networks

Complex modular networks appear frequently, notably in the biological or social sciences. We focus on two current challenges regarding network modularity: the ability to identify (i) the modules of a given network, and (ii) the hub states as nodes with highest importance in terms of the communication between modules. Our approach towards these goals uses random walks as a mean to global analysis of the topology and communication structure of the network. We show how to adapt recent research regarding coarse graining of random walks. The resulting algorithms are based on spectral analysis of random walks and allow (A) an optimal identification of fuzzy assignments of nodes to modules, (B) computation of the fraction of the overall communication between modules supported by certain nodes, and (C) determination of the hubs as the nodes with the highest communication load

Cycle-flow–based module detection in directed recurrence networks

We present a new cycle-flow–based method for finding fuzzy partitions of weighted directed networks coming from time series data. We show that this method overcomes essential problems of most existing clustering approaches, which tend to ignore important directional information by considering only one-step, one-directional node connections. Our method introduces a novel measure of communication between nodes using multi-step, bidirectional transitions encoded by a cycle decomposition of the probability flow. Symmetric properties of this measure enable us to construct an undirected graph that captures the information flow of the original graph seen by the data and apply clustering methods designed for undirected graphs. Finally, we demonstrate our algorithm by analyzing earthquake time series data, which naturally induce (time-)directed networks