25,636 research outputs found
Optimal control of molecular dynamics using Markov state models
A numerical scheme for solving high-dimensional stochastic control problems on an infinite time horizon that appear relevant in the context of molecular dynamics is outlined. The scheme rests on the interpretation of the corresponding Hamilton-Jacobi-Bellman equation as a nonlinear eigenvalue problem that, using a logarithmic transformation, can be recast as a linear eigenvalue problem, for which the principal eigenvalue and its eigenfunction are sought. The latter can be computed efficiently by approximating the underlying stochastic process with a coarse-grained Markov state model for the dominant metastable sets. We illustrate our method with two numerical examples, one of which involves the task of maximizing the population of -helices in an ensemble of small biomolecules (Alanine dipeptide), and discuss the relation to the large deviation principle of Donsker and Varadhan
Variational approach for learning Markov processes from time series data
Inference, prediction and control of complex dynamical systems from time
series is important in many areas, including financial markets, power grid
management, climate and weather modeling, or molecular dynamics. The analysis
of such highly nonlinear dynamical systems is facilitated by the fact that we
can often find a (generally nonlinear) transformation of the system coordinates
to features in which the dynamics can be excellently approximated by a linear
Markovian model. Moreover, the large number of system variables often change
collectively on large time- and length-scales, facilitating a low-dimensional
analysis in feature space. In this paper, we introduce a variational approach
for Markov processes (VAMP) that allows us to find optimal feature mappings and
optimal Markovian models of the dynamics from given time series data. The key
insight is that the best linear model can be obtained from the top singular
components of the Koopman operator. This leads to the definition of a family of
score functions called VAMP-r which can be calculated from data, and can be
employed to optimize a Markovian model. In addition, based on the relationship
between the variational scores and approximation errors of Koopman operators,
we propose a new VAMP-E score, which can be applied to cross-validation for
hyper-parameter optimization and model selection in VAMP. VAMP is valid for
both reversible and nonreversible processes and for stationary and
non-stationary processes or realizations
Optimal Kullback-Leibler Aggregation via Information Bottleneck
In this paper, we present a method for reducing a regular, discrete-time
Markov chain (DTMC) to another DTMC with a given, typically much smaller number
of states. The cost of reduction is defined as the Kullback-Leibler divergence
rate between a projection of the original process through a partition function
and a DTMC on the correspondingly partitioned state space. Finding the reduced
model with minimal cost is computationally expensive, as it requires an
exhaustive search among all state space partitions, and an exact evaluation of
the reduction cost for each candidate partition. Our approach deals with the
latter problem by minimizing an upper bound on the reduction cost instead of
minimizing the exact cost; The proposed upper bound is easy to compute and it
is tight if the original chain is lumpable with respect to the partition. Then,
we express the problem in the form of information bottleneck optimization, and
propose using the agglomerative information bottleneck algorithm for searching
a sub-optimal partition greedily, rather than exhaustively. The theory is
illustrated with examples and one application scenario in the context of
modeling bio-molecular interactions.Comment: 13 pages, 4 figure
Maximum Margin Clustering for State Decomposition of Metastable Systems
When studying a metastable dynamical system, a prime concern is how to
decompose the phase space into a set of metastable states. Unfortunately, the
metastable state decomposition based on simulation or experimental data is
still a challenge. The most popular and simplest approach is geometric
clustering which is developed based on the classical clustering technique.
However, the prerequisites of this approach are: (1) data are obtained from
simulations or experiments which are in global equilibrium and (2) the
coordinate system is appropriately selected. Recently, the kinetic clustering
approach based on phase space discretization and transition probability
estimation has drawn much attention due to its applicability to more general
cases, but the choice of discretization policy is a difficult task. In this
paper, a new decomposition method designated as maximum margin metastable
clustering is proposed, which converts the problem of metastable state
decomposition to a semi-supervised learning problem so that the large margin
technique can be utilized to search for the optimal decomposition without phase
space discretization. Moreover, several simulation examples are given to
illustrate the effectiveness of the proposed method
REinforcement learning based Adaptive samPling: REAPing Rewards by Exploring Protein Conformational Landscapes
One of the key limitations of Molecular Dynamics simulations is the
computational intractability of sampling protein conformational landscapes
associated with either large system size or long timescales. To overcome this
bottleneck, we present the REinforcement learning based Adaptive samPling
(REAP) algorithm that aims to efficiently sample conformational space by
learning the relative importance of each reaction coordinate as it samples the
landscape. To achieve this, the algorithm uses concepts from the field of
reinforcement learning, a subset of machine learning, which rewards sampling
along important degrees of freedom and disregards others that do not facilitate
exploration or exploitation. We demonstrate the effectiveness of REAP by
comparing the sampling to long continuous MD simulations and least-counts
adaptive sampling on two model landscapes (L-shaped and circular), and
realistic systems such as alanine dipeptide and Src kinase. In all four
systems, the REAP algorithm consistently demonstrates its ability to explore
conformational space faster than the other two methods when comparing the
expected values of the landscape discovered for a given amount of time. The key
advantage of REAP is on-the-fly estimation of the importance of collective
variables, which makes it particularly useful for systems with limited
structural information
Self-optimized construction of transition rate matrices from accelerated atomistic simulations with Bayesian uncertainty quantification
A massively parallel method to build large transition rate matrices from
temperature accelerated molecular dynamics trajectories is presented. Bayesian
Markov model analysis is used to estimate the expected residence time in the
known state space, providing crucial uncertainty quantification for higher
scale simulation schemes such as kinetic Monte Carlo or cluster dynamics. The
estimators are additionally used to optimize where exploration is performed and
the degree of temperature ac- celeration on the fly, giving an autonomous,
optimal procedure to explore the state space of complex systems. The method is
tested against exactly solvable models and used to explore the dynamics of C15
interstitial defects in iron. Our uncertainty quantification scheme allows for
accurate modeling of the evolution of these defects over timescales of several
seconds.Comment: 14 pages, 7 figure
A relative entropy rate method for path space sensitivity analysis of stationary complex stochastic dynamics
We propose a new sensitivity analysis methodology for complex stochastic
dynamics based on the Relative Entropy Rate. The method becomes computationally
feasible at the stationary regime of the process and involves the calculation
of suitable observables in path space for the Relative Entropy Rate and the
corresponding Fisher Information Matrix. The stationary regime is crucial for
stochastic dynamics and here allows us to address the sensitivity analysis of
complex systems, including examples of processes with complex landscapes that
exhibit metastability, non-reversible systems from a statistical mechanics
perspective, and high-dimensional, spatially distributed models. All these
systems exhibit, typically non-gaussian stationary probability distributions,
while in the case of high-dimensionality, histograms are impossible to
construct directly. Our proposed methods bypass these challenges relying on the
direct Monte Carlo simulation of rigorously derived observables for the
Relative Entropy Rate and Fisher Information in path space rather than on the
stationary probability distribution itself. We demonstrate the capabilities of
the proposed methodology by focusing here on two classes of problems: (a)
Langevin particle systems with either reversible (gradient) or non-reversible
(non-gradient) forcing, highlighting the ability of the method to carry out
sensitivity analysis in non-equilibrium systems; and, (b) spatially extended
Kinetic Monte Carlo models, showing that the method can handle high-dimensional
problems
- …