Data-driven modelling of biological multi-scale processes

Biological processes involve a variety of spatial and temporal scales. A holistic understanding of many biological processes therefore requires multi-scale models which capture the relevant properties on all these scales. In this manuscript we review mathematical modelling approaches used to describe the individual spatial scales and how they are integrated into holistic models. We discuss the relation between spatial and temporal scales and the implication of that on multi-scale modelling. Based upon this overview over state-of-the-art modelling approaches, we formulate key challenges in mathematical and computational modelling of biological multi-scale and multi-physics processes. In particular, we considered the availability of analysis tools for multi-scale models and model-based multi-scale data integration. We provide a compact review of methods for model-based data integration and model-based hypothesis testing. Furthermore, novel approaches and recent trends are discussed, including computation time reduction using reduced order and surrogate models, which contribute to the solution of inference problems. We conclude the manuscript by providing a few ideas for the development of tailored multi-scale inference methods.Comment: This manuscript will appear in the Journal of Coupled Systems and Multiscale Dynamics (American Scientific Publishers

Model reduction for stochastic CaMKII reaction kinetics in synapses by graph-constrained correlation dynamics.

A stochastic reaction network model of Ca(2+) dynamics in synapses (Pepke et al PLoS Comput. Biol. 6 e1000675) is expressed and simulated using rule-based reaction modeling notation in dynamical grammars and in MCell. The model tracks the response of calmodulin and CaMKII to calcium influx in synapses. Data from numerically intensive simulations is used to train a reduced model that, out of sample, correctly predicts the evolution of interaction parameters characterizing the instantaneous probability distribution over molecular states in the much larger fine-scale models. The novel model reduction method, 'graph-constrained correlation dynamics', requires a graph of plausible state variables and interactions as input. It parametrically optimizes a set of constant coefficients appearing in differential equations governing the time-varying interaction parameters that determine all correlations between variables in the reduced model at any time slice

Uniformisation techniques for stochastic simulation of chemical reaction networks

This work considers the method of uniformisation for continuous-time Markov chains in the context of chemical reaction networks. Previous work in the literature has shown that uniformisation can be beneficial in the context of time-inhomogeneous models, such as chemical reaction networks incorporating extrinsic noise. This paper lays focus on the understanding of uniformisation from the viewpoint of sample paths of chemical reaction networks. In particular, an efficient pathwise stochastic simulation algorithm for time-homogeneous models is presented which is complexity-wise equal to Gillespie's direct method. This new approach therefore enlarges the class of problems for which the uniformisation approach forms a computationally attractive choice. Furthermore, as a new application of the uniformisation method, we provide a novel variance reduction method for (raw) moment estimators of chemical reaction networks based upon the combination of stratification and uniformisation

Parameter estimation for biochemical reaction networks using Wasserstein distances

We present a method for estimating parameters in stochastic models of biochemical reaction networks by fitting steady-state distributions using Wasserstein distances. We simulate a reaction network at different parameter settings and train a Gaussian process to learn the Wasserstein distance between observations and the simulator output for all parameters. We then use Bayesian optimization to find parameters minimizing this distance based on the trained Gaussian process. The effectiveness of our method is demonstrated on the three-stage model of gene expression and a genetic feedback loop for which moment-based methods are known to perform poorly. Our method is applicable to any simulator model of stochastic reaction networks, including Brownian Dynamics.Comment: 22 pages, 8 figures. Slight modifications/additions to the text; added new section (Section 4.4) and Appendi

Markovian Dynamics on Complex Reaction Networks

Complex networks, comprised of individual elements that interact with each other through reaction channels, are ubiquitous across many scientific and engineering disciplines. Examples include biochemical, pharmacokinetic, epidemiological, ecological, social, neural, and multi-agent networks. A common approach to modeling such networks is by a master equation that governs the dynamic evolution of the joint probability mass function of the underling population process and naturally leads to Markovian dynamics for such process. Due however to the nonlinear nature of most reactions, the computation and analysis of the resulting stochastic population dynamics is a difficult task. This review article provides a coherent and comprehensive coverage of recently developed approaches and methods to tackle this problem. After reviewing a general framework for modeling Markovian reaction networks and giving specific examples, the authors present numerical and computational techniques capable of evaluating or approximating the solution of the master equation, discuss a recently developed approach for studying the stationary behavior of Markovian reaction networks using a potential energy landscape perspective, and provide an introduction to the emerging theory of thermodynamic analysis of such networks. Three representative problems of opinion formation, transcription regulation, and neural network dynamics are used as illustrative examples.Comment: 52 pages, 11 figures, for freely available MATLAB software, see http://www.cis.jhu.edu/~goutsias/CSS%20lab/software.htm

Approximation and inference methods for stochastic biochemical kinetics-a tutorial review

Stochastic fluctuations of molecule numbers are ubiquitous in biological systems. Important examples include gene expression and enzymatic processes in living cells. Such systems are typically modelled as chemical reaction networks whose dynamics are governed by the chemical master equation. Despite its simple structure, no analytic solutions to the chemical master equation are known for most systems. Moreover, stochastic simulations are computationally expensive, making systematic analysis and statistical inference a challenging task. Consequently, significant effort has been spent in recent decades on the development of efficient approximation and inference methods. This article gives an introduction to basic modelling concepts as well as an overview of state of the art methods. First, we motivate and introduce deterministic and stochastic methods for modelling chemical networks, and give an overview of simulation and exact solution methods. Next, we discuss several approximation methods, including the chemical Langevin equation, the system size expansion, moment closure approximations, time-scale separation approximations and hybrid methods. We discuss their various properties and review recent advances and remaining challenges for these methods. We present a comparison of several of these methods by means of a numerical case study and highlight some of their respective advantages and disadvantages. Finally, we discuss the problem of inference from experimental data in the Bayesian framework and review recent methods developed the literature. In summary, this review gives a self-contained introduction to modelling, approximations and inference methods for stochastic chemical kinetics

Parameter inference for stochastic biological models

PhD ThesisParameter inference is the field concerned with estimating reliable model parameters from data. In recent years there has been a trend in the biology community toward single cell technologies such as fluorescent flow cytometry, transcriptomics and mass cytometry: providing a rich array of stochastic time series and temporal distribution data for analysis. Deterministically, there are a wide range of parameter inference and global optimisation techniques available. However, these do not always scale well to non-deterministic (i.e., stochastic) settings — whereby the temporal evolution of the system can be described by a chemical master equation for which the solution is nearly always intractable, and the dynamic behaviour of a system is hard to predict. For systems biology, the inference of stochastic parameters remains a bottleneck for accurate model simulation. This thesis is concerned with the parameter inference problem for stochastic chemical reaction networks. Stochastic chemical reaction networks are most frequently modelled as a continuous time discretestate Markov chain using Gillespie’s stochastic simulation algorithm. Firstly, I present a new parameter inference algorithm, SPICE, that combines Gillespie’s algorithm with the cross-entropy method. The cross-entropy method is a novel approach for global optimisation inspired from the field of rare-event probability estimation. I then present recent advances in utilising the generalised method of moments for inference, and seek to provide these approaches with a direct stochastic simulation based correction. Subsequently, I present a novel use of a recent multi-level tau-leaping approach for simulating population moments efficiently, and use this to provide a simulation based correction to the generalised method of moments. I also propose a new method for moment closures based on the use of Padé approximants. The presented algorithms are evaluated on a number of challenging case studies, including bistable systems — e.g., the Schlögl System and the Genetic Toggle Switch — and real experimental data. Experimental results are presented using each of the given algorithms. We also consider ‘realistic’ data — i.e., datasets missing model species, multiple datasets originating from experiment repetitions, and datasets containing arbitrary units (e.g., fluorescence values). The developed approaches are found to be viable alternatives to existing state-ofthe-art methods, and in certain cases are able to outperform other methods in terms of either speed, or accuracyNewcastle/Liverpool/Durham BBSRC Doctoral Training Partnership for financial suppor

Bounding Mean First Passage Times in Population Continuous-Time Markov Chains

We consider the problem of bounding mean first passage times and reachability probabilities for the class of population continuous-time Markov chains, which capture stochastic interactions between groups of identical agents. The quantitative analysis of such models is notoriously difficult since typically neither state-based numerical approaches nor methods based on stochastic sampling give efficient and accurate results. Here, we propose a novel approach that leverages techniques from martingale theory and stochastic processes to generate constraints on the statistical moments of first passage time distributions. These constraints induce a semi-definite program that can be used to compute exact bounds on reachability probabilities and mean first passage times without numerically solving the transient probability distribution of the process or sampling from it. We showcase the method on some test examples and tailor it to models exhibiting multimodality, a class of particularly challenging scenarios from biology