Learning and Designing Stochastic Processes from Logical Constraints

Stochastic processes offer a flexible mathematical formalism to model and reason about systems. Most analysis tools, however, start from the premises that models are fully specified, so that any parameters controlling the system's dynamics must be known exactly. As this is seldom the case, many methods have been devised over the last decade to infer (learn) such parameters from observations of the state of the system. In this paper, we depart from this approach by assuming that our observations are {\it qualitative} properties encoded as satisfaction of linear temporal logic formulae, as opposed to quantitative observations of the state of the system. An important feature of this approach is that it unifies naturally the system identification and the system design problems, where the properties, instead of observations, represent requirements to be satisfied. We develop a principled statistical estimation procedure based on maximising the likelihood of the system's parameters, using recent ideas from statistical machine learning. We demonstrate the efficacy and broad applicability of our method on a range of simple but non-trivial examples, including rumour spreading in social networks and hybrid models of gene regulation

Model Reconstruction for Moment-based Stochastic Chemical Kinetics

Based on the theory of stochastic chemical kinetics, the inherent randomness and stochasticity of biochemical reaction networks can be accurately described by discrete-state continuous-time Markov chains. The analysis of such processes is, however, computationally expensive and sophisticated numerical methods are required. Here, we propose an analysis framework in which we integrate a number of moments of the process instead of the state probabilities. This results in a very efficient simulation of the time evolution of the process. In order to regain the state probabilities from the moment representation, we combine the fast moment-based simulation with a maximum entropy approach for the reconstruction of the underlying probability distribution. We investigate the usefulness of this combined approach in the setting of stochastic chemical kinetics and present numerical results for three reaction networks showing its efficiency and accuracy. Besides a simple dimerization system, we study a bistable switch system and a multi-attractor network with complex dynamics.Comment: 20 pages,5 figure

Approximate maximum likelihood estimation for stochastic chemical kinetics

Recent experimental imaging techniques are able to tag and count molecular populations in a living cell. From these data mathematical models are inferred and calibrated. If small populations are present, discrete-state stochastic models are widely-used to describe the discreteness and randomness of molecular interactions. Based on time-series data of the molecular populations, the corresponding stochastic reaction rate constants can be estimated. This procedure is computationally very challenging, since the underlying stochastic process has to be solved for different parameters in order to obtain optimal estimates. Here, we focus on the maximum likelihood method and estimate rate constants, initial populations and parameters representing measurement errors

Numerical analysis of stochastic biochemical reaction networks

Numerical solution of the chemical master equation for stochastic reaction networks typically suffers from the state space explosion problem due to the curse of dimensionality and from stiffness due to multiple time scales. The dimension of the state space equals the number of molecular species involved in the reaction network and the size of the system of differential equations equals the number of states in the corresponding continuous-time Markov chain, which is usually enormously huge and often even infinite. Thus, efficient numerical solution approaches must be able to handle huge, possibly infinite and stiff systems of differential equations efficiently. In this thesis, we present efficient techniques for the numerical analysis of the biochemical reaction networks. We present an approximate numerical integration approach that combines a dynamical state space truncation procedure with efficient numerical integration schemes for systems of ordinary differential equations including adaptive step size selection based on local error estimates. We combine our dynamical state space truncation with the method of conditional moments, and present the implementation details and numerical results. We also incorporate ideas from importance sampling simulations into a non-simulative numerical method that approximates transient rare event probabilities based on a dynamical truncation of the state space. Finally, we present a maximum likelihood method for the estimation of the model parameters given noisy time series measurements of molecular counts. All approaches presented in this thesis are implemented as part of the tool STAR, which allows to model and simulate the biochemical reaction networks. The efficiency and accuracy is demonstrated by numerical examples.Numerische Lösungen der chemischen Master-Gleichung für stochastische Reaktionsnetzwerke leiden typischerweise an dem Zustandsraumexplosionsproblem aufgrund der hohen Dimensionalität und der Steifigkeit durch mehrfache Zeitskalen. Die Dimension des Zustandsraumes entspricht der Anzahl der molekularen Spezies von dem Reaktionsnetzwerk und die Größe des Systems von Differentialgleichungen entspricht der Anzahl der Zustände in der entsprechenden kontinuierlichen Markov-Kette, die in der Regel enorm gross und oft sogar unendlich gross ist. Daher müssen numerische Methoden in der Lage sein, riesige, eventuell unendlich grosse und steife Systeme von Differentialgleichungen effizient lösen zu können. In dieser Arbeit beschreiben wir effiziente Methoden für die numerische Analyse biochemischer Reaktionsnetzwerke. Wir betrachten einen inexakten numerischen Integrationsansatz, bei dem eine dynamische Zustandsraumbeschneidung und ein Verfahren mit einem effizienten numerischen Integrationsschema für Systeme von gewöhnlichen Differentialgleichungen benutzt werden. Wir kombinieren unsere dynamische Zustandsraumbeschneidungsmethode mit der Methode der bedingten Momente und beschreiben die Implementierungdetails und numerischen Ergebnisse. Wir benutzen auch Ideen des importance sampling für eine nicht-simulative numerische Methode, die basierend auf der Zustandsraumbeschneidung die Wahrscheinlichkeiten von seltenen Ereignissen berechnen kann. Schließlich beschreiben wir eine Maximum-Likelihood-Methode für die Schätzung der Modellparameter bei verrauschten Zeitreihenmessungen von molekularen Anzahlen. Alle in dieser Arbeit beschriebenen Ansätze sind in dem Software-Tool STAR implementiert, das erlaubt, biochemische Reaktionsnetzwerke zu modellieren und zu simulieren. Die Effizienz und die Genauigkeit werden durch numerische Beispiele gezeigt

Model reconstruction for moment-based stochastic chemical kinetics

Based on the theory of stochastic chemical kinetics, the inherent randomness and stochasticity of biochemical reaction networks can be accurately described by discrete-state continuous-time Markov chains, where each chemical reaction corresponds to a state transition of the process. However, the analysis of such processes is computationally expensive and sophisticated numerical methods are required. The main complication comes due to the largeness problem of the state space, so that analysis techniques based on an exploration of the state space are often not feasible and the integration of the moments of the underlying probability distribution has become a very popular alternative. In this thesis we propose an analysis framework in which we integrate a number of moments of the process instead of the state probabilities. This results in a more timeefficient simulation of the time evolution of the process. In order to regain the state probabilities from the moment representation, we combine the moment-based simulation (MM) with a maximum entropy approach: the maximum entropy principle is applied to derive a distribution that fits best to a given sequence of moments. We further extend this approach by incorporating the conditional moments (MCM) which allows not only to reconstruct the distribution of the species present in high amount in the system, but also to approximate the probabilities of species with low molecular counts. For the given distribution reconstruction framework, we investigate the numerical accuracy and stability using case studies from systems biology, compare two different moment approximation methods (MM and MCM), examine if it can be used for the reaction rates estimation problem and describe the possible future applications. In this thesis we propose an analysis framework in which we integrate a number of moments of the process instead of the state probabilities. This results in a more time-efficient simulation of the time evolution of the process. In order to regain the state probabilities from the moment representation, we combine the moment-based simulation (MM) with a maximum entropy approach: the maximum entropy principle is applied to derive a distribution that fits best to a given sequence of moments. We further extend this approach by incorporating the conditional moments (MCM) which allows not only to reconstruct the distribution of the species present in high amount in the system, but also to approximate the probabilities of species with low molecular counts. For the given distribution reconstruction framework, we investigate the numerical accuracy and stability using case studies from systems biology, compare two different moment approximation methods (MM and MCM), examine if it can be used for the reaction rates estimation problem and describe the possible future applications.Basierend auf der Theorie der stochastischen chemischen Kinetiken können die inhärente Zufälligkeit und Stochastizität von biochemischen Reaktionsnetzwerken durch diskrete zeitkontinuierliche Markow-Ketten genau beschrieben werden, wobei jede chemische Reaktion einem Zustandsübergang des Prozesses entspricht. Die Analyse solcher Prozesse ist jedoch rechenaufwendig und komplexe numerische Verfahren sind erforderlich. Analysetechniken, die auf dem Abtasten des Zustandsraums basieren, sind durch dessen Größe oft nicht anwendbar. Als populäre Alternative wird heute häufig die Integration der Momente der zugrundeliegenden Wahrscheinlichkeitsverteilung genutzt. In dieser Arbeit schlagen wir einen Analyserahmen vor, in dem wir, anstatt der Zustandswahrscheinlichkeiten, zugrundeliegende Momente des Prozesses integrieren. Dies führt zu einer zeiteffizienteren Simulation der zeitlichen Entwicklung des Prozesses. Um die Zustandswahrscheinlichkeiten aus der Momentreprsentation wiederzugewinnen, kombinieren wir die momentbasierte Simulation (MM) mit Entropiemaximierung: Die Maximum- Entropie-Methode wird angewendet, um eine Verteilung abzuleiten, die am besten zu einer bestimmten Sequenz von Momenten passt. Wir erweitern diesen Ansatz durch das Einbeziehen bedingter Momente (MCM), die es nicht nur erlauben, die Verteilung der in großer Menge im System enthaltenen Spezies zu rekonstruieren, sondern es ebenso ermöglicht, sich den Wahrscheinlichkeiten von Spezies mit niedrigen Molekulargewichten anzunähern. Für das gegebene System zur Verteilungsrekonstruktion untersuchen wir die numerische Genauigkeit und Stabilität anhand von Fallstudien aus der Systembiologie, vergleichen zwei unterschiedliche Verfahren der Momentapproximation (MM und MCM), untersuchen, ob es für das Problem der Abschätzung von Reaktionsraten verwendet werden kann und beschreiben die mögliche zukünftige Anwendungen

Cauchy integrals for computational solutions of master equations

Cauchy contour integrals are demonstrated to be effective in computationally solving master equations. A fractional generalization of a bimolecular master equation is one interesting application. References A. Andreychenko, L. Mikeev, D. Spieler, and V. Wolf. Approximate maximum likelihood estimation for stochastic chemical kinetics. EURASIP J. Bioinf. Sys. Biol., 2012:9, 2012. doi:10.1186/1687-4153-2012-9 C. N. Angstmann, I. C. Donnelly, B. I. Henry, and J. A. Nichols. A discrete time random walk model for anomalous diffusion. J. Comput. Phys., 293:53–69, 2014. doi:10.1016/j.jcp.2014.08.003 Y. Berkowitz, Y. Edery, H. Scher, and B. Berkowitz. Fickian and non-Fickian diffusion with bimolecular reactions. Phys. Rev. E, 87:032812, 2013. doi:10.1103/PhysRevE.87.032812 J. C. Butcher. On the numerical inversion of Laplace and Mellin transforms. Conference on Data Processing and Automatic Computing Machines, 117:1–8, 1957. D. Ding, D. A. Benson, A. Paster, and D. Bolster. Modeling bimolecular reactions and transport in porous media via particle tracking. Adv. Water Resour., 53:56–65, 2013. doi:10.1016/j.advwatres.2012.11.001 B. Drawert, S. Engblom, and A. Hellander. URDME: a modular framework for stochastic simulation of reaction-transport processes in complex geometries. BMC Syst. Biol., 6:76, 2012. doi:10.1186/1752-0509-6-76 T. A Driscoll, N. Hale, and L. N. Trefethen. Chebfun Guide. Pafnuty Publications, 2014. http://www.chebfun.org/docs/guide/ N. Dunford and J. Schwartz. Linear Operators I, II, III. Wiley New York, 1971. http://au.wiley.com/WileyCDA/WileyTitle/productCd-0471608483.html, http://au.wiley.com/WileyCDA/WileyTitle/productCd-0471608475.html, http://au.wiley.com/WileyCDA/WileyTitle/productCd-0471608467.html D. Fulger, E. Scalas, and G. Germano. Monte Carlo simulation of uncoupled continuous-time random walks yielding a stochastic solution of the space-time fractional diffusion equation. Phys. Rev. E, 77:021122, 2008. doi:10.1103/PhysRevE.77.021122 D. Gillespie. Markov Processes: An Introduction for Physical Scientists. Academic Press, 1991. http://www.elsevier.com/books/markov-processes/gillespie/978-0-12-283955-9 M. Hegland, C. Burden, L. Santoso, S. MacNamara, and H. Booth. A solver for the stochastic master equation applied to gene regulatory networks. J. Comput. Appl. Math., 205(2):708–724, 2006. doi:10.1016/j.cam.2006.02.053 B. I. Henry, T. A. M. Langlands, and S. L. Wearne. Anomalous diffusion with linear reaction dynamics: From continuous time random walks to fractional reaction-diffusion equations. Phys. Rev. E, 74(3):031116, 2006. doi:10.1103/PhysRevE.74.031116 N. J. Higham. Functions of Matrices. SIAM, 2008. doi:10.1137/1.9780898717778 R. Hilfer and L. Anton. Fractional master equations and fractal time random walks. Phys. Rev. E, 51:R848, 1995. doi:10.1103/PhysRevE.51.R848 K. J. in 't Hout and J. A. C. Weideman. A contour integral method for the Black–Scholes and Heston equations. SIAM J. Sci. Comput., 33:763–785, 2011. doi:10.1137/090776081 T. Jahnke and D. Altintan. Efficient simulation of discrete stochastic reaction systems with a splitting method. BIT, 50:797–822, 2010. doi:doi:10.1007/s10543-010-0286-0 T. Kato. Perturbation theory for linear operators. Springer-Verlag, 1976. http://link.springer.com/book/10.1007%2F978-3-642-66282-9 V. M. Kenkre, E. W. Montroll, and M. F. Shlesinger. Generalized master equations for continuous-time random walks. J. Stat. Phys., 9(1):45, 1973. doi:10.1007/BF01016796 M. Lopez-Fernandez and C. Palencia. On the numerical inversion of the Laplace transform in certain holomorphic mappings. Appl. Numer. Math., 51:289–303, 2004. doi:10.1016/j.apnum.2004.06.015 S. MacNamara, K. Burrage, and R. B. Sidje. Multiscale modeling of chemical kinetics via the master equation. SIAM Multiscale Model. Simul., 6(4):1146–1168, 2008. doi:10.1137/060678154 M. Magdziarz, A. Weron, and K. Weron. Fractional Fokker–Planck dynamics: Stochastic representation and computer simulation. Phys. Rev. E, 75:016708, 2007. doi:10.1103/PhysRevE.75.016708 F. Mainardi, R. Gorenflo, and A. Vivoli. Beyond the Poisson renewal process: A tutorial survey. J. Comput. Appl. Math., 2007. doi:10.1016/j.cam.2006.04.060 W. McLean. Regularity of solutions to a time-fractional diffusion equation. ANZIAM J., 52(2):123–138, 2010. doi:10.1017/S1446181111000617 W. McLean and V. Thomee. Time discretization of an evolution equation via Laplace transforms. IMA J. Numer. Anal., 24:439–463, 2004. doi:10.1093/imanum/24.3.439 R. Metzler and J. Klafter. The random walk's guide to anomalous diffusion: a fractional dynamics approach. Phys. Rep., 339:1–77, 2000. doi:10.1016/S0370-1573(00)00070-3 C. Moler and C. Van Loan. Nineteen Dubious Ways to Compute the Exponential of a Matrix, 25 Years Later. SIAM Rev., 45(1):3–49, 2003. doi:10.1137/S00361445024180 E. W. Montroll and G. H. Weiss. Random Walks on Lattices. II. J. Math. Phys., 6(2):167–181, 1965. doi:10.1063/1.1704269 I. Moret and P. Novati. On the Convergence of Krylov Subspace Methods for Matrix Mittag–Leffler Functions. SIAM J. Numer. Anal., 49(5):2144–2164, 2011. doi:10.1137/080738374 I. Podlubny. Fractional Differential Equations. Academic Press, San Diego, 1999. http://www.elsevier.com/books/fractional-differential-equations/podlubny/978-0-12-558840-9 M. Raberto, F. Rapallo, and E. Scalas. Semi-Markov Graph Dynamics. PLoS ONE, 6(8):e23370, 2011. doi:10.1371/journal.pone.0023370 S. C. Reddy and L. N. Trefethen. Pseudospectra of the convection-diffusion operator. SIAM J. Appl. Math., 54(6):1634–1649, 1994. doi:10.1137/S0036139993246982 E. B. Saff and A. D. Snider. Fundamentals of complex analysis with applications to engineering and science. Pearson Education, 2003. http://www.pearsonhighered.com/educator/product/Fundamentals-of-Complex-Analysis-with-Applications-to-Engineering-Science-and-Mathematics/9780139078743.page E. Scalas, R. Gorenflo, and F. Mainardi. Uncoupled continuous-time random walks: Solution and limiting behavior of the master equation. Phys. Rev. E, 69:011107, 2004. doi:10.1103/PhysRevE.69.011107 D. Sheen, I. H. Sloan, and V. Thomee. A parallel method for time discretization of parabolic equations based on Laplace transformation and quadrature. IMA J. Numer. Anal., 23:269–299, 2003. doi:10.1093/imanum/23.2.269 M. J. Shon and A. E. Cohen. Mass action at the single-molecule level. J. Am. Chem. Soc., 134(35):14618–14623, 2012. doi:10.1021/ja3062425 G. Strang and S. MacNamara. Functions of difference matrices are Toeplitz plus Hankel. SIAM Rev., 56(3):525–546, 2014. doi:10.1137/120897572 A. Talbot. The accurate numerical inversion of Laplace transforms. J. Inst. Math. Appl., 23:97–120, 1979. doi:10.1093/imamat/23.1.97 L. N. Trefethen. Approximation Theory and Approximation Practice. SIAM, Philadelphia, 2013. http://bookstore.siam.org/ot128/ L. N. Trefethen and M. Embree. Spectra and pseudospectra: the behavior of nonnormal matrices and operators. Princeton University Press, 2005. http://press.princeton.edu/titles/8113.html L. N. Trefethen and J. A. C. Weideman. The exponentially convergent trapezoidal rule. SIAM Rev., 56(3):385–458, 2014. doi:10.1137/130932132 N. G. van Kampen. Stochastic Processes in Physics and Chemistry. Elsevier Science, 2001. http://www.elsevier.com/books/stochastic-processes-in-physics-and-chemistry/van-kampen/978-0-444-52965-7 J. A. C. Weideman. Improved contour integral methods for parabolic PDEs. IMA J. Numer. Anal., 30:334–350, 2010. doi:10.1093/imanum/drn074 J. A. C. Weideman and L. N. Trefethen. Parabolic and hyperbolic contours for computing the bromwich integral. Math. Comput., 76:1341–1356, 2007. doi:10.1090/S0025-5718-07-01945-X T. G. Wright. Eigtool, 2002. http://www.comlab.ox.ac.uk/pseudospectra/eigtool/. Q. Yang, T. Moroney, K. Burrage, I. Turner, and F. Liu. Novel numerical methods for time-space fractional reaction diffusion equations in two dimensions. ANZIAM J., 52:395–409, 2011. http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/3791/146