Accelerated maximum likelihood parameter estimation for stochastic biochemical systems

Background: A prerequisite for the mechanistic simulation of a biochemical system is detailed knowledge of its kinetic parameters. Despite recent experimental advances, the estimation of unknown parameter values from observed data is still a bottleneck for obtaining accurate simulation results. Many methods exist for parameter estimation in deterministic biochemical systems; methods for discrete stochastic systems are less well developed. Given the probabilistic nature of stochastic biochemical models, a natural approach is to choose parameter values that maximize the probability of the observed data with respect to the unknown parameters, a.k.a. the maximum likelihood parameter estimates (MLEs). MLE computation for all but the simplest models requires the simulation of many system trajectories that are consistent with experimental data. For models with unknown parameters, this presents a computational challenge, as the generation of consistent trajectories can be an extremely rare occurrence. Results: We have developed Monte Carlo Expectation-Maximization with Modified Cross-Entropy Method (MCEM2): an accelerated method for calculating MLEs that combines advances in rare event simulation with a computationally efficient version of the Monte Carlo expectation-maximization (MCEM) algorithm. Our method requires no prior knowledge regarding parameter values, and it automatically provides a multivariate parameter uncertainty estimate. We applied the method to five stochastic systems of increasing complexity, progressing from an analytically tractable pure-birth model to a computationally demanding model of yeast-polarization. Our results demonstrate that MCEM2 substantially accelerates MLE computation on all tested models when compared to a stand-alone version of MCEM. Additionally, we show how our method identifies parameter values for certain classes of models more accurately than two recently proposed computationally efficient methods. Conclusions: This work provides a novel, accelerated version of a likelihood-based parameter estimation method that can be readily applied to stochastic biochemical systems. In addition, our results suggest opportunities for added efficiency improvements that will further enhance our ability to mechanistically simulate biological processes.This article is from BMC Bioinformatics 13 (2012): 68, doi: 10.1186/1471-2105-13-68. Posted with permission.</p

Accelerated maximum likelihood parameter estimation for stochastic biochemical systems

<p>Abstract</p> <p>Background</p> <p>A prerequisite for the mechanistic simulation of a biochemical system is detailed knowledge of its kinetic parameters. Despite recent experimental advances, the estimation of unknown parameter values from observed data is still a bottleneck for obtaining accurate simulation results. Many methods exist for parameter estimation in deterministic biochemical systems; methods for discrete stochastic systems are less well developed. Given the probabilistic nature of stochastic biochemical models, a natural approach is to choose parameter values that maximize the probability of the observed data with respect to the unknown parameters, a.k.a. the maximum likelihood parameter estimates (MLEs). MLE computation for all but the simplest models requires the simulation of many system trajectories that are consistent with experimental data. For models with unknown parameters, this presents a computational challenge, as the generation of consistent trajectories can be an extremely rare occurrence.</p> <p>Results</p> <p>We have developed Monte Carlo Expectation-Maximization with Modified Cross-Entropy Method (MCEM²): an accelerated method for calculating MLEs that combines advances in rare event simulation with a computationally efficient version of the Monte Carlo expectation-maximization (MCEM) algorithm. Our method requires no prior knowledge regarding parameter values, and it automatically provides a multivariate parameter uncertainty estimate. We applied the method to five stochastic systems of increasing complexity, progressing from an analytically tractable pure-birth model to a computationally demanding model of yeast-polarization. Our results demonstrate that MCEM² substantially accelerates MLE computation on all tested models when compared to a stand-alone version of MCEM. Additionally, we show how our method identifies parameter values for certain classes of models more accurately than two recently proposed computationally efficient methods.</p> <p>Conclusions</p> <p>This work provides a novel, accelerated version of a likelihood-based parameter estimation method that can be readily applied to stochastic biochemical systems. In addition, our results suggest opportunities for added efficiency improvements that will further enhance our ability to mechanistically simulate biological processes.</p

METHODS FOR COMPUTATION AND ANALYSIS OF MARKOVIAN DYNAMICS ON COMPLEX NETWORKS

A problem central to many scientific and engineering disciplines is how to deal with noisy dynamic processes that take place on networks. Examples include the ebb and flow of biochemical concentrations within cells, the firing patterns of neurons in the brain, and the spread of disease on social networks. In this thesis, we present a general formalism capable of representing many such problems by means of a master equation. Our study begins by synthesizing the literature to provide a toolkit of known mathematical and computational analysis techniques for dealing with this equation. Subsequently a novel exact numerical solution technique is developed, which can be orders of magnitude faster than the state-of-the-art numerical solver. However, numerical solutions are only applicable to a small subset of processes on networks. Thus, many approximate solution techniques exist in the literature to deal with this problem. Unfortunately, no practical tools exist to quantitatively evaluate the quality of an approximate solution in a given system. Therefore, a statistical tool that is capable of evaluating any analytical or Monte Carlo based approximation to the master equation is developed herein. Finally, we note that larger networks with more complex dynamical phenomena suffer from the same curse of dimensionality as the classical mechanics of a gas. We therefore propose that thermodynamic analysis techniques, adapted from statistical mechanics, may provide a new way forward in analyzing such systems. The investigation focuses on a behavior known as avalanching—complex bursting patterns with fractal properties. By developing thermodynamic analysis techniques along with a potential energy landscape perspective, we are able to demonstrate that increasing intrinsic noise causes a phase transition that results in avalanching. This novel result is utilized to characterize avalanching in an epidemiological model for the first time and to explain avalanching in biological neural networks, in which the cause has been falsely attributed to specific neural architectures. This thesis contributes to the existing literature by providing a novel solution technique, enhances existing and future literature by providing a general method for statistical evaluation of approximative solution techniques, and paves the way towards a promising approach to the thermodynamic analysis of large complex processes on networks