Parameter estimation in biochemical systems models with alternating regression

BACKGROUND: The estimation of parameter values continues to be the bottleneck of the computational analysis of biological systems. It is therefore necessary to develop improved methods that are effective, fast, and scalable. RESULTS: We show here that alternating regression (AR), applied to S-system models and combined with methods for decoupling systems of differential equations, provides a fast new tool for identifying parameter values from time series data. The key feature of AR is that it dissects the nonlinear inverse problem of estimating parameter values into iterative steps of linear regression. We show with several artificial examples that the method works well in many cases. In cases of no convergence, it is feasible to dedicate some computational effort to identifying suitable start values and search settings, because the method is fast in comparison to conventional methods that the search for suitable initial values is easily recouped. Because parameter estimation and the identification of system structure are closely related in S-system modeling, the AR method is beneficial for the latter as well. Specifically, we show with an example from the literature that AR is three to five orders of magnitudes faster than direct structure identifications in systems of nonlinear differential equations. CONCLUSION: Alternating regression provides a strategy for the estimation of parameter values and the identification of structure and regulation in S-systems that is genuinely different from all existing methods. Alternating regression is usually very fast, but its convergence patterns are complex and will require further investigation. In cases where convergence is an issue, the enormous speed of the method renders it feasible to select several initial guesses and search settings as an effective countermeasure

Identification of neutral biochemical network models from time series data

<p>Abstract</p> <p>Background</p> <p>The major difficulty in modeling biological systems from multivariate time series is the identification of parameter sets that endow a model with dynamical behaviors sufficiently similar to the experimental data. Directly related to this parameter estimation issue is the task of identifying the structure and regulation of ill-characterized systems. Both tasks are simplified if the mathematical model is canonical, <it>i.e</it>., if it is constructed according to strict guidelines.</p> <p>Results</p> <p>In this report, we propose a method for the identification of admissible parameter sets of canonical S-systems from biological time series. The method is based on a Monte Carlo process that is combined with an improved version of our previous parameter optimization algorithm. The method maps the parameter space into the network space, which characterizes the connectivity among components, by creating an ensemble of decoupled S-system models that imitate the dynamical behavior of the time series with sufficient accuracy. The concept of sloppiness is revisited in the context of these S-system models with an exploration not only of different parameter sets that produce similar dynamical behaviors but also different network topologies that yield dynamical similarity.</p> <p>Conclusion</p> <p>The proposed parameter estimation methodology was applied to actual time series data from the glycolytic pathway of the bacterium <it>Lactococcus lactis </it>and led to ensembles of models with different network topologies. In parallel, the parameter optimization algorithm was applied to the same dynamical data upon imposing a pre-specified network topology derived from prior biological knowledge, and the results from both strategies were compared. The results suggest that the proposed method may serve as a powerful exploration tool for testing hypotheses and the design of new experiments.</p

Parameter estimation methods for biological systems

The inverse problem of modeling biochemical processes mathematically from measured time course data falls into the category of system identification and parameter estimation. Analyzing the time course data would provide valuable insights into the model structure and dynamics of the biochemical system. Based on the types of biochemical reactions, such as metabolic networks and genetic networks, several modeling frameworks have been proposed, developed and proved effective, including the Michaelis-Menten equation, the Biochemical System Theory (BST), etc. One bottleneck in analyzing the obtained data is the estimation of parameter values within the system model. As most models for molecular biological systems are nonlinear with respect to both parameters and system state variables, estimation of parameters in these models from experimental measurement data is thus a nonlinear estimation problem. In principle, all algorithms for nonlinear optimization can be used to deal with this problem, for example, the Gauss-Newton iteration method and its variants. However, these methods do not take the special structures of biological system models into account. When the number of parameters to be determined increases, it will be challenging and computationally expensive to apply these conventional methods. In this research, several methods are proposed for estimating parameters in two classes of widely used biological system models: the S-system model and the linear fractional model (LFM), by utilizing their structure specialties. For the S-system, two estimation methods are designed. 1) Based on the two-term structure (production and degradation) of the model, an alternating iterative least squares method is proposed. 2) A separation nonlinear least squares method is proposed to deal with the partially linear structure of the model. For the LFM, two estimation methods are provided. 1) The separation nonlinear least squares method can also be adopted to treat the partially linear structure of the LFM, and moreover a modified iterative version is included. 2) A special strategy using the separation principle and the weighted least squares method is implemented to turn the cost function into a quadratic form and thus the estimates for parameters can be analytically solved. Simulation results have demonstrated the effectiveness of the proposed methods, which have shown better performance in terms of estimation accuracy and computation time, compared with those conventional nonlinear estimation methods

Data-driven modelling of biological multi-scale processes

Biological processes involve a variety of spatial and temporal scales. A holistic understanding of many biological processes therefore requires multi-scale models which capture the relevant properties on all these scales. In this manuscript we review mathematical modelling approaches used to describe the individual spatial scales and how they are integrated into holistic models. We discuss the relation between spatial and temporal scales and the implication of that on multi-scale modelling. Based upon this overview over state-of-the-art modelling approaches, we formulate key challenges in mathematical and computational modelling of biological multi-scale and multi-physics processes. In particular, we considered the availability of analysis tools for multi-scale models and model-based multi-scale data integration. We provide a compact review of methods for model-based data integration and model-based hypothesis testing. Furthermore, novel approaches and recent trends are discussed, including computation time reduction using reduced order and surrogate models, which contribute to the solution of inference problems. We conclude the manuscript by providing a few ideas for the development of tailored multi-scale inference methods.Comment: This manuscript will appear in the Journal of Coupled Systems and Multiscale Dynamics (American Scientific Publishers

Structure Learning in Coupled Dynamical Systems and Dynamic Causal Modelling

Identifying a coupled dynamical system out of many plausible candidates, each of which could serve as the underlying generator of some observed measurements, is a profoundly ill posed problem that commonly arises when modelling real world phenomena. In this review, we detail a set of statistical procedures for inferring the structure of nonlinear coupled dynamical systems (structure learning), which has proved useful in neuroscience research. A key focus here is the comparison of competing models of (ie, hypotheses about) network architectures and implicit coupling functions in terms of their Bayesian model evidence. These methods are collectively referred to as dynamical casual modelling (DCM). We focus on a relatively new approach that is proving remarkably useful; namely, Bayesian model reduction (BMR), which enables rapid evaluation and comparison of models that differ in their network architecture. We illustrate the usefulness of these techniques through modelling neurovascular coupling (cellular pathways linking neuronal and vascular systems), whose function is an active focus of research in neurobiology and the imaging of coupled neuronal systems

Learning stable and predictive structures in kinetic systems: Benefits of a causal approach

Learning kinetic systems from data is one of the core challenges in many fields. Identifying stable models is essential for the generalization capabilities of data-driven inference. We introduce a computationally efficient framework, called CausalKinetiX, that identifies structure from discrete time, noisy observations, generated from heterogeneous experiments. The algorithm assumes the existence of an underlying, invariant kinetic model, a key criterion for reproducible research. Results on both simulated and real-world examples suggest that learning the structure of kinetic systems benefits from a causal perspective. The identified variables and models allow for a concise description of the dynamics across multiple experimental settings and can be used for prediction in unseen experiments. We observe significant improvements compared to well established approaches focusing solely on predictive performance, especially for out-of-sample generalization

System identification, time series analysis and forecasting:The Captain Toolbox handbook.

CAPTAIN is a MATLAB compatible toolbox for non stationary time series analysis, system identification, signal processing and forecasting, using unobserved components models, time variable parameter models, state dependent parameter models and multiple input transfer function models. CAPTAIN also includes functions for true digital control

Time-varying parametric modelling and time-dependent spectral characterisation with applications to EEG signals using multi-wavelets

A new time-varying autoregressive (TVAR) modelling approach is proposed for nonstationary signal processing and analysis, with application to EEG data modelling and power spectral estimation. In the new parametric modelling framework, the time-dependent coefficients of the TVAR model are represented using a novel multi-wavelet decomposition scheme. The time-varying modelling problem is then reduced to regression selection and parameter estimation, which can be effectively resolved by using a forward orthogonal regression algorithm. Two examples, one for an artificial signal and another for an EEG signal, are given to show the effectiveness and applicability of the new TVAR modelling method