Extended State Dependent Parameter modelling with a Data-Based Mechanistic approach to nonlinear model structure identification

Abstract A unified approach to Multiple and single State Dependent Parameter modelling, termed Extended State Dependent Parameters (ESDP) modelling, of nonlinear dynamic systems described by time-varying dynamic models applied to ARX or transfer-function model structures. Crucially, the approach proposes an effective model structure identification method using a novel Information Criterion (IC) taking into account model complexity in terms of the number of states involved. In ESDP, model structure involves not only the model orders, but also selection of the states driving the parameters, which effectively prevents the use of most current IC. This leads to a powerful methodology for investigating nonlinear systems building on the Data-Based Mechanistic (DBM) philosophy of Young and expanding the applications of the existing DBM methods. The methodologies presented are tested and demonstrated on both simulated data and on high frequency hydrological observations, showing how structure identification leads to discovery of dynamic relationships between system variables

Data-driven modelling of biological multi-scale processes

Biological processes involve a variety of spatial and temporal scales. A holistic understanding of many biological processes therefore requires multi-scale models which capture the relevant properties on all these scales. In this manuscript we review mathematical modelling approaches used to describe the individual spatial scales and how they are integrated into holistic models. We discuss the relation between spatial and temporal scales and the implication of that on multi-scale modelling. Based upon this overview over state-of-the-art modelling approaches, we formulate key challenges in mathematical and computational modelling of biological multi-scale and multi-physics processes. In particular, we considered the availability of analysis tools for multi-scale models and model-based multi-scale data integration. We provide a compact review of methods for model-based data integration and model-based hypothesis testing. Furthermore, novel approaches and recent trends are discussed, including computation time reduction using reduced order and surrogate models, which contribute to the solution of inference problems. We conclude the manuscript by providing a few ideas for the development of tailored multi-scale inference methods.Comment: This manuscript will appear in the Journal of Coupled Systems and Multiscale Dynamics (American Scientific Publishers

A Manifesto for the Equifinality Thesis.

This essay discusses some of the issues involved in the identification and predictions of hydrological models given some calibration data. The reasons for the incompleteness of traditional calibration methods are discussed. The argument is made that the potential for multiple acceptable models as representations of hydrological and other environmental systems (the equifinality thesis) should be given more serious consideration than hitherto. It proposes some techniques for an extended GLUE methodology to make it more rigorous and outlines some of the research issues still to be resolved

Developing models for the data-based mechanistic approach to systems analysis:Increasing objectivity and reducing assumptions

Stochastic State-Space Time-Varying Random Walk models have been developed, allowing the existing Stochastic State Space models to operate directly on irregularly sampled time-series. These TVRW models have been successfully applied to two different classes of models benefiting each class in different ways. The first class of models - State Dependent Parameter (SDP) models and used to investigate the dominant dynamic modes of nonlinear dynamic systems and the non-linearities in these models affected by arbitrary State Variables. In SDP locally linearised models it is assumed that the parameters that describe system’s behaviour changes are dependent upon some aspect of the system (it’s ‘state’). Each parameter can be dependent on one or more states. To estimate the parameters that are changing at a rate related to that of it’s states, the estimation procedure is conducted in the state-space along the potentially multivariate trajectory of the states which drive the parameters. The introduction of the newly developed TVRW models significantly improves parameter estimation, particularly in data rich neighbourhoods of the state-space when the parameter is dependent on more than one state, and the ends of the data-series when the parameter is dependent on one state with few data points. The second class of models are known as Dynamic Harmonic Regression (DHR) models and are used to identify the dominant cycles and trends of time-series. DHR models the assumption is that a signal (such as a time-series) can be broken down into four (unobserved) components occupying different parts of the spectrum: trend, seasonal cycle, other cycles, and a high frequency irregular component. DHR is confined to uniformly sampled time-series. The introduction of the TVRW models allows DHR to operate on irregularly sampled time-series, with the added benefit of forecasting origin no longer being confined to starting at the end of the time-series but can now begin at any point in the future. Additionally, the forecasting sampling rate is no longer limited to the sampling rate of the time-series. Importantly, both classes of model were designed to follow the Data-Based Mechanistic (DBM) approach to modelling environmental systems, where the model structure and parameters are to be determined by the data (Data-Based) and then the subsequent models are to be validated based on their physical interpretation (Mechanistic). The aim is to remove the researcher’s preconceptions from model development in order to eliminate any bias, and then use the researcher’s knowledge to validate the models presented to them. Both classes of model lacked model structure identification procedures and so model structure was determined by the researcher, against the DBM approach. Two different model structure identification procedures, one for SDP and the other for DHR, were developed to bring both classes of models back within the DBM framework. These developments have been presented and tested here on both simulated data and real environmental data, demonstrating their importance, benefits and role in environmental modelling and exploratory data analysis

Cohort aggregation modelling for complex forest stands: Spruce-aspen mixtures in British Columbia

Mixed-species growth models are needed as a synthesis of ecological knowledge and for guiding forest management. Individual-tree models have been commonly used, but the difficulties of reliably scaling from the individual to the stand level are often underestimated. Emergent properties and statistical issues limit their effectiveness. A more holistic modelling of aggregates at the whole stand level is a potentially attractive alternative. This work explores methodology for developing biologically consistent dynamic mixture models where the state is described by aggregate stand-level variables for species or age/size cohorts. The methods are demonstrated and tested with a two-cohort model for spruce-aspen mixtures named SAM. The models combine single-species submodels and submodels for resource partitioning among the cohorts. The partitioning allows for differences in competitive strength among species and size classes, and for complementarity effects. Height growth reduction in suppressed cohorts is also modelled. SAM fits well the available data, and exhibits behaviors consistent with current ecological knowledge. The general framework can be applied to any number of cohorts, and should be useful as a basis for modelling other mixed-species or uneven-aged stands.Comment: Accepted manuscript, to appear in Ecological Modellin

Structure Learning in Coupled Dynamical Systems and Dynamic Causal Modelling

Identifying a coupled dynamical system out of many plausible candidates, each of which could serve as the underlying generator of some observed measurements, is a profoundly ill posed problem that commonly arises when modelling real world phenomena. In this review, we detail a set of statistical procedures for inferring the structure of nonlinear coupled dynamical systems (structure learning), which has proved useful in neuroscience research. A key focus here is the comparison of competing models of (ie, hypotheses about) network architectures and implicit coupling functions in terms of their Bayesian model evidence. These methods are collectively referred to as dynamical casual modelling (DCM). We focus on a relatively new approach that is proving remarkably useful; namely, Bayesian model reduction (BMR), which enables rapid evaluation and comparison of models that differ in their network architecture. We illustrate the usefulness of these techniques through modelling neurovascular coupling (cellular pathways linking neuronal and vascular systems), whose function is an active focus of research in neurobiology and the imaging of coupled neuronal systems

Advances and Future Perspectives

Agharafeie , R., Ramos, J. R. C., Mendes, J. M., & Oliveira, R. M. F. (2023). From Shallow to Deep Bioprocess Hybrid Modeling: Advances and Future Perspectives. Fermentation, 9(10), 1-22. [922]. https://doi.org/10.20944/preprints202310.0107.v1, https://doi.org/10.3390/fermentation9100922--- This work was supported by the Associate Laboratory for Green Chemistry - LAQV which is financed by national funds from FCT/MCTES (UIDB/50006/2020 and UIDP/50006/2020). This work received funding from the European Union’s Horizon 2020 research and innovation program under the grant agreement no. 101099487- BioLaMer-HORIZON-EIC-2022-PATHFINDEROPEN-01 (BioLaMer)Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.publishersversionpublishe