Krylov Subspace Model Order Reduction for Nonlinear and Bilinear Control Systems

The use of Krylov subspace model order reduction for nonlinear/bilinear systems, over the past few years, has become an increasingly researched area of study. The need for model order reduction has never been higher, as faster computations for control, diagnosis and prognosis have never been higher to achieve better system performance. Krylov subspace model order reduction techniques enable this to be done more quickly and efficiently than what can be achieved at present. The most recent advances in the use of Krylov subspaces for reducing bilinear models match moments and multimoments at some expansion points which have to be obtained through an optimisation scheme. This therefore removes the computational advantage of the Krylov subspace techniques implemented at an expansion point zero. This thesis demonstrates two improved approaches for the use of one-sided Krylov subspace projection for reducing bilinear models at the expansion point zero. This work proposes that an alternate linear approximation can be used for model order reduction. The advantages of using this approach are improved input-output preservation at a simulation cost similar to some earlier works and reduction of bilinear systems models which have singular state transition matrices. The comparison of the proposed methods and other original works done in this area of research is illustrated using various examples of single input single output (SISO) and multi input multi output (MIMO) models

Parameter inference for stochastic biological models

PhD ThesisParameter inference is the field concerned with estimating reliable model parameters from data. In recent years there has been a trend in the biology community toward single cell technologies such as fluorescent flow cytometry, transcriptomics and mass cytometry: providing a rich array of stochastic time series and temporal distribution data for analysis. Deterministically, there are a wide range of parameter inference and global optimisation techniques available. However, these do not always scale well to non-deterministic (i.e., stochastic) settings — whereby the temporal evolution of the system can be described by a chemical master equation for which the solution is nearly always intractable, and the dynamic behaviour of a system is hard to predict. For systems biology, the inference of stochastic parameters remains a bottleneck for accurate model simulation. This thesis is concerned with the parameter inference problem for stochastic chemical reaction networks. Stochastic chemical reaction networks are most frequently modelled as a continuous time discretestate Markov chain using Gillespie’s stochastic simulation algorithm. Firstly, I present a new parameter inference algorithm, SPICE, that combines Gillespie’s algorithm with the cross-entropy method. The cross-entropy method is a novel approach for global optimisation inspired from the field of rare-event probability estimation. I then present recent advances in utilising the generalised method of moments for inference, and seek to provide these approaches with a direct stochastic simulation based correction. Subsequently, I present a novel use of a recent multi-level tau-leaping approach for simulating population moments efficiently, and use this to provide a simulation based correction to the generalised method of moments. I also propose a new method for moment closures based on the use of Padé approximants. The presented algorithms are evaluated on a number of challenging case studies, including bistable systems — e.g., the Schlögl System and the Genetic Toggle Switch — and real experimental data. Experimental results are presented using each of the given algorithms. We also consider ‘realistic’ data — i.e., datasets missing model species, multiple datasets originating from experiment repetitions, and datasets containing arbitrary units (e.g., fluorescence values). The developed approaches are found to be viable alternatives to existing state-ofthe-art methods, and in certain cases are able to outperform other methods in terms of either speed, or accuracyNewcastle/Liverpool/Durham BBSRC Doctoral Training Partnership for financial suppor

Automatic Extraction of Ordinary Differential Equations from Data: Sparse Regression Tools for System Identification

Studying nonlinear systems across engineering, physics, economics, biology, and chemistry often hinges upon successfully discovering their underlying dynamics. However, despite the abundance of data in today's world, a complete comprehension of these governing equations often remains elusive, posing a significant challenge. Traditional system identification methods for building mathematical models to describe these dynamics can be time-consuming, error-prone, and limited by data availability. This thesis presents three comprehensive strategies to address these challenges and automate model discovery. The procedures outlined here employ classic statistical and machine learning methods, such as signal filtering, sparse regression, bootstrap sampling, Bayesian inference, and unsupervised learning algorithms, to capture complex and nonlinear relationships in data. Building on these foundational techniques, the proposed processes offer a reliable and efficient approach to identifying models of ordinary differential equations from data, differing from and complementing existing frameworks. The results presented here provide rigorous benchmarking against state-of-the-art algorithms, demonstrating the proposed methods' effectiveness in model discovery and highlighting the potential for discovering governing equations across applications such as weather forecasting, chemical reaction and electrical circuit modelling, and predator-prey dynamics. These methods can aid in solving critical decision-making problems, including optimising resource allocation, predicting system failures, and facilitating adaptive control in various domains. Ultimately, the strategies developed in this thesis are designed to integrate seamlessly into current workflows, thereby promoting data-driven decision-making and enhancing understanding of complex system dynamics

Optimally-weighted Estimators of the Maximum Mean Discrepancy for Likelihood-Free Inference

Likelihood-free inference methods typically make use of a distance between simulated and real data. A common example is the maximum mean discrepancy (MMD), which has previously been used for approximate Bayesian computation, minimum distance estimation, generalised Bayesian inference, and within the nonparametric learning framework. The MMD is commonly estimated at a root-m rate, where m is the number of simulated samples. This can lead to significant computational challenges since a large m is required to obtain an accurate estimate, which is crucial for parameter estimation. In this paper, we propose a novel estimator for the MMD with significantly improved sample complexity. The estimator is particularly well suited for computationally expensive smooth simulators with low- to mid-dimensional inputs. This claim is supported through both theoretical results and an extensive simulation study on benchmark simulators

The stability of model ecosystems

Ecologists would like to understand how complexity persists in nature. In this thesis I have taken two fundamentally different routes to study ecosystem stability of model ecosystems: classical community ecology and classical population ecology. In community ecology models, we can study the mathematical mechanisms of stability in general, large model ecosystems. In population ecology models, fewer species are studied but greater detail of species interactions can be incorporated. Within these alternative contexts, this thesis contributes to two consuming issues concerning the stability of ecological systems: the ecosystem stability-complexity debate; and the causes of cyclic population dynamics. One of the major unresolved issues in community ecology is the relationship between ecosystem stability and complexity. In 1958 Charles Elton made the conjecture that the stability of an ecological system was coupled to its complexity and this could be a “wise principle of co-existence between man and nature” with which ecologists could argue the case for the conservation of nature for all species, including man. The earliest and simplest model systems were randomly constructed and exhibited a negative association between stability and complexity. This finding sparked the stability-complexity debate and initiated the search for organising principles that enhanced stability in real ecosystems. One of the universal laws of ecology is that ecosystems contain many rare and few common species. In this thesis, I present analytical arguments and numerical results to show that the stability of an ecosystem can increase with complexity when the abundance distribution is characterized by a skew towards many rare species. This work adds to the growing number of conditions under which the negative stability - complexity relationship can been inverted in theoretical studies. While there is growing evidence that the stability-complexity debate is progressing towards a resolution, community ecology has become increasingly subject to major criticism. A long-standing criticism is the reliance on local stability analysis. There is growing recognition that a global property called permanence is a more satisfactory definition of ecosystem stability because it tests only whether species can coexist. Here I identify and explain a positive correlation between the probability of local stability and permanence, which suggests local stability is a better measure of species coexistence than previously thought. While this offers some relief, remaining issues cause the stability-complexity debate to evade clear resolution and leave community ecology in a poor position to argue for the conservation of natural diversity for the benefit of all species. In classical population ecology, a major unresolved issue is the cause of non-equilibrium population dynamics. In this thesis, I use models to study the drivers of cyclic dynamics in Scottish populations of mountain hares (Lepus timidus), for the first time in this system. Field studies currently favour the hypothesis that parasitism by a nematode Trichostrongylus retortaeformis drives the hare cycles, and theory predicts that the interaction should induce cycling. Initially I used a simple, strategic host-parasite model parameterised using available empirical data to test the superficial concordance between theory and observation. I find that parasitism could not account for hare cycles. This verdict leaves three options: either the parameterisation was inadequate, there were missing important biological details or simply that parasites do not drive host cycles. Regarding the first option, reliable information for some hare-parasite model parameters was lacking. Using a rejection-sampling approach motivated by Bayesian methods, I identify the most likely parameter set to predict observed dynamics. The results imply that the current formulation of the hare-parasite model can only generate realistic dynamics when parasite effects are significantly larger than current empirical estimates, and I conclude it is likely that the model contains an inadequate level of detail. The simple strategic model was mathematically elegant and allowed mathematical concepts to be employed in analysis, but the model was biologically naïve. The second model is the antipode of the first, an individual based model (IBM) steeped in biological reality that can only be studied by simulation. Whilst most highly detailed tactical models are developed as a predictive tool, I instead structurally perturb the IBM to study the ecological processes that may drive population cycles in mountain hares. The model allows delayed responses to life history by linking maternal body size and parasite infection to the future survival and fecundity of offspring. By systematically removing model structure I show that these delayed life history effects are weakly destabilising and allow parameters to lie closer to empirical estimates to generate observed hare population cycles. In a third model I structurally modify the simple strategic host-parasite model to make it spatially explicit by including diffusion of mountain hares and corresponding advection of parasites (transportation with host). From initial simulations I show that the spatially extended host-parasite equations are able to generate periodic travelling waves (PTWs) of hare and parasite abundance. This is a newly documented behaviour in these widely used host-parasite equations. While PTWs are a new potential scenario under which cyclic hare dynamics could be explained, further mathematical development is required to determine whether adding space can generate realistic dynamics with parameters that lie closer to empirical estimates. In the general thesis discussion I deliberate on whether a hare-parasite model has been identified which can be considered the right balance between abstraction and relevant detail for this system

Estimating Parameters of Governing Equations of Non-Linear Systems from Data Using Synchronisation and Machine Learning

Deriving governing equations from a measured time series is an ongoing topic of research across different disciplines in science. One method studied can derive the form of governing equations, however it cannot infer the coefficients in front of each term. This is where our work comes in: given the form of governing equations and a measured time series, we propose an algorithm for finding the correct coefficients of the governing equations describing the observed data best. We achieve this by treating the data as primary system and coupling a secondary system to it. Then by inducing synchronisation, we can change the parameters of the secondary system in the direction minimising a loss function. After the loss has reached its minimum, the found parameters are a good estimation of the real parameters producing the data. We applied our algorithm successfully on a number on synthetic systems and even found a method for reconstructing signals masked by chaotic data

Identification of the dynamic characteristics of nonlinear structures

Imperial Users onl