DATA-DRIVEN STUDIES OF TRANSIENT EVENTS AND APERIODIC MOTIONS

The era of big data, high-performance computing, and machine learning has witnessed a paradigm shift from physics-based modeling to data-driven modeling across many scientific fields. In this dissertation work, transient events and aperiodic motions of complex nonlinear dynamical system are studied with the aid of a data- driven modeling approach. The goal of the work has been to further the ability for future behavior prediction, state estimation, and control of related behaviors. It is shown that data on extreme waves can be used to carry out stability analysis and ascertain the nature of the transient phenomenon. In addition, it is demonstrated that a low number of soliton elements can be used to realize a rogue wave on the basis of nonlinear interactions amongst the basic elements. The pro- posed nonlinear phase interference model provides an appealing explanation for the formation of ocean extreme wave and related statistics, and a superior reconstruction of the Draupner wave event than that obtained on the basis of linear superposition. Chaotic data, another manifestation of aperiodic motions, which are obtained from prototypical ordinary differential and partial differential systems are considered and a neural machine is realized to predict the corresponding responses based on a limited training set as well to forecast the system behavior. A specific neural architecture, called the inhibitor mechanism, has been designed to enable chaotic time series forecasting. Without this mechanism, even the short-term predictions would be intractable. Both autonomous and non-autonomous dynamical systems have been studied to demonstrate the long-term forecasting possibilities with the de- veloped neural machine. For each dynamical system considered in this dissertation, a long forecasting horizon is achieved with a short historical data set. Furthermore, with the developed neural machine, one can relax the requirement of continuous historical data measurements, thus, providing for a more pragmatic approach than the previous approaches available in the literature. It is expected that the efforts of this dissertation work will lead to a better understanding of the underlying mechanism of transient and aperiodic events in complex systems and useful techniques for forecasting their future occurrences

Optimizing the combination of data-driven and model-based elements in hybrid reservoir computing

Hybrid reservoir computing combines purely data-driven machine learning predictions with a physical model to improve the forecasting of complex systems. In this study, we investigate in detail the predictive capabilities of three different architectures for hybrid reservoir computing: the input hybrid (IH), output hybrid (OH), and full hybrid (FH), which combines IH and OH. By using nine different threedimensional chaotic model systems and the high-dimensional spatiotemporal chaotic Kuramoto–Sivashinsky system, we demonstrate that all hybrid reservoir computing approaches significantly improve the prediction results, provided that the model is sufficiently accurate. For accurate models, we find that the OH and FH results are equivalent and significantly outperform the IH results, especially for smaller reservoir sizes. For totally inaccurate models, the predictive capabilities of IH and FH may decrease drastically, while the OH architecture remains as accurate as the purely data-driven results. Furthermore, OH allows for the separation of the reservoir and the model contributions to the output predictions. This enables an interpretation of the roles played by the data-driven and model-based elements in output hybrid reservoir computing, resulting in higher explainability of the prediction results. Overall, our findings suggest that the OH approach is the most favorable architecture for hybrid reservoir computing, when taking accuracy, interpretability, robustness to model error, and simplicity into account

Controlled Sequential Monte Carlo

Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques for approximating high-dimensional probability distributions and their normalizing constants. These methods have found numerous applications in statistics and related fields; e.g. for inference in non-linear non-Gaussian state space models, and in complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which crucially impact their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. This method builds upon a number of existing algorithms in econometrics, physics, and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models. We provide a theoretical analysis concerning the fluctuation and stability of this methodology that also provides insight into the properties of related algorithms. We demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications

Reconstruction and Parameter Estimation of Dynamical Systems using Neural Networks

Dynamical systems can be loosely regarded as systems whose dynamics is entirely determined by en evolution function and an initial condition, being therefore completely deterministic and a priori predictable. Nevertheless, their phenomenology is surprisingly rich, including intriguing phenomena such as chaotic dynamics, fractal dimensions and entropy production. In Climate Science for example, the emergence of chaos forbids us to have meteorological forecasts going beyond fourteen days in the future in the current epoch and therefore building predictive systems that overcome this limitation, at least partially, are of the extreme importance since we live in fast-changing climate world, as proven by the recent not-so-extreme-anymore climate phenomena. At the same time, Machine Learning techniques have been widely applied to practically every field of human knowledge starting from approximately ten years ago, when essentially two factors contributed to the so-called rebirth of Deep Learning: the availability of larger datasets, putting us in the era of Big Data, and the improvement of computational power. However, the possibility to apply Neural Networks to chaotic systems have been widely debated, since these models are very data hungry and rely thus on the availability of large datasets, whereas often Climate data are rare and sparse. Moreover, chaotic dynamics should not rely much on past statistics, which these models are built on. In this thesis, we explore the possibility to study dynamical systems, seen as simple proxies of Climate models, by using Neural Networks, possibly adding prior knowledge on the underlying physical processes in the spirit of Physics Informed Neural Networks, aiming to the reconstruction of the Weather (short term dynamics) and Climate (long term dynamics) of these dynamical systems as well as the estimation of unknown parameters from Data.Dynamical systems can be loosely regarded as systems whose dynamics is entirely determined by en evolution function and an initial condition, being therefore completely deterministic and a priori predictable. Nevertheless, their phenomenology is surprisingly rich, including intriguing phenomena such as chaotic dynamics, fractal dimensions and entropy production. In Climate Science for example, the emergence of chaos forbids us to have meteorological forecasts going beyond fourteen days in the future in the current epoch and therefore building predictive systems that overcome this limitation, at least partially, are of the extreme importance since we live in fast-changing climate world, as proven by the recent not-so-extreme-anymore climate phenomena. At the same time, Machine Learning techniques have been widely applied to practically every field of human knowledge starting from approximately ten years ago, when essentially two factors contributed to the so-called rebirth of Deep Learning: the availability of larger datasets, putting us in the era of Big Data, and the improvement of computational power. However, the possibility to apply Neural Networks to chaotic systems have been widely debated, since these models are very data hungry and rely thus on the availability of large datasets, whereas often Climate data are rare and sparse. Moreover, chaotic dynamics should not rely much on past statistics, which these models are built on. In this thesis, we explore the possibility to study dynamical systems, seen as simple proxies of Climate models, by using Neural Networks, possibly adding prior knowledge on the underlying physical processes in the spirit of Physics Informed Neural Networks, aiming to the reconstruction of the Weather (short term dynamics) and Climate (long term dynamics) of these dynamical systems as well as the estimation of unknown parameters from Data

Lagrangian ocean analysis: fundamentals and practices

Lagrangian analysis is a powerful way to analyse the output of ocean circulation models and other ocean velocity data such as from altimetry. In the Lagrangian approach, large sets of virtual particles are integrated within the three-dimensional, time-evolving velocity fields. Over several decades, a variety of tools and methods for this purpose have emerged. Here, we review the state of the art in the field of Lagrangian analysis of ocean velocity data, starting from a fundamental kinematic framework and with a focus on large-scale open ocean applications. Beyond the use of explicit velocity fields, we consider the influence of unresolved physics and dynamics on particle trajectories. We comprehensively list and discuss the tools currently available for tracking virtual particles. We then showcase some of the innovative applications of trajectory data, and conclude with some open questions and an outlook. The overall goal of this review paper is to reconcile some of the different techniques and methods in Lagrangian ocean analysis, while recognising the rich diversity of codes that have and continue to emerge, and the challenges of the coming age of petascale computing

A Meshless Modelling Framework for Simulation and Control of Nonlinear Synthetic Biological Systems

Synthetic biology is a relatively new discipline that incorporates biology and engineering principles. It builds upon the advances in molecular, cell and systems biology and aims to transform these principles to the same effect that synthesis transformed chemistry. What distinguishes synthetic biology from traditional molecular or cellular biology is the focus on design and construction of components (e.g. parts of a cell) that can be modelled, characterised and altered to meet specific performance criteria. Integration of these parts into larger systems is a core principle of synthetic biology. However, unlike some areas of engineering, biology is highly non-linear and less predictable. In this thesis the work that has been conducted to combat some of the complexities associated with dynamic modelling and control of biological systems will be presented. Whilst traditional techniques, such as Orthogonal Collocation on Finite Elements (OCFE) are common place for dynamic modelling they have significant complexity when sampling points are increased and offer discrete solutions or solutions with limited differentiability. To circumvent these issues a meshless modelling framework that incorporates an Artificial Neural Network (ANN) to solve Ordinary Differential Equations (ODEs) and model dynamic processes is utilised. Neural networks can be considered as mesh-free numerical methods as they are likened to approximation schemes where the input data for a design of a network consists of a set of unstructured discrete data points. The use of the ANN provides a solution that is differentiable and is of a closed analytic form, which can be further utilised in subsequent calculations. Whilst there have been advances in modelling biological systems, there has been limited work in controlling their outputs. The benefits of control allow the biological system to alter its state and either upscale production of its primary output, or alter its behaviour within an integrated system. In this thesis a novel meshless Nonlinear Model Predictive Control (NLMPC) framework is presented to address issues related to nonlinearities and complexity. The presented framework is tested on a number of case studies. A significant case study within this work concerns simulation and control of a gene metabolator. The metabolator is a synthetic gene circuit that consists of two metabolite pools which oscillate under the influence of glycolytic flux (a combination of sugars, fatty acids and glycerol). In this work it is demonstrated how glycolytic flux can be used as a control variable for the metabolator. The meshless NLMPC framework allows for both Single-Input Single-Output (SISO) and Multiple-Input Multiple-Output (MIMO) control. The dynamic behaviour of the metabolator allows for both top-down control (using glycolytic flux) and bottom-up control (using acetate). The benefit of using MIMO (by using glycolytic flux and acetate as the control variables) for the metabolator is that it allows the system to reach steady state due to the interactions between the two metabolite pools. Biological systems can also encounter various uncertainties, especially when performing experimental validation. These can have profound effect on the system and can alter the dynamics or overall behaviour. In this work the meshless NLMPC framework addresses uncertainty through the use of Zone Model Predictive Control (Zone MPC), where the control profile is set as a range, rather than a fixed set point. The performance of Zone MPC under the presence of various magnitudes of random disturbances is analysed. The framework is also applied to biological systems architecture, for instance the development of biological circuits from well-characterised and known parts. The framework has shown promise in determining feasible circuits and can be extended in future to incorporate a full list of biological parts. This can give rise to new circuits that could potentially be used in various applications. The meshless NLMPC framework proposed in this work can be extended and applied to other biological systems and heralds a novel method for simulation and control

Mimetic Coastal Ocean Modeling In General Coordinates And Using Machine Learning Based Predictions

Nonlinear internal waves are a ubiquitous and fundamental aspect of the coastal ecosystem understanding. However, they rely on extreme geographical conditions and precise dimensional equilibrium to be captured accurately. The General Curvilinear Coastal Ocean Model (GCCOM) was validated, serial and parallel versions for a set of experiments showcasing stratified and non-hydrostatic flow phenomena. Still, the 3D curvilinear capability has proven to be elusive. We apply cutting-edge numerical methods to improve upon the previously validated GCCOM, elevating it to field-scale capacity. This reformulation of the GCCOM equations uses novel 3D curvilinear mimetic operators, a buoyancy body force, and mimetic upwind and gradient-based momentum equations developed for this work. This model represents the most complete implementation of the 3D curvilinear mimetic operators utilizing the MOLE library or any other mimetic applications in literature to date. Results show it to be more physically accurate and better energy conserving than the validated GCCOM and other similar models, permitting the use of 3D curvilinear grids for arbitrary geometries, parallelizable arbitrary domain decomposition, and order-of-magnitude wider time steps. Additionally, we implement machine learning models to coastal ocean data to predict Dissolved Oxygen (DO) content with supervised methods; results show a Median Absolute Percentage Error (MAPE) of 2-6% for instantaneous indirect readings of DO and 0.18% for five days forecast of DO in coastal areas, using a previously predicted temperature of 1.60% MAPE. Dissolved Oxygen is known to be a critically important component to track in coastal environments but also expensive to measure and almost impossible to model with traditional methods due to high nonlinearity. The ML component of this thesis opens the possibility of high precision indirect estimates of biogeochemical quantities, along with highly accurate time series forecasts and a host of new applications of machine learning to environmental sciences

Controlled Sequential Monte Carlo

Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques to approximate high-dimensional probability distributions and their normalizing constants. They have found numerous applications in statistics and related fields as they can be applied to perform state estimation for non-linear non-Gaussian state space models and Bayesian inference for complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which have a crucial impact on their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. We provide theoretical analysis of our proposed methodology and demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications