Gaussian processes for state space models and change point detection

This thesis details several applications of Gaussian processes (GPs) for enhanced time series modeling. We first cover different approaches for using Gaussian processes in time series problems. These are extended to the state space approach to time series in two different problems. We also combine Gaussian processes and Bayesian online change point detection (BOCPD) to increase the generality of the Gaussian process time series methods. These methodologies are evaluated on predictive performance on six real world data sets, which include three environmental data sets, one financial, one biological, and one from industrial well drilling. Gaussian processes are capable of generalizing standard linear time series models. We cover two approaches: the Gaussian process time series model (GPTS) and the autoregressive Gaussian process (ARGP). We cover a variety of methods that greatly reduce the computational and memory complexity of Gaussian process approaches, which are generally cubic in computational complexity. Two different improvements to state space based approaches are covered. First, Gaussian process inference and learning (GPIL) generalizes linear dynamical systems (LDS), for which the Kalman filter is based, to general nonlinear systems for nonparametric system identification. Second, we address pathologies in the unscented Kalman filter (UKF). We use Gaussian process optimization (GPO) to learn UKF settings that minimize the potential for sigma point collapse. We show how to embed mentioned Gaussian process approaches to time series into a change point framework. Old data, from an old regime, that hinders predictive performance is automatically and elegantly phased out. The computational improvements for Gaussian process time series approaches are of even greater use in the change point framework. We also present a supervised framework learning a change point model when change point labels are available in training.I would like to thank Rockwell Collins, formerly DataPath, Inc., which funded my studentship

Computer Algebra Solving of Second Order ODEs Using Symmetry Methods

An update of the ODEtools Maple package, for the analytical solving of 1st and 2nd order ODEs using Lie group symmetry methods, is presented. The set of routines includes an ODE-solver and user-level commands realizing most of the relevant steps of the symmetry scheme. The package also includes commands for testing the returned results, and for classifying 1st and 2nd order ODEs.Comment: 24 pages, LaTeX, Soft-package (On-Line help) and sample MapleV sessions available at: http://dft.if.uerj.br/odetools.htm or http://lie.uwaterloo.ca/odetools.ht

Feedback methods for inverse simulation of dynamic models for engineering systems applications

Inverse simulation is a form of inverse modelling in which computer simulation methods are used to find the time histories of input variables that, for a given model, match a set of required output responses. Conventional inverse simulation methods for dynamic models are computationally intensive and can present difficulties for high-speed applications. This paper includes a review of established methods of inverse simulation,giving some emphasis to iterative techniques that were first developed for aeronautical applications. It goes on to discuss the application of a different approach which is based on feedback principles. This feedback method is suitable for a wide range of linear and nonlinear dynamic models and involves two distinct stages. The first stage involves design of a feedback loop around the given simulation model and, in the second stage, that closed-loop system is used for inversion of the model. Issues of robustness within closed-loop systems used in inverse simulation are not significant as there are no plant uncertainties or external disturbances. Thus the process is simpler than that required for the development of a control system of equivalent complexity. Engineering applications of this feedback approach to inverse simulation are described through case studies that put particular emphasis on nonlinear and multi-input multi-output models

Climate dynamics and fluid mechanics: Natural variability and related uncertainties

The purpose of this review-and-research paper is twofold: (i) to review the role played in climate dynamics by fluid-dynamical models; and (ii) to contribute to the understanding and reduction of the uncertainties in future climate-change projections. To illustrate the first point, we focus on the large-scale, wind-driven flow of the mid-latitude oceans which contribute in a crucial way to Earth's climate, and to changes therein. We study the low-frequency variability (LFV) of the wind-driven, double-gyre circulation in mid-latitude ocean basins, via the bifurcation sequence that leads from steady states through periodic solutions and on to the chaotic, irregular flows documented in the observations. This sequence involves local, pitchfork and Hopf bifurcations, as well as global, homoclinic ones. The natural climate variability induced by the LFV of the ocean circulation is but one of the causes of uncertainties in climate projections. Another major cause of such uncertainties could reside in the structural instability in the topological sense, of the equations governing climate dynamics, including but not restricted to those of atmospheric and ocean dynamics. We propose a novel approach to understand, and possibly reduce, these uncertainties, based on the concepts and methods of random dynamical systems theory. As a very first step, we study the effect of noise on the topological classes of the Arnol'd family of circle maps, a paradigmatic model of frequency locking as occurring in the nonlinear interactions between the El Nino-Southern Oscillations (ENSO) and the seasonal cycle. It is shown that the maps' fine-grained resonant landscape is smoothed by the noise, thus permitting their coarse-grained classification. This result is consistent with stabilizing effects of stochastic parametrization obtained in modeling of ENSO phenomenon via some general circulation models.Comment: Invited survey paper for Special Issue on The Euler Equations: 250 Years On, in Physica D: Nonlinear phenomen

Time Dependent Saddle Node Bifurcation: Breaking Time and the Point of No Return in a Non-Autonomous Model of Critical Transitions

There is a growing awareness that catastrophic phenomena in biology and medicine can be mathematically represented in terms of saddle-node bifurcations. In particular, the term `tipping', or critical transition has in recent years entered the discourse of the general public in relation to ecology, medicine, and public health. The saddle-node bifurcation and its associated theory of catastrophe as put forth by Thom and Zeeman has seen applications in a wide range of fields including molecular biophysics, mesoscopic physics, and climate science. In this paper, we investigate a simple model of a non-autonomous system with a time-dependent parameter $p(\tau)$ and its corresponding `dynamic' (time-dependent) saddle-node bifurcation by the modern theory of non-autonomous dynamical systems. We show that the actual point of no return for a system undergoing tipping can be significantly delayed in comparison to the {\em breaking time} $\hat{\tau}$ at which the corresponding autonomous system with a time-independent parameter $p_{a}= p(\hat{\tau})$ undergoes a bifurcation. A dimensionless parameter $\alpha=\lambda p_0^3V^{-2}$ is introduced, in which $\lambda$ is the curvature of the autonomous saddle-node bifurcation according to parameter $p(\tau)$, which has an initial value of $p_{0}$ and a constant rate of change $V$. We find that the breaking time $\hat{\tau}$ is always less than the actual point of no return $\tau^*$ after which the critical transition is irreversible; specifically, the relation $\tau^*-\hat{\tau}\simeq 2.338(\lambda V)^{-\frac{1}{3}}$ is analytically obtained. For a system with a small $\lambda V$, there exists a significant window of opportunity $(\hat{\tau},\tau^*)$ during which rapid reversal of the environment can save the system from catastrophe

Initial Conditions for Models of Dynamical Systems

The long-time behaviour of many dynamical systems may be effectively predicted by a low-dimensional model that describes the evolution of a reduced set of variables. We consider the question of how to equip such a low-dimensional model with appropriate initial conditions, so that it faithfully reproduces the long-term behaviour of the original high-dimensional dynamical system. Our method involves putting the dynamical system into normal form, which not only generates the low-dimensional model, but also provides the correct initial conditions for the model. We illustrate the method with several examples. Keywords: normal form, isochrons, initialisation, centre manifoldComment: 24 pages in standard LaTeX, 66K, no figure

Linear frictional forces cause orbits to neither circularize nor precess

For the undamped Kepler potential the lack of precession has historically been understood in terms of the Runge-Lenz symmetry. For the damped Kepler problem this result may be understood in terms of the generalization of Poisson structure to damped systems suggested recently by Tarasov[1]. In this generalized algebraic structure the orbit-averaged Runge-Lenz vector remains a constant in the linearly damped Kepler problem to leading order in the damping coeComment: 16 pages. 1 figure, Rewrite for resubmissio

Numerical Integration and Dynamic Discretization in Heuristic Search Planning over Hybrid Domains

In this paper we look into the problem of planning over hybrid domains, where change can be both discrete and instantaneous, or continuous over time. In addition, it is required that each state on the trajectory induced by the execution of plans complies with a given set of global constraints. We approach the computation of plans for such domains as the problem of searching over a deterministic state model. In this model, some of the successor states are obtained by solving numerically the so-called initial value problem over a set of ordinary differential equations (ODE) given by the current plan prefix. These equations hold over time intervals whose duration is determined dynamically, according to whether zero crossing events take place for a set of invariant conditions. The resulting planner, FS+, incorporates these features together with effective heuristic guidance. FS+ does not impose any of the syntactic restrictions on process effects often found on the existing literature on Hybrid Planning. A key concept of our approach is that a clear separation is struck between planning and simulation time steps. The former is the time allowed to observe the evolution of a given dynamical system before committing to a future course of action, whilst the later is part of the model of the environment. FS+ is shown to be a robust planner over a diverse set of hybrid domains, taken from the existing literature on hybrid planning and systems.Comment: 17 page