Combining statistical methods with dynamical insight to improve nonlinear estimation

Physical processes such as the weather are usually modelled using nonlinear dynamical systems. Statistical methods are found to be difficult to draw the dynamical information from the observations of nonlinear dynamics. This thesis is focusing on combining statistical methods with dynamical insight to improve the nonlinear estimate of the initial states, parameters and future states. In the perfect model scenario (PMS), method based on the Indistin-guishable States theory is introduced to produce initial conditions that are consistent with both observations and model dynamics. Our meth-ods are demonstrated to outperform the variational method, Four-dimensional Variational Assimilation, and the sequential method, En-semble Kalman Filter. Problem of parameter estimation of deterministic nonlinear models is considered within the perfect model scenario where the mathematical structure of the model equations are correct, but the true parameter values are unknown. Traditional methods like least squares are known to be not optimal as it base on the wrong assumption that the distribu-tion of forecast error is Gaussian IID. We introduce two approaches to address the shortcomings of traditional methods. The first approach forms the cost function based on probabilistic forecasting; the second approach focuses on the geometric properties of trajectories in short term while noting the global behaviour of the model in the long term. Both methods are tested on a variety of nonlinear models, the true parameter values are well identified. Outside perfect model scenario, to estimate the current state of the model one need to account the uncertainty from both observatiOnal noise and model inadequacy. Methods assuming the model is perfect are either inapplicable or unable to produce the optimal results. It is almost certain that no trajectory of the model is consistent with an infinite series of observations. There are pseudo-orbits, however, that are consistent with observations and these can be used to estimate the model states. Applying the Indistinguishable States Gradient De-scent algorithm with certain stopping criteria is introduced to find rel-evant pseudo-orbits. The difference between Weakly Constraint Four-dimensional Variational Assimilation (WC4DVAR) method and Indis-tinguishable States Gradient Descent method is discussed. By testing on two system-model pairs, our method is shown to produce more consistent results than the WC4DVAR method. Ensemble formed from the pseudo-orbit generated by Indistinguishable States Gradient Descent method is shown to outperform the Inverse Noise ensemble in estimating the current states. Outside perfect model scenario, we demonstrate that forecast with relevant adjustment can produce better forecast than ignoring the existence of model error and using the model directly to make fore-casts. Measurement based on probabilistic forecast skill is suggested to measure the predictability outside PMS

Perspectives and advances in parameter estimation of nonlinear models.

Nonlinear methodologies to estimate parameters of deterministic nonlinear models are investigated in the case where experimental observations are available and uncertainty sources are present, e.g. model inadequacy, model error and noise. The problem of parameter estimation is interpreted from a nonlinear dynamical time series analysis perspective; however deterministic and probabilistic techniques originated outside the nonlinear deterministic framework are studied, implemented and discussed. Conceptually, the Thesis is divided in two parts that explore two fundamentally different approaches: (a) Bayesian and (b) Geometrical estimation. Both approaches attempt to estimate parameters and model states in the case where the system and the model used to represent it are identical, i.e. Perfect Model Scenario (PMS), even though the implications of the results obtained are considered for Imperfect Model cases. The performance of the resulting model parameter estimates in control monitoring and forecasting of the corresponding system is assessed in an application-oriented fashion and contrasted where possible with system observations, in order to look for a consistent way to combine probabilistic and deterministic approaches. Given the presence of uncertainty in the model used to represent a system and in the observations available, combined methodologies enable us to best interpret the resulting estimates in a probabilistic framework as well as in the context of a particular application. The first conceptual part relates to the REMIND project, which is to find a way to meld advances in nonlinear dynamics with those in Bayesian estimation for both mathematical systems and real industrial settings, i.e. for control monitoring the UK's electricity grid system efficiently. Bayesian inference is used to estimate model parameters and model states using Markov Chain Monte Carlo (MCMC) techniques. For the observations of grid frequency and demand, the operational constraints of the data sets are maintained through the estimation process, for example in the situation where the data are provided at rates that restrict on-line storage and post processing. When MCMC is applied to the Logistic Map, curious behaviour of the convergence of the Markov Chain and in the resulting parameter and states estimates are observed and are suspected to be a consequence of high multimodality in the resulting posterior, which in turn generates estimates with a low dynamical informational content. In the case when the MCMC is applied to a UK's grid frequency dynamical model, the technique is implemented in such a way that gradually transform from the PMS case into a more realistic model representation of the system. Convergence of the MCMC algorithm for the grid frequency models is highly dependent on the quality of operational data, which fails to provide the information required by the tailor-made MCMC implementation. In addition, sanity checks are proposed to establish meaningful convergence of MCMC analyses of time series in general. The second conceptual part explores a new approach to parameter estimation in nonlinear modelling, based on the geometric properties of short term model trajectories, whilst keeping track of the global behaviour of the model. Geometric properties are defined in the context of indistinguishable states theory. Parameter estimates are found for low dimensional chaotic systems by means of Gradient Descent methods (GD) in the PMS. Some of the advances are made possible by means of improving the balance between information extracted from the observations and from the dynamical equations. As a result of this investigation, it is noted that, even with perfect knowledge of system and noise in both models, the uncertainty in the dynamics cannot be distinguished from the uncertainty in the observations. In addition, the Geometric approach and Bayesian approach of the problem of model parameter and state estimation for nonlinear models in the PMS are compared aiming to distinguish them based on dynamical features of the estimates. In the Bayesian formulation there are still fundamental challenges when a perfect model is not available

On shadowing methods for data assimilation

Combining orbits from a model of a (chaotic) dynamical system with measured data to arrive at an improved estimate of the state of a physical system is known as data assimilation. This thesis deals with various algorithms for data assimilation. These algorithms are based on shadowing. Shadowing is a concept from the theory of dynamical systems. When a dynamical system has the property that an exact orbit of the dynamical system is located in a neighborhood of each pseudo-orbit, then this exact orbit shadows the pseudo-orbit. Shadowing can be used to show that a numerical solution of a dynamical system is located in a neighborhood of an exact solution. Shadowing refinement is a numerical technique in which an improved approximation to an exact solution is found from a pseudo-orbit. It is possible to use a shadowing refinement technique for data assimilation. Starting from observations, Newton's method is applied to approximate a zero of a cost operator, where the cost operator assigns costs to deviations from model solutions. The algorithms of Chapter 2 are based on a numerical time-dependent split between stable and unstable directions. The algorithm uses time-dependent projections onto the unstable subspace determined by using Lyapunov exponents and Lyapunov vectors. A shadowing algorithm is used in the unstable subspace, while synchronization is used in the stable subspace. The method is further extended to include parameter estimation and to some cases where only partial observations are available. Chapter 3 discusses data assimilation for imperfect models. Through regularization according to the Levenberg-Marquardt method, imperfections in the model are considered. It also describes how the shadowing method compares, both analytically and numerically, with the weak constraint 4DVar method and shows that the shadowing method is consistent with the measurement error distribution, which is not the case for the weak constraint 4DVar method. This effect is particularly evident when there are fewer observations. Moreover, when there are few observations, they have a smaller impact on unobserved variables in the shadowing method than in the weak constraint 4DVar method. Chapter 4 extends the method of Chapter 2 to other cases of partial observations, in a similar way to Chapter 3. Local convergence to a solution manifold is proved and a lower bound on an algorithmic time step is provided. Numerical experiments with the Lorenz-'63 and Lorenz-'96 models show convergence of the algorithm and further show that the method compares favorably with the weak constraint 4DVar method and another shadowing method called pseudo-orbit data assimilation. Chapter 5 further develops the method of the previous chapters. The algorithm is extended to an ensemble of states for estimating uncertainties of the algorithm, based on the concept of indistinguishable states. The chapter also includes some proofs on uniqueness, accuracy and consistency of the algorithm. The algorithm is applied to an imperfect model to show how the unmodeled components of the model can be estimated using the data assimilation algorithm

On Variational Data Assimilation in Continuous Time

Variational data assimilation in continuous time is revisited. The central techniques applied in this paper are in part adopted from the theory of optimal nonlinear control. Alternatively, the investigated approach can be considered as a continuous time generalisation of what is known as weakly constrained four dimensional variational assimilation (WC--4DVAR) in the geosciences. The technique allows to assimilate trajectories in the case of partial observations and in the presence of model error. Several mathematical aspects of the approach are studied. Computationally, it amounts to solving a two point boundary value problem. For imperfect models, the trade off between small dynamical error (i.e. the trajectory obeys the model dynamics) and small observational error (i.e. the trajectory closely follows the observations) is investigated. For (nearly) perfect models, this trade off turns out to be (nearly) trivial in some sense, yet allowing for some dynamical error is shown to have positive effects even in this situation. The presented formalism is dynamical in character; no assumptions need to be made about the presence (or absence) of dynamical or observational noise, let alone about their statistics.Comment: 28 Pages, 12 Figure

Shadowing-based data assimilation method for partially observed models

In this article we develop further an algorithm for data assimilation based upon a shadowing refinement technique [de Leeuw et al., SIAM J. Appl. Dyn. Syst., 17 (2018), pp. 2446-2477] to take partial observations into account. Our method is based on a regularized Gauss-Newton method. We prove local convergence to the solution manifold and provide a lower bound on the algorithmic time step. We use numerical experiments with the Lorenz 63 and Lorenz 96 models to illustrate convergence of the algorithm and show that the results compare favorably with a variational technique --- weak-constraint four-dimensional variational method --- and a shadowing technique-pseudo-orbit data assimilation. Numerical experiments show that a preconditioner chosen based on a cost function allows the algorithm to find an orbit of the dynamical system in the vicinity of the true solution