Existence and uniqueness for four-dimensional variational data assimilation in discrete time

Variational techniques for data assimilation, i.e., estimating orbits of dynamical models from observations, are revisited. It is shown that under mild hypotheses a solution to this variational problem exists. Using ideas from optimal control theory, the problem of uniqueness is investigated and a number of results (well known from optimal control) are established in the present context. The value function is introduced as the minimal cost over all feasible trajectories starting from a given initial condition. By combining the necessary conditions with an envelope theorem, it is shown that the solution is unique if and only if the value function has a derivative at the given initial condition. Further, the value function is Lipschitz and hence has a derivative for almost all (with respect to the Lebesgue measure) initial conditions. Several examples are studied which demonstrate that points of nondifferentiability of the value function (and hence nonuniqueness of solutions) are nevertheless to be expected in practice

Asymptotic stability of the optimal filter for random chaotic maps

The asymptotic stability of the optimal filtering process in discrete time is revisited. The filtering process is the conditional probability of the state of a Markov process, called the signal process, given a series of observations. Asymptotic stability means that the distance between the true filtering process and a wrongly initialised filter converges to zero as time progresses. In the present setting, the signal process arises through iterating an i.i.d. sequence of uniformly expanding random maps. It is showed that for such a signal, the asymptotic stability is exponential provided that its initial conditions are sufficiently smooth. Similar to previous work on this problem, Hilbert’s projective metric on cones is employed as well as certain mixing properties of the signal, albeit with important differences. Mixing and ultimately filter stability in the present situation are due to the expanding dynamics rather than the stochasticity of the signal process. In fact, the conditions even permit iterations of a fixed (nonrandom) expanding map

Uniform calibration tests for forecasting systems with small lead time

A long noted difficulty when assessing calibration (or reliability) of forecasting systems is that calibration, in general, is a hypothesis not about a finite dimensional parameter but about an entire functional relationship. A calibrated probability forecast for binary events for instance should equal the conditional probability of the event given the forecast, whatever the value of the forecast. A new class of tests is presented that are based on estimating the cumulative deviations from calibration. The supremum of those deviations is taken as a test statistic, and the asymptotic distribution of the test statistic is established rigorously. It turns out to be universal, provided the forecasts “look one step ahead” only, or in other words, verify at the next time step in the future. The new tests apply to various different forecasting problems and are compared with established approaches which work in a regression based framework. In comparison to those approaches, the new tests develop power against a wider class of alternatives. Numerical experiments for both artificial data as well as operational weather forecasting systems are presented, and possible extensions to longer lead times are discussed

Existence and uniqueness for variational data assimilation in continuous time

A variant of the optimal control problem is considered which is nonstandard in that the performance index contains "stochastic" integrals, that is, integrals against very irregular functions. The motivation for considering such performance indices comes from dynamical estimation problems where observed time series need to be "fitted" with trajectories of dynamical models. The observations may be contaminated with white noise, which gives rise to the nonstandard performance indices. Problems of this kind appear in engineering, physics, and the geosciences where this is referred to as data assimilation. The fact that typical models in the geosciences do not satisfy linear growth nor monotonicity conditions represents an additional difficulty. Pathwise existence of minimisers is obtained, along with a maximum principle as well as preliminary results in dynamic programming. The results also extend previous work on the maximum aposteriori estimator of trajectories of diffusion processes

Almost sure error bounds for data assimilation in dissipative systems with unbounded observation noise

Data assimilation is uniquely challenging in weather forecasting due to the high dimensionality of the employed models and the nonlinearity of the governing equations. Although current operational schemes are used successfully, our understanding of their long-term error behaviour is still incomplete. In this work, we study the error of some simple data assimilation schemes in the presence of unbounded (e.g. Gaussian) noise on a wide class of dissipative dynamical systems with certain properties, including the Lorenz models and the 2D incompressible Navier-Stokes equations. We exploit the properties of the dynamics to derive analytic bounds on the long-term error for individual realisations of the noise in time. These bounds are proportional to the variance of the noise. Furthermore, we find that the error exhibits a form of stationary behaviour, and in particular an accumulation of error does not occur. This improves on previous results in which either the noise was bounded or the error was considered in expectation only

The evolution of a non-autonomous chaotic system under non-periodic forcing: a climate change example

Complex Earth System Models are widely utilised to make conditional statements about the future climate under some assumptions about changes in future atmospheric greenhouse gas concentrations; these statements are often referred to as climate projections. The models themselves are high-dimensional nonlinear systems and it is common to discuss their behaviour in terms of attractors and low-dimensional nonlinear systems such as the canonical Lorenz `63 system. In a non-autonomous situation, for instance due to anthropogenic climate change, the relevant object is sometimes considered to be the pullback or snapshot attractor. The pullback attractor, however, is a collection of {\em all} plausible states of the system at a given time and therefore does not take into consideration our knowledge of the current state of the Earth System when making climate projections, and are therefore not very informative regarding annual to multi-decadal climate projections. In this article, we approach the problem of measuring and interpreting the mid-term climate of a model by using a low-dimensional, climate-like, nonlinear system with three timescales of variability, and non-periodic forcing. We introduce the concept of an {\em evolution set} which is dependent on the starting state of the system, and explore its links to different types of initial condition uncertainty and the rate of external forcing. We define the {\em convergence time} as the time that it takes for the distribution of one of the dependent variables to lose memory of its initial conditions. We suspect a connection between convergence times and the classical concept of mixing times but the precise nature of this connection needs to be explored. These results have implications for the design of influential climate and Earth System Model ensembles, and raise a number of issues of mathematical interest.Comment: The model output data used in this study is freely available on Zenodo: https://doi.org/10.5281/zenodo.836802