Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation

The Ensemble Kalman filter and Ensemble square root filters are data assimilation methods used to combine high dimensional nonlinear models with observed data. These methods have proved to be indispensable tools in science and engineering as they allow computationally cheap, low dimensional ensemble state approximation for extremely high dimensional turbulent forecast models. From a theoretical perspective, these methods are poorly understood, with the exception of a recently established but still incomplete nonlinear stability theory. Moreover, recent numerical and theoretical studies of catastrophic filter divergence have indicated that stability is a genuine mathematical concern and can not be taken for granted in implementation. In this article we propose a simple modification of ensemble based methods which resolves these stability issues entirely. The method involves a new type of adaptive covariance inflation, which comes with minimal additional cost. We develop a complete nonlinear stability theory for the adaptive method, yielding Lyapunov functions and geometric ergodicity under weak assumptions. We present numerical evidence which suggests the adaptive methods have improved accuracy over standard methods and completely eliminate catastrophic filter divergence. This enhanced stability allows for the use of extremely cheap, unstable forecast integrators, which would otherwise lead to widespread filter malfunction.Comment: 34 pages. 4 figure

Nonlinear stability and ergodicity of ensemble based Kalman filters

The ensemble Kalman filter (EnKF) and ensemble square root filter (ESRF) are data assimilation methods used to combine high dimensional, nonlinear dynamical models with observed data. Despite their widespread usage in climate science and oil reservoir simulation, very little is known about the long-time behavior of these methods and why they are effective when applied with modest ensemble sizes in large dimensional turbulent dynamical systems. By following the basic principles of energy dissipation and controllability of filters, this paper establishes a simple, systematic and rigorous framework for the nonlinear analysis of EnKF and ESRF with arbitrary ensemble size, focusing on the dynamical properties of boundedness and geometric ergodicity. The time uniform boundedness guarantees that the filter estimate will not diverge to machine infinity in finite time, which is a potential threat for EnKF and ESQF known as the catastrophic filter divergence. Geometric ergodicity ensures in addition that the filter has a unique invariant measure and that initialization errors will dissipate exponentially in time. We establish these results by introducing a natural notion of observable energy dissipation. The time uniform bound is achieved through a simple Lyapunov function argument, this result applies to systems with complete observations and strong kinetic energy dissipation, but also to concrete examples with incomplete observations. With the Lyapunov function argument established, the geometric ergodicity is obtained by verifying the controllability of the filter processes; in particular, such analysis for ESQF relies on a careful multivariate perturbation analysis of the covariance eigen-structure.Comment: 38 page

Kalman-filter control schemes for fringe tracking. Development and application to VLTI/GRAVITY

The implementation of fringe tracking for optical interferometers is inevitable when optimal exploitation of the instrumental capacities is desired. Fringe tracking allows continuous fringe observation, considerably increasing the sensitivity of the interferometric system. In addition to the correction of atmospheric path-length differences, a decent control algorithm should correct for disturbances introduced by instrumental vibrations, and deal with other errors propagating in the optical trains. We attempt to construct control schemes based on Kalman filters. Kalman filtering is an optimal data processing algorithm for tracking and correcting a system on which observations are performed. As a direct application, control schemes are designed for GRAVITY, a future four-telescope near-infrared beam combiner for the Very Large Telescope Interferometer (VLTI). We base our study on recent work in adaptive-optics control. The technique is to describe perturbations of fringe phases in terms of an a priori model. The model allows us to optimize the tracking of fringes, in that it is adapted to the prevailing perturbations. Since the model is of a parametric nature, a parameter identification needs to be included. Different possibilities exist to generalize to the four-telescope fringe tracking that is useful for GRAVITY. On the basis of a two-telescope Kalman-filtering control algorithm, a set of two properly working control algorithms for four-telescope fringe tracking is constructed. The control schemes are designed to take into account flux problems and low-signal baselines. First simulations of the fringe-tracking process indicate that the defined schemes meet the requirements for GRAVITY and allow us to distinguish in performance. In a future paper, we will compare the performances of classical fringe tracking to our Kalman-filter control.Comment: 17 pages, 8 figures, accepted for publication in A&

Computable infinite dimensional filters with applications to discretized diffusion processes

Let us consider a pair signal-observation ((xn,yn),n 0) where the unobserved signal (xn) is a Markov chain and the observed component is such that, given the whole sequence (xn), the random variables (yn) are independent and the conditional distribution of yn only depends on the corresponding state variable xn. The main problems raised by these observations are the prediction and filtering of (xn). We introduce sufficient conditions allowing to obtain computable filters using mixtures of distributions. The filter system may be finite or infinite dimensional. The method is applied to the case where the signal xn = Xn is a discrete sampling of a one dimensional diffusion process: Concrete models are proved to fit in our conditions. Moreover, for these models, exact likelihood inference based on the observation (y0,...,yn) is feasable

Norm minimized Scattering Data from Intensity Spectra

We apply the $l_1$ minimizing technique of compressive sensing (CS) to non-linear quadratic observations. For the example of coherent X-ray scattering we provide the formulae for a Kalman filter approach to quadratic CS and show how to reconstruct the scattering data from their spatial intensity distribution.Comment: 26 pages, 10 figures, reordered section

Kalman Filtering with Unknown Noise Covariances

Since it is often difficult to identify the noise covariances for a Kalman filter, they are commonly considered design variables. If so, we can as well try to choose them so that the corresponding Kalman filter has some nice form. In this paper, we introduce a one-parameter subfamily of Kalman filters with the property that the covariance parameters cancel in the expression for the Kalman gain. We provide a simple criterion which guarantees that the implicitly defined process covariance matrix is positive definite