Design and implementation of data assimilation methods based on Cholesky decomposition

In Data Assimilation, analyses of a system are obtained by combining a previous numerical model of the system and observations or measurements from it. These numerical models are typically expressed as a set of ordinary differential equations and/or a set of partial differential equations wherein all knowledge about dynamics and physics of, for instance, the ocean and or the atmosphere are encapsulated. We treat numerical forecasts and observations as random variables and therefore, error dynamics can be estimated by using Bayes’ rule. For the estimation of hyper-parameters in error distributions, an ensemble of model realizations is employed. In practice, model resolutions are several order of magnitudes larger than ensemble sizes, and consequently, sampling errors impact the quality of analysis corrections and besides, models can be highly non-linear and well-common Gaussian assumptions on prior errors can be broken. To overcome these situations, we replace prior errors by a mixture of Gaussians and even more, precision covariance matrices intra-clusters are estimated by means of the modified Cholesky decomposition. Four different methods are proposed, namely the Posterior EnKF with its deterministic and stochastic variations, a Non-Gaussian method and a MCMC filter, which used the Bickel-Levina estimator; these methods are based on a modified Cholesky decomposition and tested with the Lorenz 96 model. Their implementations are shown to provide equivalent solutions compared to another EnKF methods like the LETKF and the EnSRF.DoctoradoDoctor en Ingeniería de Sistemas y Computació

A Unification of Ensemble Square Root Kalman Filters

In recent years, several ensemble-based Kalman filter algorithms have been developed that have been classified as ensemble square-root Kalman filters. Parallel to this development, the SEIK (Singular ``Evolutive'' Interpolated Kalman) filter has been introduced and applied in several studies. Some publications note that the SEIK filter is an ensemble Kalman filter or even an ensemble square-root Kalman filter. This study examines the relation of the SEIK filter to ensemble square-root filters in detail. It shows that the SEIK filter is indeed an ensemble-square root Kalman filter. Furthermore, a variant of the SEIK filter, the Error Subspace Transform Kalman Filter (ESTKF), is presented that results in identical ensemble transformations to those of the Ensemble Transform Kalman Filter (ETKF) while having a slightly lower computational cost. Numerical experiments are conducted to compare the performance of three filters (SEIK, ETKF, and ESTKF) using deterministic and random ensemble transformations. The results show better performance for the ETKF and ESTKF methods over the SEIK filter as long as this filter is not applied with a symmetric square root. The findings unify the separate developments that have been performed for the SEIK filter and the other ensemble square-root Kalman filters

Efficient Matrix-Free Ensemble Kalman Filter Implementations: Accounting for Localization

This chapter discusses efficient and practical matrix-free implementations of the ensemble Kalman filter (EnKF) in order to account for localization during the assimilation of observations. In the EnKF context, an ensemble of model realizations is utilized in order to estimate the moments of its underlying error distribution. Since ensemble members come at high computational costs (owing to current operational model resolutions) ensemble sizes are constrained by the hundreds while, typically, their error distributions range in the order of millions. This induces spurious correlations in estimates of prior error correlations when these are approximated via the ensemble covariance matrix. Localization methods are commonly utilized in order to counteract this effect. EnKF implementations in this context are based on a modified Cholesky decomposition. Different flavours of Cholesky-based filters are discussed in this chapter. Furthermore, the computational effort in all formulations is linear with regard to model resolutions. Experimental tests are performed making use of the Lorenz 96 model. The results reveal that, in terms of root-mean-square-errors, all formulations perform equivalently

Improved tabu search and simulated annealing methods for nonlinear data assimilation

Nonlinear data assimilation can be a very challenging task. Four local search methods are proposed for nonlinear data assimilation in this paper. The methods work as follows: At each iteration, the observation operator is linearized around the current solution, and a gradient approximation of the three dimensional variational (3D-Var) cost function is obtained. Then, samples along potential steepest descent directions of the 3D-Var cost function are generated, and the acceptance/rejection criteria for such samples are similar to those proposed by the Tabu Search and the Simulated Annealing framework. In addition, such samples can be drawn within certain sub-spaces so as to reduce the computational effort of computing search directions. Once a posterior mode is estimated, matrix-free ensemble Kalman filter approaches can be implemented to estimate posterior members. Furthermore, the convergence of the proposed methods is theoretically proven based on the necessary assumptions and conditions. Numerical experiments have been performed by using the Lorenz-96 model. The numerical results show that the cost function values on average can be reduced by several orders of magnitudes by using the proposed methods. Even more, the proposed methods can converge faster to posterior modes when sub-space approximations are employed to reduce the computational efforts among iterations

Sequential data assimilation methods for atmospheric general circulation models

The data assimilation (DA) process has gained some spotlight in recent years as computers have become more powerful, and models more complex. Even so, most natural phenomena have many correlations among variables that are very challenging to capture. In this proposal, we discuss the impact of an intermediate step in the leaping strategy used as a numerical integrator for Atmospheric General Circulation Models during the assimilation process, and its explicit update, particularly, for the Simplified Parameterizations, privitivE-Equation DYnamics model, nicknamed as SPEEDY. Using literature validated formulations of the Ensemble Kalman Filters the Local Ensemble Kalman Filter (LEnKF), Local Ensemble Transform Kalman Filter (LETKF), and the Ensemble Kalman Filter based on a Modified Cholesky Decomposition (EnKF-MC) experimental test are performed using the leaping step in the update process, and using only the forecast step, and letting the model propagate the updates. For the EnKF-MC formulation, we propose a formulation onto the observations space. As well, we present an intuitive Python package to perform sequential data assimilation on atmospheric general circulation models. We denote our package by Applied Math and Computer Science Lab - Data Assimilation AMLCS-DA. This package contains the efficient implementations of the previously mentioned formulations. The results reveal that our proposed framework can properly estimate model variables within reasonable accuracies in terms of Root-Mean-Square-Error when we update only the forecast state, even when using sparse operational observators (25%, 11%, 6%, 4%).MaestríaMagister en Ingeniería de Sistemas y Computació

A Stochastic Covariance Shrinkage Approach in Ensemble Transform Kalman Filtering

The Ensemble Kalman Filters (EnKF) employ a Monte-Carlo approach to represent covariance information, and are affected by sampling errors in operational settings where the number of model realizations is much smaller than the model state dimension. To alleviate the effects of these errors EnKF relies on model-specific heuristics such as covariance localization, which takes advantage of the spatial locality of correlations among the model variables. This work proposes an approach to alleviate sampling errors that utilizes a locally averaged-in-time dynamics of the model, described in terms of a climatological covariance of the dynamical system. We use this covariance as the target matrix in covariance shrinkage methods, and develop a stochastic covariance shrinkage approach where synthetic ensemble members are drawn to enrich both the ensemble subspace and the ensemble transformation