A Bayesian Consistent Dual Ensemble Kalman Filter for State-Parameter Estimation in Subsurface Hydrology

Ensemble Kalman filtering (EnKF) is an efficient approach to addressing uncertainties in subsurface groundwater models. The EnKF sequentially integrates field data into simulation models to obtain a better characterization of the model's state and parameters. These are generally estimated following joint and dual filtering strategies, in which, at each assimilation cycle, a forecast step by the model is followed by an update step with incoming observations. The Joint-EnKF directly updates the augmented state-parameter vector while the Dual-EnKF employs two separate filters, first estimating the parameters and then estimating the state based on the updated parameters. In this paper, we reverse the order of the forecast-update steps following the one-step-ahead (OSA) smoothing formulation of the Bayesian filtering problem, based on which we propose a new dual EnKF scheme, the Dual-EnKF$_{\rm OSA}$. Compared to the Dual-EnKF, this introduces a new update step to the state in a fully consistent Bayesian framework, which is shown to enhance the performance of the dual filtering approach without any significant increase in the computational cost. Numerical experiments are conducted with a two-dimensional synthetic groundwater aquifer model to assess the performance and robustness of the proposed Dual-EnKF$_{\rm OSA}$, and to evaluate its results against those of the Joint- and Dual-EnKFs. The proposed scheme is able to successfully recover both the hydraulic head and the aquifer conductivity, further providing reliable estimates of their uncertainties. Compared with the standard Joint- and Dual-EnKFs, the proposed scheme is found more robust to different assimilation settings, such as the spatial and temporal distribution of the observations, and the level of noise in the data. Based on our experimental setups, it yields up to 25% more accurate state and parameters estimates

Towards an end-to-end analysis and prediction system for weather, climate, and Marine applications in the Red Sea

AbstractThe Red Sea, home to the second-longest coral reef system in the world, is a vital resource for the Kingdom of Saudi Arabia. The Red Sea provides 90% of the Kingdom’s potable water by desalinization, supporting tourism, shipping, aquaculture, and fishing industries, which together contribute about 10%–20% of the country’s GDP. All these activities, and those elsewhere in the Red Sea region, critically depend on oceanic and atmospheric conditions. At a time of mega-development projects along the Red Sea coast, and global warming, authorities are working on optimizing the harnessing of environmental resources, including renewable energy and rainwater harvesting. All these require high-resolution weather and climate information. Toward this end, we have undertaken a multipronged research and development activity in which we are developing an integrated data-driven regional coupled modeling system. The telescopically nested components include 5-km- to 600-m-resolution atmospheric models to address weather and climate challenges, 4-km- to 50-m-resolution ocean models with regional and coastal configurations to simulate and predict the general and mesoscale circulation, 4-km- to 100-m-resolution ecosystem models to simulate the biogeochemistry, and 1-km- to 50-m-resolution wave models. In addition, a complementary probabilistic transport modeling system predicts dispersion of contaminant plumes, oil spill, and marine ecosystem connectivity. Advanced ensemble data assimilation capabilities have also been implemented for accurate forecasting. Resulting achievements include significant advancement in our understanding of the regional circulation and its connection to the global climate, development, and validation of long-term Red Sea regional atmospheric–oceanic–wave reanalyses and forecasting capacities. These products are being extensively used by academia, government, and industry in various weather and marine studies and operations, environmental policies, renewable energy applications, impact assessment, flood forecasting, and more.</jats:p

Towards an end-to-end analysis and prediction system for weather, climate, and marine applications in the Red Sea

Author Posting. © American Meteorological Society, 2021. This article is posted here by permission of American Meteorological Society for personal use, not for redistribution. The definitive version was published in Bulletin of the American Meteorological Society 102(1), (2021): E99-E122, https://doi.org/10.1175/BAMS-D-19-0005.1.The Red Sea, home to the second-longest coral reef system in the world, is a vital resource for the Kingdom of Saudi Arabia. The Red Sea provides 90% of the Kingdom’s potable water by desalinization, supporting tourism, shipping, aquaculture, and fishing industries, which together contribute about 10%–20% of the country’s GDP. All these activities, and those elsewhere in the Red Sea region, critically depend on oceanic and atmospheric conditions. At a time of mega-development projects along the Red Sea coast, and global warming, authorities are working on optimizing the harnessing of environmental resources, including renewable energy and rainwater harvesting. All these require high-resolution weather and climate information. Toward this end, we have undertaken a multipronged research and development activity in which we are developing an integrated data-driven regional coupled modeling system. The telescopically nested components include 5-km- to 600-m-resolution atmospheric models to address weather and climate challenges, 4-km- to 50-m-resolution ocean models with regional and coastal configurations to simulate and predict the general and mesoscale circulation, 4-km- to 100-m-resolution ecosystem models to simulate the biogeochemistry, and 1-km- to 50-m-resolution wave models. In addition, a complementary probabilistic transport modeling system predicts dispersion of contaminant plumes, oil spill, and marine ecosystem connectivity. Advanced ensemble data assimilation capabilities have also been implemented for accurate forecasting. Resulting achievements include significant advancement in our understanding of the regional circulation and its connection to the global climate, development, and validation of long-term Red Sea regional atmospheric–oceanic–wave reanalyses and forecasting capacities. These products are being extensively used by academia, government, and industry in various weather and marine studies and operations, environmental policies, renewable energy applications, impact assessment, flood forecasting, and more.The development of the Red Sea modeling system is being supported by the Virtual Red Sea Initiative and the Competitive Research Grants (CRG) program from the Office of Sponsored Research at KAUST, Saudi Aramco Company through the Saudi ARAMCO Marine Environmental Center at KAUST, and by funds from KAEC, NEOM, and RSP through Beacon Development Company at KAUST

UNSUPERVISED SIGNAL RESTORATION IN PARTIALLY OBSERVED MARKOV CHAINS

An important problem in signal processing consists in estimating an unobservable process x = {xn}n∈IN from an observed process y = {yn}n∈IN. In Linear Gaussian Hidden Markov Chains (LGHMC), recursive solutions are given by Kalman-like Bayesian restoration algorithms. In this paper, we consider the more general framework of Linear Gaussian Triplet Markov Chains (LGTMC), i.e. of models in which the triplet (x, r, y) (where r = {rn}n∈IN is some additional process) is Markovian and Gaussian. We address unsupervised restoration in LGTMC by extending to LGTMC the EM parameter estimation algorithm which was already developed in classical state-space models. 1

Variational Bayesian Kalman Filtering in Dynamical Tomography

International audienc

Direct versus prediction-based particle filter algorithm

Particle Filtering (PF) algorithms propagate in time a Monte Carlo (MC) approximation of the a posteriori filtering measure in a Hidden Markov Chain (HMC) model. In this paper we first shed some new light on two classical PF algorithms, which can be considered as natural MC implementations of two two-step direct recursive formulas for computing the filtering distribution. We next address the Particle Prediction (PP) problem, which happens to be simpler than the PF problem because the optimal prediction conditional importance distribution (CID) is much easier to sample from. Motivated by this result we finally develop two PP-based PF algorithms, and we compare our algorithms via simulations

On Bayesian Fixed-Interval Smoothing Algorithms

Fixed-interval Bayesian smoothing in state-space systems has been addressed for a long time. However, as far as the measurement noise is concerned, only two cases have been addressed so far: the regular case, i.e. with positive definite covariance matrix; and the perfect measurement case, i.e. with zero measurement noise. In this paper we address the smoothing problem in the intermediate case where the measurement noise covariance is positive semi definite (p.s.d.) with arbitrary rank. We exploit the singularity of the model in order to transform the original state-space system into a pairwise Markov model (PMC) with reduced state dimension. Finally, the a posteriori Markovianity of the reduced state enables us to propose a family of fixed-interval smoothing algorithms

Direct, prediction-based and smoothing-based particle filter algorithms

International audienceWe address the recursive computation of the a posteriori filtering probability density function (pdf) $p_{n|n}$ in a Hidden Markov Chain (HMC) model. We first observe that the classical path $p_{n-1|n-1}$ $\rightarrow$ $p_{n|n-1}$ $\rightarrow$ $p_{n|n}$ is not the only possible one that enables to compute $p_{n|n}$ recursively, and we explore the direct, prediction-based and smoothing-based recursive loops for computing $p_{n|n}$. We next propose a common methodology for computing these equations in practice. Since each path can be decomposed into a Bayesian step and a Markovian step, in the Gaussian case these two elementary operations are implemented by Gaussian transforms, and in the general case by elementary simulation techniques. By proceeding this way we obtain in parallel, for each filtering path, one set of Kalman filter (KF) equations and one generic sequential Monte Carlo (SMC) algorithm. Finally we get four KF (two of which are original), which themselves correspond to four generic SMC algorithms (two of which are original