Data Assimilation Fundamentals

This open-access textbook's significant contribution is the unified derivation of data-assimilation techniques from a common fundamental and optimal starting point, namely Bayes' theorem. Unique for this book is the "top-down" derivation of the assimilation methods. It starts from Bayes theorem and gradually introduces the assumptions and approximations needed to arrive at today's popular data-assimilation methods. This strategy is the opposite of most textbooks and reviews on data assimilation that typically take a bottom-up approach to derive a particular assimilation method. E.g., the derivation of the Kalman Filter from control theory and the derivation of the ensemble Kalman Filter as a low-rank approximation of the standard Kalman Filter. The bottom-up approach derives the assimilation methods from different mathematical principles, making it difficult to compare them. Thus, it is unclear which assumptions are made to derive an assimilation method and sometimes even which problem it aspires to solve. The book's top-down approach allows categorizing data-assimilation methods based on the approximations used. This approach enables the user to choose the most suitable method for a particular problem or application. Have you ever wondered about the difference between the ensemble 4DVar and the "ensemble randomized likelihood" (EnRML) methods? Do you know the differences between the ensemble smoother and the ensemble-Kalman smoother? Would you like to understand how a particle flow is related to a particle filter? In this book, we will provide clear answers to several such questions. The book provides the basis for an advanced course in data assimilation. It focuses on the unified derivation of the methods and illustrates their properties on multiple examples. It is suitable for graduate students, post-docs, scientists, and practitioners working in data assimilation

Data Assimilation Fundamentals

This open-access textbook's significant contribution is the unified derivation of data-assimilation techniques from a common fundamental and optimal starting point, namely Bayes' theorem. Unique for this book is the "top-down" derivation of the assimilation methods. It starts from Bayes theorem and gradually introduces the assumptions and approximations needed to arrive at today's popular data-assimilation methods. This strategy is the opposite of most textbooks and reviews on data assimilation that typically take a bottom-up approach to derive a particular assimilation method. E.g., the derivation of the Kalman Filter from control theory and the derivation of the ensemble Kalman Filter as a low-rank approximation of the standard Kalman Filter. The bottom-up approach derives the assimilation methods from different mathematical principles, making it difficult to compare them. Thus, it is unclear which assumptions are made to derive an assimilation method and sometimes even which problem it aspires to solve. The book's top-down approach allows categorizing data-assimilation methods based on the approximations used. This approach enables the user to choose the most suitable method for a particular problem or application. Have you ever wondered about the difference between the ensemble 4DVar and the "ensemble randomized likelihood" (EnRML) methods? Do you know the differences between the ensemble smoother and the ensemble-Kalman smoother? Would you like to understand how a particle flow is related to a particle filter? In this book, we will provide clear answers to several such questions. The book provides the basis for an advanced course in data assimilation. It focuses on the unified derivation of the methods and illustrates their properties on multiple examples. It is suitable for graduate students, post-docs, scientists, and practitioners working in data assimilation

Optimizing the use of InSAR observations in data assimilation problems to estimate reservoir compaction

Hydrocarbon production may cause subsidence as a result of the pressure reduction in the gas-producing layer and reservoir compaction. To analyze the process of subsidence and estimate reservoir parameters, we use a particle method to assimilate Interferometric synthetic-aperture radar (InSAR) observations of surface deformation with a conceptual model of reservoir. As example, we use an analytical model of the Groningen gas reservoir based on a geometry representing the compartmentalized structure of the subsurface at the reservoir depth. The efficacy of the particle method becomes less when the degree of freedom is large compared to the ensemble size. This degree of freedom, in turn, varies because of spatial correlation in the observed field. The resolution of the InSAR data and the number of observations affect the performance of the particle method. In this study, we quantify the information in a Sentinel-1 SAR dataset using the concept of Shannon entropy from information theory. We investigate how to best capture the level of detail in model resolved by the InSAR data while maximizing their information content for a data assimilation use. We show that incorrect representation of the existing correlations leads to weight collapse when the number of observation increases, unless the ensemble size growths. However, simulations of mutual information show that we could optimize data reduction by choosing an adequate mesh given the spatial correlation in the observed subsidence. Our approach provides a means to achieve a better information use from available InSAR data reducing weight collapse without additional computational cost

An international initiative of predicting the Sars-Cov-2 pandemic using ensemble data assimilation

This work demonstrates the efficiency of using iterative ensemble smoothers to estimate the parameters of an SEIR model. We have extended a standard SEIR model with age-classes and compartments of sick, hospitalized, and dead. The data conditioned on are the daily numbers of accumulated deaths and the number of hospitalized. Also, it is possible to condition the model on the number of cases obtained from testing. We start from a wide prior distribution for the model parameters; then, the ensemble conditioning leads to a posterior ensemble of estimated parameters yielding model predictions in close agreement with the observations. The updated ensemble of model simulations has predictive capabilities and include uncertainty estimates. In particular, we estimate the effective reproductive number as a function of time, and we can assess the impact of different intervention measures. By starting from the updated set of model parameters, we can make accurate short-term predictions of the epidemic development assuming knowledge of the future effective reproductive number. Also, the model system allows for the computation of long-term scenarios of the epidemic under different assumptions. We have applied the model system on data sets from several countries, i.e., the four European countries Norway, England, The Netherlands, and France; the province of Quebec in Canada; the South American countries Argentina and Brazil; and the four US states Alabama, North Carolina, California, and New York. These countries and states all have vastly different developments of the epidemic, and we could accurately model the SARS-CoV-2 outbreak in all of them. We realize that more complex models, e.g., with regional compartments, may be desirable, and we suggest that the approach used here should be applicable also for these models

Iterative Ensemble Smoothers for Data Assimilation in Coupled Nonlinear Multiscale Models

This paper identifies and explains particular differences and properties of adjoint-free iterative ensemble methods initially developed for parameter estimation in petroleum models. The aim is to demonstrate the methods’ potential for sequential data assimilation in coupled and multiscale unstable dynamical systems. For this study, we have introduced a new nonlinear and coupled multiscale model based on two Kuramoto–Sivashinsky equations operating on different scales where a coupling term relaxes the two model variables toward each other. This model provides a convenient testbed for studying data assimilation in highly nonlinear and coupled multiscale systems. We show that the model coupling leads to cross covariance between the two models’ variables, allowing for a combined update of both models. The measurements of one model’s variable will also influence the other and contribute to a more consistent estimate. Second, the new model allows us to examine the properties of iterative ensemble smoothers and assimilation updates over finite-length assimilation windows. We discuss the impact of varying the assimilation windows’ length relative to the model’s predictability time scale. Furthermore, we show that iterative ensemble smoothers significantly improve the solution’s accuracy compared to the standard ensemble Kalman filter update. Results and discussion provide an enhanced understanding of the ensemble methods’ potential implementation and use in operational weather- and climate-prediction systems.publishedVersio

Optimizing the use of InSAR observations in data assimilation problems to estimate reservoir compaction

Hydrocarbon production may cause subsidence as a result of the pressure reduction in the gas-producing layer and reservoir compaction. To analyze the process of subsidence and estimate reservoir parameters, we use a particle method to assimilate Interferometric synthetic-aperture radar (InSAR) observations of surface deformation with a conceptual model of reservoir. As example, we use an analytical model of the Groningen gas reservoir based on a geometry representing the compartmentalized structure of the subsurface at the reservoir depth. The efficacy of the particle method becomes less when the degree of freedom is large compared to the ensemble size. This degree of freedom, in turn, varies because of spatial correlation in the observed field. The resolution of the InSAR data and the number of observations affect the performance of the particle method. In this study, we quantify the information in a Sentinel-1 SAR dataset using the concept of Shannon entropy from information theory. We investigate how to best capture the level of detail in model resolved by the InSAR data while maximizing their information content for a data assimilation use. We show that incorrect representation of the existing correlations leads to weight collapse when the number of observation increases, unless the ensemble size growths. However, simulations of mutual information show that we could optimize data reduction by choosing an adequate mesh given the spatial correlation in the observed subsidence. Our approach provides a means to achieve a better information use from available InSAR data reducing weight collapse without additional computational cost

Parameter estimation using a particle method: inferring mixing coefficients from sea level observations

This paper presents a first attempt to estimate mixing parameters from sea level observations using a particle method based on importance sampling. The method is applied to an ensemble of 128 members of model simulations with a global ocean general circulation model of high complexity. Idealized twin experiments demonstrate that the method is able to accurately reconstruct mixing parameters from an observed mean sea level field when mixing is assumed to be spatially homogeneous. An experiment with inhomogeneous eddy coefficients fails because of the limited ensemble size. This is overcome by the introduction of local weighting, which is able to capture spatial variations in mixing qualitatively. As the sensitivity of sea level for variations in mixing is higher for low values of mixing coefficients, the method works relatively well in regions of low eddy activity

Balanced Ocean-Data Assimilation near the Equator

The question is addressed whether using unbalanced updates in ocean-data assimilation schemes for seasonal forecasting systems can result in a relatively poor simulation of zonal currents. An assimilation scheme, where temperature observations are used for updating only the density field, is compared to a scheme where updates of density field and zonal velocities are related by geostrophic balance. This is done for an equatorial linear shallow-water model. It is found that equatorial zonal velocities can be detoriated if velocity is not updated in the assimilation procedure. Adding balanced updates to the zonal velocity is shown to be a simple remedy for the shallow-water model. Next, optimal interpolation (OI) schemes with balanced updates of the zonal velocity are implemented in two ocean general circulation models. First tests indicate a beneficial impact on equatorial upper-ocean zonal currents