Massively parallel implicit equal-weights particle filter for ocean drift trajectory forecasting

Forecasting of ocean drift trajectories are important for many applications, including search and rescue operations, oil spill cleanup and iceberg risk mitigation. In an operational setting, forecasts of drift trajectories are produced based on computationally demanding forecasts of three-dimensional ocean currents. Herein, we investigate a complementary approach for shorter time scales by using the recently proposed two-stage implicit equal-weights particle filter applied to a simplified ocean model. To achieve this, we present a new algorithmic design for a data-assimilation system in which all components – including the model, model errors, and particle filter – take advantage of massively parallel compute architectures, such as graphical processing units. Faster computations can enable in-situ and ad-hoc model runs for emergency management, and larger ensembles for better uncertainty quantification. Using a challenging test case with near-realistic chaotic instabilities, we run data-assimilation experiments based on synthetic observations from drifting and moored buoys, and analyze the trajectory forecasts for the drifters. Our results show that even sparse drifter observations are sufficient to significantly improve short-term drift forecasts up to twelve hours. With equidistant moored buoys observing only 0.1% of the state space, the ensemble gives an accurate description of the true state after data assimilation followed by a high-quality probabilistic forecast

Why do the maximum intensities in modeled tropical cyclones vary under the same environmental conditions?

In this study w e explored why the different initial tropical cyclone structures can result in different steady‐state maximum intensities in model simulations with the same environmental conditions. We discovered a linear relationsh ip between the radius of maximum wind (rm) and the absolute angular momentum that passes through rm (Mm) in the model simulated steady‐state tropical cyclones that rm = aMm+b. This nonnegligible intercept b is found to be the key to making a steady‐state storm with a larger Mm more intense. The sensitivity experiments show that this nonzero b results mainly from horizontal turbulent mixing and decreases with decreased horizontal mixing. Using this linear relationship from the simulations, it is also found that the degree of supergradient wind is a function of Mm as well as the turbulent mixing length such that both a larger Mm and/or a reduced turbulent mixing length result in larger supergradient winds

Observation impact in data assimilation: the effect of non-Gaussian observation error.

Data assimilation methods which avoid the assumption of Gaussian error statistics are being developed for geoscience applications. We investigate how the relaxation of the Gaussian assumption affects the impact observations have within the assimilation process. The effect of non-Gaussian observation error (described by the likelihood) is compared to previously published work studying the effect of a non-Gaussian prior. The observation impact is measured in three ways: the sensitivity of the analysis to the observations, the mutual information, and the relative entropy. These three measures have all been studied in the case of Gaussian data assimilation and, in this case, have a known analytical form. It is shown that the analysis sensitivity can also be derived analytically when at least one of the prior or likelihood is Gaussian. This derivation shows an interesting asymmetry in the relationship between analysis sensitivity and analysis error covariance when the two different sources of non-Gaussian structure are considered (likelihood vs. prior). This is illustrated for a simple scalar case and used to infer the effect of the non-Gaussian structure on mutual information and relative entropy, which are more natural choices of metric in non-Gaussian data assimilation. It is concluded that approximating non-Gaussian error distributions as Gaussian can give significantly erroneous estimates of observation impact. The degree of the error depends not only on the nature of the non-Gaussian structure, but also on the metric used to measure the observation impact and the source of the non-Gaussian structure

Time-correlated model error in the (ensemble) Kalman smoother

Data assimilation is often performed in a perfect-model scenario, where only errors in initial conditions and observations are considered. Errors in model equations are increasingly being included, but typically using rather ad-hoc approximations with limited understanding of how these approximations affect the solution and how these approximations interfere with approximations inherent in finite-size ensembles. We provide the first systematic evaluation of the influence of approximations to model errors within a time window of weak-constraint ensemble smoothers. In particular, we study the effects of prescribing temporal correlations in the model errors incorrectly in a Kalman Smoother, and in interaction with finite ensemble-size effects in an Ensemble Kalman Smoother. For the Kalman Smoother we find that an incorrect correlation time scale for additive model errors can have substantial negative effects on the solutions, and we find that overestimating of the correlation time scale leads to worse results than underestimating. In the Ensemble Kalman Smoother case, the resulting ensemble-based space-time gain can be written as the true gain multiplied by two factors, a linear factor containing the errors due to both time-correlation errors and finite ensemble effects, and a non-linear factor related to the inverse part of the gain. Assuming that both errors are relatively small, we are able to disentangle the contributions from the different approximations. The analysis mean is affected by the time-correlation errors, but also substantially by finite ensemble effects, which was unexpected. The analysis covariance is affected by both time-correlation errors and an in-breeding term. This first thorough analysis of the influence of time-correlation errors and finite ensemble size errors on weak-constraint ensemble smoothers will aid further development of these methods and help to make them robust for e.g. numerical weather prediction

Efficient fully nonlinear data assimilation for geophysical fluid dynamics

A potential problem with Ensemble Kalman Filter is the implicit Gaussian assumption at analysis times. Here we explore the performance of a recently proposed fully nonlinear particle filter on a high-dimensional but simplified ocean model, in which the Gaussian assumption is not made. The model simulates the evolution of the vorticity field in time, described by the barotropic vorticity equation, in a highly nonlinear flow regime. While common knowledge is that particle filters are inefficient and need large numbers of model runs to avoid degeneracy, the newly developed particle filter needs only of the order of 10-100 particles on large scale problems. The crucial new ingredient is that the proposal density cannot only be used to ensure all particles end up in high-probability regions of state space as defined by the observations, but also to ensure that most of the particles have similar weights. Using identical twin experiments we found that the ensemble mean follows the truth reliably, and the difference from the truth is captured by the ensemble spread. A rank histogram is used to show that the truth run is indistinguishable from any of the particles, showing statistical consistency of the method

Sequential Monte Carlo with kernel embedded mappings: the mapping particle filter

In this work, a novel sequential Monte Carlo filter is introduced which aims at an efficient sampling of the state space. Particles are pushed forward from the prediction to the posterior density using a sequence of mappings that minimizes the Kullback-Leibler divergence between the posterior and the sequence of intermediate densities. The sequence of mappings represents a gradient flow based on the principles of local optimal transport. A key ingredient of the mappings is that they are embedded in a reproducing kernel Hilbert space, which allows for a practical and efficient Monte Carlo algorithm. The kernel embedding provides a direct means to calculate the gradient of the Kullback-Leibler divergence leading to quick convergence using well-known gradient-based stochastic optimization algorithms. Evaluation of the method is conducted in the chaotic Lorenz-63 system, the Lorenz-96 system, which is a coarse prototype of atmospheric dynamics, and an epidemic model that describes cholera dynamics. No resampling is required in the mapping particle filter even for long recursive sequences. The number of effective particles remains close to the total number of particles in all the sequence. Hence, the mapping particle filter does not suffer from sample impoverishment

Efficient non-linear data assimilation in geophysical fluid dynamics

New ways of combining observations with numerical models are discussed in which the size of the state space can be very large, and the model can be highly nonlinear. Also the observations of the system can be related to the model variables in highly nonlinear ways, making this data-assimilation (or inverse) problem highly nonlinear. First we discuss the connection between data assimilation and inverse problems, including regularization. We explore the choice of proposal density in a Particle Filter and show how the ’curse of dimensionality’ might be beaten. In the standard Particle Filter ensembles of model runs are propagated forward in time until observations are encountered, rendering it a pure Monte-Carlo method. In large-dimensional systems this is very inefficient and very large numbers of model runs are needed to solve the data-assimilation problem realistically. In our approach we steer all model runs towards the observations resulting in a much more efficient method. By further ’ensuring almost equal weight’ we avoid performing model runs that are useless in the end. Results are shown for the 40 and 1000 dimensional Lorenz 1995 model

Kernel embedding of maps for Bayesian inference: the variational mapping particle filter

Data assimilation for high-dimensional highly nonlinear systems is becoming crucial for several geosciences applications. In this work, a novel particle filter is introduced which aims to an efficient sampling of the posterior pdf in high-dimensional state spaces considering a limited number of particles. Particles are mapped from the proposal to the posterior density using the principles of optimal transport. The Kullback-Leibler divergence between the posterior density and the proposal divergence is minimised using variational principles, leading to an iterative gradient-descent like algorithm. A key ingredient of the mapping is that the transformations are embedded in a reproducing kernel Hilbert space which constrains the dimensions of the space for the optimal transport to the number of particles. Gradient information of the Kullback-Leibler divergence allows a quick convergence using well known gradient-based optimization algorithms from machine learning, adadelta and adam, which do not require cost function calculations. Evaluation of the method and comparison with a SIR filter is conducted as a proof-of-concept in the Lorenz-63 system, where the exact solution is known. No resampling is required even for long recursive implementations. The number of effective particles remains close to the total number of particles in all the recursions. Hence, the mapping particle filter does not suffer from sample impoverishment, even in highly nonlinear settings. Finally, results from experiments on a high-dimensional turbulent geophysical system will be presented, and the performance of the new method compared to other existing method will be discussed.Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mar del Plata. Instituto de Investigaciones En Biodiversidad y Biotecnología. Grupo de Investigación en Química Analítica y Modelado Molecular; Argentina. University of Reading; Reino UnidoFil: van Leeuwen, Peter Jan. University of Reading; Reino UnidoEGU General AssemblyViennaAustriaEuropean Geosciences Unio