Particle filtering in high-dimensional chaotic systems

We present an efficient particle filtering algorithm for multiscale systems, that is adapted for simple atmospheric dynamics models which are inherently chaotic. Particle filters represent the posterior conditional distribution of the state variables by a collection of particles, which evolves and adapts recursively as new information becomes available. The difference between the estimated state and the true state of the system constitutes the error in specifying or forecasting the state, which is amplified in chaotic systems that have a number of positive Lyapunov exponents. The purpose of the present paper is to show that the homogenization method developed in Imkeller et al. (2011), which is applicable to high dimensional multi-scale filtering problems, along with important sampling and control methods can be used as a basic and flexible tool for the construction of the proposal density inherent in particle filtering. Finally, we apply the general homogenized particle filtering algorithm developed here to the Lorenz'96 atmospheric model that mimics mid-latitude atmospheric dynamics with microscopic convective processes.Comment: 28 pages, 12 figure

An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems

We propose an ensemble score filter (EnSF) for solving high-dimensional nonlinear filtering problems with superior accuracy. A major drawback of existing filtering methods, e.g., particle filters or ensemble Kalman filters, is the low accuracy in handling high-dimensional and highly nonlinear problems. EnSF attacks this challenge by exploiting the score-based diffusion model, defined in a pseudo-temporal domain, to characterizing the evolution of the filtering density. EnSF stores the information of the recursively updated filtering density function in the score function, in stead of storing the information in a set of finite Monte Carlo samples (used in particle filters and ensemble Kalman filters). Unlike existing diffusion models that train neural networks to approximate the score function, we develop a training-free score estimation that uses mini-batch-based Monte Carlo estimator to directly approximate the score function at any pseudo-spatial-temporal location, which provides sufficient accuracy in solving high-dimensional nonlinear problems as well as saves tremendous amount of time spent on training neural networks. Another essential aspect of EnSF is its analytical update step, gradually incorporating data information into the score function, which is crucial in mitigating the degeneracy issue faced when dealing with very high-dimensional nonlinear filtering problems. High-dimensional Lorenz systems are used to demonstrate the performance of our method. EnSF provides surprisingly impressive performance in reliably tracking extremely high-dimensional Lorenz systems (up to 1,000,000 dimension) with highly nonlinear observation processes, which is a well-known challenging problem for existing filtering methods.Comment: arXiv admin note: text overlap with arXiv:2306.0928

Stability of Filters for the Navier-Stokes Equation

Data assimilation methodologies are designed to incorporate noisy observations of a physical system into an underlying model in order to infer the properties of the state of the system. Filters refer to a class of data assimilation algorithms designed to update the estimation of the state in a on-line fashion, as data is acquired sequentially. For linear problems subject to Gaussian noise filtering can be performed exactly using the Kalman filter. For nonlinear systems it can be approximated in a systematic way by particle filters. However in high dimensions these particle filtering methods can break down. Hence, for the large nonlinear systems arising in applications such as weather forecasting, various ad hoc filters are used, mostly based on making Gaussian approximations. The purpose of this work is to study the properties of these ad hoc filters, working in the context of the 2D incompressible Navier-Stokes equation. By working in this infinite dimensional setting we provide an analysis which is useful for understanding high dimensional filtering, and is robust to mesh-refinement. We describe theoretical results showing that, in the small observational noise limit, the filters can be tuned to accurately track the signal itself (filter stability), provided the system is observed in a sufficiently large low dimensional space; roughly speaking this space should be large enough to contain the unstable modes of the linearized dynamics. Numerical results are given which illustrate the theory. In a simplified scenario we also derive, and study numerically, a stochastic PDE which determines filter stability in the limit of frequent observations, subject to large observational noise. The positive results herein concerning filter stability complement recent numerical studies which demonstrate that the ad hoc filters perform poorly in reproducing statistical variation about the true signal

Subgradient-Based Markov Chain Monte Carlo Particle Methods for Discrete-Time Nonlinear Filtering

This work shows how a carefully designed instrumental distribution can improve the performance of a Markov chain Monte Carlo (MCMC) filter for systems with a high state dimension. We propose a special subgradient-based kernel from which candidate moves are drawn. This facilitates the implementation of the filtering algorithm in high dimensional settings using a remarkably small number of particles. We demonstrate our approach in solving a nonlinear non-Gaussian high-dimensional problem in comparison with a recently developed block particle filter and over a dynamic compressed sensing (l1 constrained) algorithm. The results show high estimation accuracy

Accuracy and stability of filters for dissipative PDEs

Data assimilation methodologies are designed to incorporate noisy observations of a physical system into an underlying model in order to infer the properties of the state of the system. Filters refer to a class of data assimilation algorithms designed to update the estimation of the state in an on-line fashion, as data is acquired sequentially. For linear problems subject to Gaussian noise, filtering can be performed exactly using the Kalman filter. For nonlinear systems filtering can be approximated in a systematic way by particle filters. However in high dimensions these particle filtering methods can break down. Hence, for the large nonlinear systems arising in applications such as oceanography and weather forecasting, various ad hoc filters are used, mostly based on making Gaussian approximations. The purpose of this work is to study the accuracy and stability properties of these ad hoc filters. We work in the context of the 2D incompressible Navier-Stokes equation, although the ideas readily generalize to a range of dissipative partial differential equations (PDEs). By working in this infinite dimensional setting we provide an analysis which is useful for the understanding of high dimensional filtering, and is robust to mesh-refinement. We describe theoretical results showing that, in the small observational noise limit, the filters can be tuned to perform accurately in tracking the signal itself (filter accuracy), provided the system is observed in a sufficiently large low dimensional space; roughly speaking this space should be large enough to contain the unstable modes of the linearized dynamics. The tuning corresponds to what is known as variance inflation in the applied literature. Numerical results are given which illustrate the theory. The positive results herein concerning filter stability complement recent numerical studies which demonstrate that the ad hoc filters can perform poorly in reproducing statistical variation about the true signal

A Two-stage Particle Filter in High Dimension

Particle Filter (PF) is a popular sequential Monte Carlo method to deal with non-linear non-Gaussian filtering problems. However, it suffers from the so-called curse of dimensionality in the sense that the required number of particle (needed for a reasonable performance) grows exponentially with the dimension of the system. One of the techniques found in the literature to tackle this is to split the high-dimensional state in to several lower dimensional (sub)spaces and run a particle filter on each subspace, the so-called multiple particle filter (MPF). It is also well-known from the literature that a good proposal density can help to improve the performance of a particle filter. In this article, we propose a new particle filter consisting of two stages. The first stage derives a suitable proposal density that incorporates the information from the measurements. In the second stage a PF is employed with the proposal density obtained in the first stage. Through a simulated example we show that in high-dimensional systems, the proposed two-stage particle filter performs better than the MPF with much fewer number of particles

Nonlinear filtering of high dimensional, chaotic, multiple timescale correlated systems

This dissertation addresses theoretical and numerical questions in nonlinear filtering theory for high dimensional, chaotic, multiple timescale correlated systems. The research is motivated by problems in the geosciences, in particular oceanic or atmospheric estimation and climate prediction. As the capability and need to further resolve the physics models on finer scales continues, greater spatial and temporal scales become present and the dimension of the models becomes increasingly large. In the atmospheric sciences, these models can be of the order $\mathcal{O}(10^9)$ degrees of freedom and require assimilation of the order $\mathcal{O}(10^7)$ observations during a single day. The models are chaotic and the observing sensors may be correlated with the physical processes themselves. The goal of the dissertation is to develop theoretical results that can provide the mathematical justification for new filtering algorithms on a lower dimensional problem, and to develop novel methods for dealing with issues that plague particle filtering when applied to high dimensional, chaotic, multiple timescale correlated systems. The first half of the dissertation is theoretical and addresses the question of approximating the continuous time nonlinear filtering equation for a multiple timescale correlated system by an averaged filtering equation in the limit of large timescale separation. The first result in this direction is within the context of a slow-fast system with correlation between the slow process and the observation process, and when we are only interested in estimating functions of the slow process. The main result is that we can retrieve a rate of convergence and that there is a metric generating the topology of weak convergence, such that the marginal filter converges to the averaged filter at the given rate in the limit of large timescale separation. The proof uses a probabilistic representation (backward doubly stochastic differential equation) of the dual process to the unnormalized filter, and sharp estimates on the transition density and semigroup of the fast process. The second theoretical result of the dissertation addresses the same question for a broader problem, where the slow signal dynamics include an intermediate timescale forcing. We prove that the marginal filter converges in probability to the average filter for a metric that generates the topology of weak convergence. The method of proof is by showing tightness of the measure-valued process, characterizing the weak limits, and proving the limit is unique. The perturbation test function (also known as method of corrector) is used to deal with the intermediate timescale forcing term, where the corrector is the solution of a Poisson equation. The second half of the dissertation develops filtering algorithms that leverage the theoretical results from the first half of the thesis to produce particle filtering methods for the averaged filtering equation. We also develop particle methods that address the issue of particle collapse for filtering on general high dimensional chaotic systems. Using the two timescale Lorenz 1996 atmospheric model, we show that the reduced order particle filtering methods are shown to be at least an order of magnitude faster than standard particle methods. We develop a method for particle filtering when the signal and observation processes are correlated. We also develop extensions to controlled optimal proposal particle filters that improve the diversity of the particle ensemble when tested on the Lorenz 1963 model. In the last chapter of the dissertation, we adopt a dynamical systems viewpoint to address the issue of particle collapse. This time the goal is to exploit the chaotic properties of the system being filtered to perform assimilation in a lower dimensional subspace. A new approach is developed which enables data assimilation in the unstable subspace for particle filtering. We introduce the idea of future right-singular vectors to produce projection operators, enabling assimilation in a lower dimensional subspace. We show that particle filtering algorithms using dynamically generator projection operators, in particular the future right-singular vectors, outperforms standard particle methods in terms of root-mean-square-error, diversity of the particle ensemble, and robustness when applied to the single timescale Lorenz 1996 model

Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data

While nonlinear stochastic partial differential equations arise naturally in spatiotemporal modeling, inference for such systems often faces two major challenges: sparse noisy data and ill-posedness of the inverse problem of parameter estimation. To overcome the challenges, we introduce a strongly regularized posterior by normalizing the likelihood and by imposing physical constraints through priors of the parameters and states. We investigate joint parameter-state estimation by the regularized posterior in a physically motivated nonlinear stochastic energy balance model (SEBM) for paleoclimate reconstruction. The high-dimensional posterior is sampled by a particle Gibbs sampler that combines MCMC with an optimal particle filter exploiting the structure of the SEBM. In tests using either Gaussian or uniform priors based on the physical range of parameters, the regularized posteriors overcome the ill-posedness and lead to samples within physical ranges, quantifying the uncertainty in estimation. Due to the ill-posedness and the regularization, the posterior of parameters presents a relatively large uncertainty, and consequently, the maximum of the posterior, which is the minimizer in a variational approach, can have a large variation. In contrast, the posterior of states generally concentrates near the truth, substantially filtering out observation noise and reducing uncertainty in the unconstrained SEBM

Langevin and Hamiltonian based Sequential MCMC for Efficient Bayesian Filtering in High-dimensional Spaces

Nonlinear non-Gaussian state-space models arise in numerous applications in statistics and signal processing. In this context, one of the most successful and popular approximation techniques is the Sequential Monte Carlo (SMC) algorithm, also known as particle filtering. Nevertheless, this method tends to be inefficient when applied to high dimensional problems. In this paper, we focus on another class of sequential inference methods, namely the Sequential Markov Chain Monte Carlo (SMCMC) techniques, which represent a promising alternative to SMC methods. After providing a unifying framework for the class of SMCMC approaches, we propose novel efficient strategies based on the principle of Langevin diffusion and Hamiltonian dynamics in order to cope with the increasing number of high-dimensional applications. Simulation results show that the proposed algorithms achieve significantly better performance compared to existing algorithms