Particle filtering in high-dimensional chaotic systems

We present an efficient particle filtering algorithm for multiscale systems, that is adapted for simple atmospheric dynamics models which are inherently chaotic. Particle filters represent the posterior conditional distribution of the state variables by a collection of particles, which evolves and adapts recursively as new information becomes available. The difference between the estimated state and the true state of the system constitutes the error in specifying or forecasting the state, which is amplified in chaotic systems that have a number of positive Lyapunov exponents. The purpose of the present paper is to show that the homogenization method developed in Imkeller et al. (2011), which is applicable to high dimensional multi-scale filtering problems, along with important sampling and control methods can be used as a basic and flexible tool for the construction of the proposal density inherent in particle filtering. Finally, we apply the general homogenized particle filtering algorithm developed here to the Lorenz'96 atmospheric model that mimics mid-latitude atmospheric dynamics with microscopic convective processes.Comment: 28 pages, 12 figure

On sequential multiscale inversion and data assimilation

Multiscale approaches are very popular for example for solving partial differential equations and in many applied fields dealing with phenomena which take place on different levels of detail. The broad idea of a multiscale approach is to decompose your problem into different scales or levels and to use these decompositions either for constructing appropriate approximations or to solve smaller problems on each of these levels, leading to increased stability or increased efficiency. The idea of sequential multiscale is to first solve the problem in a large-scale subspace and then successively move to finer scale spaces. Our goal is to analyse the sequential multiscale approach applied to an inversion or state estimation problem. We work in a generic setup given by a Hilbert space environment. We work out the analysis both for an unregularized and a regularized sequential multiscale inversion. In general the sequential multiscale approach is not equivalent to a full solution, but we show that under appropriate assumptions we obtain convergence of an iterative sequential multiscale version of the method. For the regularized case we develop a strategy to appropriately adapt the regularization when an iterative approach is taken. We demonstrate the validity of the iterative sequential multiscale approach by testing the method on an integral equation as it appears for atmospheric temperature retrieval from infrared satellite radiances

Data assimilation in slow-fast systems using homogenized climate models

A deterministic multiscale toy model is studied in which a chaotic fast subsystem triggers rare transitions between slow regimes, akin to weather or climate regimes. Using homogenization techniques, a reduced stochastic parametrization model is derived for the slow dynamics. The reliability of this reduced climate model in reproducing the statistics of the slow dynamics of the full deterministic model for finite values of the time scale separation is numerically established. The statistics however is sensitive to uncertainties in the parameters of the stochastic model. It is investigated whether the stochastic climate model can be beneficial as a forecast model in an ensemble data assimilation setting, in particular in the realistic setting when observations are only available for the slow variables. The main result is that reduced stochastic models can indeed improve the analysis skill, when used as forecast models instead of the perfect full deterministic model. The stochastic climate model is far superior at detecting transitions between regimes. The observation intervals for which skill improvement can be obtained are related to the characteristic time scales involved. The reason why stochastic climate models are capable of producing superior skill in an ensemble setting is due to the finite ensemble size; ensembles obtained from the perfect deterministic forecast model lacks sufficient spread even for moderate ensemble sizes. Stochastic climate models provide a natural way to provide sufficient ensemble spread to detect transitions between regimes. This is corroborated with numerical simulations. The conclusion is that stochastic parametrizations are attractive for data assimilation despite their sensitivity to uncertainties in the parameters.Comment: Accepted for publication in Journal of the Atmospheric Science

Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations

Climate projections continue to be marred by large uncertainties, which originate in processes that need to be parameterized, such as clouds, convection, and ecosystems. But rapid progress is now within reach. New computational tools and methods from data assimilation and machine learning make it possible to integrate global observations and local high-resolution simulations in an Earth system model (ESM) that systematically learns from both. Here we propose a blueprint for such an ESM. We outline how parameterization schemes can learn from global observations and targeted high-resolution simulations, for example, of clouds and convection, through matching low-order statistics between ESMs, observations, and high-resolution simulations. We illustrate learning algorithms for ESMs with a simple dynamical system that shares characteristics of the climate system; and we discuss the opportunities the proposed framework presents and the challenges that remain to realize it.Comment: 32 pages, 3 figure

Multiscale assimilation of Advanced Microwave Scanning Radiometer-EOS snow water equivalent and Moderate Resolution Imaging Spectroradiometer snow cover fraction observations in northern Colorado

Eight years (2002–2010) of Advanced Microwave Scanning Radiometer–EOS (AMSR-E) snow water equivalent (SWE) retrievals and Moderate Resolution Imaging Spectroradiometer (MODIS) snow cover fraction (SCF) observations are assimilated separately or jointly into the Noah land surface model over a domain in Northern Colorado. A multiscale ensemble Kalman filter (EnKF) is used, supplemented with a rule-based update. The satellite data are either left unscaled or are scaled for anomaly assimilation. The results are validated against in situ observations at 14 high-elevation Snowpack Telemetry (SNOTEL) sites with typically deep snow and at 4 lower-elevation Cooperative Observer Program (COOP) sites. Assimilation of coarse-scale AMSR-E SWE and fine-scale MODIS SCF observations both result in realistic spatial SWE patterns. At COOP sites with shallow snowpacks, AMSR-E SWE and MODIS SCF data assimilation are beneficial separately, and joint SWE and SCF assimilation yields significantly improved root-mean-square error and correlation values for scaled and unscaled data assimilation. In areas of deep snow where the SNOTEL sites are located, however, AMSR-E retrievals are typically biased low and assimilation without prior scaling leads to degraded SWE estimates. Anomaly SWE assimilation could not improve the interannual SWE variations in the assimilation results because the AMSR-E retrievals lack realistic interannual variability in deep snowpacks. SCF assimilation has only a marginal impact at the SNOTEL locations because these sites experience extended periods of near-complete snow cover. Across all sites, SCF assimilation improves the timing of the onset of the snow season but without a net improvement of SWE amounts

On finite-size Lyapunov exponents in multiscale systems

We study the effect of regime switches on finite size Lyapunov exponents (FSLEs) in determining the error growth rates and predictability of multiscale systems. We consider a dynamical system involving slow and fast regimes and switches between them. The surprising result is that due to the presence of regimes the error growth rate can be a non-monotonic function of initial error amplitude. In particular, troughs in the large scales of FSLE spectra is shown to be a signature of slow regimes, whereas fast regimes are shown to cause large peaks in the spectra where error growth rates far exceed those estimated from the maximal Lyapunov exponent. We present analytical results explaining these signatures and corroborate them with numerical simulations. We show further that these peaks disappear in stochastic parametrizations of the fast chaotic processes, and the associated FSLE spectra reveal that large scale predictability properties of the full deterministic model are well approximated whereas small scale features are not properly resolved.Comment: Accepted for publication in Chao

Linear theory for filtering nonlinear multiscale systems with model error

We study filtering of multiscale dynamical systems with model error arising from unresolved smaller scale processes. The analysis assumes continuous-time noisy observations of all components of the slow variables alone. For a linear model with Gaussian noise, we prove existence of a unique choice of parameters in a linear reduced model for the slow variables. The linear theory extends to to a non-Gaussian, nonlinear test problem, where we assume we know the optimal stochastic parameterization and the correct observation model. We show that when the parameterization is inappropriate, parameters chosen for good filter performance may give poor equilibrium statistical estimates and vice versa. Given the correct parameterization, it is imperative to estimate the parameters simultaneously and to account for the nonlinear feedback of the stochastic parameters into the reduced filter estimates. In numerical experiments on the two-layer Lorenz-96 model, we find that parameters estimated online, as part of a filtering procedure, produce accurate filtering and equilibrium statistical prediction. In contrast, a linear regression based offline method, which fits the parameters to a given training data set independently from the filter, yields filter estimates which are worse than the observations or even divergent when the slow variables are not fully observed