158 research outputs found
Predictive verification for the design of partially exchangeable multi-model ensembles
This is the final version. Available on open access from Taylor & Francis via the DOI in this recordThe performance of an ensemble forecast, as measured by scoring rules, depends on its number of members. Under the assumption of ensemble member exchangeability, ensemble-adjusted scores provide unbiased estimates of the ensemble-size effect. In this study, the concept of ensemble-adjusted scores is revisited and exploited in the general context of multi-model ensemble forecasting. In particular, an ensemblesize adjustment is proposed for the continuous ranked probability score in a multi-model ensemble setting. The method requires that the ensemble forecasts satisfy generalized multi-model exchangeability
conditions. These conditions do not require the models themselves to be exchangeable.
The adjusted scores are tested here on a dual-resolution ensemble, an ensemble which combines
members drawn from the same numerical model but run at two different grid resolutions. It is shown
that performance of different ensemble combinations can be robustly estimated based on a small subset
of members from each model. At no additional cost, the ensemble-size effect is investigated not only
considering the pooling of potential extra-members but also including the impact of optimal weighting
strategies. With simple and efficient tools, the proposed methodology paves the way for predictive
verification of multi-model ensemble forecasts; the derived statistics can provide guidance for the
design of future operational ensemble configurations without having to run additional ensemble forecast
experiments for all the potential configurations
The effect of stochastically perturbed parametrisation tendencies (SPPT) on rapidly ascending air streams
Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model
Stochastic parameterizations account for uncertainty in the representation of
unresolved sub-grid processes by sampling from the distribution of possible
sub-grid forcings. Some existing stochastic parameterizations utilize
data-driven approaches to characterize uncertainty, but these approaches
require significant structural assumptions that can limit their scalability.
Machine learning models, including neural networks, are able to represent a
wide range of distributions and build optimized mappings between a large number
of inputs and sub-grid forcings. Recent research on machine learning
parameterizations has focused only on deterministic parameterizations. In this
study, we develop a stochastic parameterization using the generative
adversarial network (GAN) machine learning framework. The GAN stochastic
parameterization is trained and evaluated on output from the Lorenz '96 model,
which is a common baseline model for evaluating both parameterization and data
assimilation techniques. We evaluate different ways of characterizing the input
noise for the model and perform model runs with the GAN parameterization at
weather and climate timescales. Some of the GAN configurations perform better
than a baseline bespoke parameterization at both timescales, and the networks
closely reproduce the spatio-temporal correlations and regimes of the Lorenz
'96 system. We also find that in general those models which produce skillful
forecasts are also associated with the best climate simulations.Comment: Submitted to Journal of Advances in Modeling Earth Systems (JAMES
Recommended from our members
Impact of the mesoscale range on error growth and the limits to atmospheric predictability
Global numerical weather prediction (NWP) models have begun to resolve the mesoscale k^{-5/3} range of the energy spectrum, which is known to impose an inherently finite range of deterministic predictability per se as errors develop more rapidly on these scales than on the larger scales. However, the dynamics of these errors under the influence of the synoptic-scale k^{-3} range is little studied. Within a perfect-model context, the present work examines the error growth behavior under such a hybrid spectrum in Lorenz’s original model of 1969, and in a series of identical-twin perturbation experiments using an idealized two-dimensional barotropic turbulence model at a range of resolutions. With the typical resolution of today’s global NWP ensembles, error growth remains largely uniform across scales. The theoretically expected fast error growth characteristic of a k^{-5/3} spectrum is seen to be largely suppressed in the first decade of the mesoscale range by the synoptic-scale k^{-3} range. However, it emerges once models become fully able to resolve features on something like a 20-kilometer scale, which corresponds to a grid resolution on the order of a few kilometers
Recommended from our members
Working group 3: What are and how do we measure the pros and cons of existing approaches?
WG3 discussed both the pros and cons of existing schemes as well as metrics to measure relative advantages and disadvantages. We first provide a list of the current operational techniques and their respective advantages and disadvantages that were discussed in the WG. We do not claim
that the list is complete, and we note that the pros and cons are neither exhaustive nor quantitative. Nevertheless, it may be useful to note the WG’s consensus on the general
advantages and disadvantages of the most commonly-used schemes. We then list our recommendations for evaluating model uncertainty schemes. At the end is a short list pertaining to recommendations for further development of methods to represent model uncertainty
Recommended from our members
Resonant gravity-wave drag enhancement in linear stratified flow over mountains
High-drag states produced in stratified flow over a 2D ridge and an axisymmetric mountain are investigated using a linear, hydrostatic, analytical model. A wind profile is assumed where the background velocity is constant up to a height z1 and then decreases linearly, and the internal gravity-wave solutions are calculated exactly. In flow over a 2D ridge, the normalized surface drag is given by a closed-form analytical expression, while in flow over an axisymmetric mountain it is given by an expression involving a simple 1D integral. The drag is found to depend on two dimensionless parameters: a dimensionless height formed with z_1, and the Richardson number, Ri, in the shear layer. The drag oscillates as z_1 increases, with a period of half the hydrostatic vertical wavelength of the gravity waves. The amplitude of this modulation increases as Ri decreases. This behaviour is due to wave reflection at z_1. Drag maxima correspond to constructive interference of the upward- and downward-propagating waves in the region z < z_1, while drag minima correspond to destructive interference. The reflection coefficient at the interface z = z_1 increases as Ri decreases. The critical level, z_c, plays no role in the drag amplification. A preliminary numerical treatment of nonlinear effects is presented, where z_c appears to become more relevant, and flow over a 2D ridge qualitatively changes its character. But these effects, and their connection with linear theory, still need to be better understood
Evaluating Data Assimilation Algorithms
Data assimilation leads naturally to a Bayesian formulation in which the
posterior probability distribution of the system state, given the observations,
plays a central conceptual role. The aim of this paper is to use this Bayesian
posterior probability distribution as a gold standard against which to evaluate
various commonly used data assimilation algorithms.
A key aspect of geophysical data assimilation is the high dimensionality and
low predictability of the computational model. With this in mind, yet with the
goal of allowing an explicit and accurate computation of the posterior
distribution, we study the 2D Navier-Stokes equations in a periodic geometry.
We compute the posterior probability distribution by state-of-the-art
statistical sampling techniques. The commonly used algorithms that we evaluate
against this accurate gold standard, as quantified by comparing the relative
error in reproducing its moments, are 4DVAR and a variety of sequential
filtering approximations based on 3DVAR and on extended and ensemble Kalman
filters.
The primary conclusions are that: (i) with appropriate parameter choices,
approximate filters can perform well in reproducing the mean of the desired
probability distribution; (ii) however they typically perform poorly when
attempting to reproduce the covariance; (iii) this poor performance is
compounded by the need to modify the covariance, in order to induce stability.
Thus, whilst filters can be a useful tool in predicting mean behavior, they
should be viewed with caution as predictors of uncertainty. These conclusions
are intrinsic to the algorithms and will not change if the model complexity is
increased, for example by employing a smaller viscosity, or by using a detailed
NWP model
Recommended from our members
Ensemble downscaling in coupled solar wind-magnetosphere modeling for space weather forecasting
Advanced forecasting of space weather requires simulation of the whole Sun-to-Earth system, which necessitates driving magnetospheric models with the outputs from solar wind models. This presents a fundamental difficulty, as the magnetosphere is sensitive to both large-scale solar wind structures, which can be captured by solar wind models, and small-scale solar wind “noise,” which is far below typical solar wind model resolution and results primarily from stochastic processes. Following similar approaches in terrestrial climate modeling, we propose statistical “downscaling” of solar wind model results prior to their use as input to a magnetospheric model. As magnetospheric response can be highly nonlinear, this is
preferable to downscaling the results of magnetospheric modeling. To demonstrate the benefit of this approach, we first approximate solar wind model output by smoothing solar wind observations with an 8 h filter, then add small-scale structure back in through the addition of random noise with the observed spectral characteristics. Here we use a very simple parameterization of noise based upon the observed probability distribution functions of solar wind parameters, but more sophisticated methods will be developed in the future. An ensemble of results from the simple downscaling scheme are tested using a
model-independent method and shown to add value to the magnetospheric forecast, both improving the best estimate and quantifying the uncertainty. We suggest a number of features desirable in an operational solar wind downscaling scheme
Recommended from our members
Probabilistic solar wind and geomagnetic forecasting using an analogue ensemble or "Similar Day" approach
Effective space-weather prediction and mitigation requires accurate forecasting of near-Earth solar-wind conditions. Numerical magnetohydrodynamic models of the solar wind, driven by remote solar observations, are gaining skill at forecasting the large-scale solar-wind features that give rise to near-Earth variations over days and weeks. There remains a need for accurate short-term (hours to days) solar-wind forecasts, however. In this study we investigate the analogue ensemble (AnEn), or “similar day”, approach that was developed for atmospheric weather forecasting. The central premise of the AnEn is that past variations that are analogous or similar to current conditions can be used to provide a good estimate of future variations. By considering an ensemble of past analogues, the AnEn forecast is inherently probabilistic and provides a measure of the forecast uncertainty. We show that forecasts of solar-wind speed can be improved by considering both speed and density when determining past analogues, whereas forecasts of the out-of-ecliptic magnetic field [ BNBN ] are improved by also considering the in-ecliptic magnetic-field components. In general, the best forecasts are found by considering only the previous 6 – 12 hours of observations. Using these parameters, the AnEn provides a valuable probabilistic forecast for solar-wind speed, density, and in-ecliptic magnetic field over lead times from a few hours to around four days. For BNBN , which is central to space-weather disturbance, the AnEn only provides a valuable forecast out to around six to seven hours. As the inherent predictability of this parameter is low, this is still likely a marked improvement over other forecast methods. We also investigate the use of the AnEn in forecasting geomagnetic indices Dst and Kp. The AnEn provides a valuable probabilistic forecast of both indices out to around four days. We outline a number of future improvements to AnEn forecasts of near-Earth solar-wind and geomagnetic conditions
- …