13 research outputs found
Recommended from our members
Stratified rank histograms for ensemble forecast verification under serial dependence
Rank histograms are a popular way to assess the reliability of ensemble forecasting systems. If the ensemble forecasting system is reliable, the rank histogram should be flat, ``up to statistical fluctuations''. There are two long noted challenges to this approach. Firstly, uniformity of the overall distribution is implied by but does not imply reliability; ideally the distribution of the ranks should be uniform even conditionally on different forecast scenarios. Secondly, the ranks are serially dependent in general, precluding the use of standard goodness--of--fit tests to assess the uniformity of rank distributions without any further precautions. The present paper deals with both these issues by drawing together the concept of stratified rank histograms, which have been developed to deal with the first issue, with ideas that exploit the reliability condition to manage the serial correlations, thus dealing with the second issue. As a result, tests for uniformity of stratified rank histograms are presented that are valid under serial correlations
Statistical post-processing of heat index ensemble forecasts: is there a royal road?
We investigate the effect of statistical post-processing on the probabilistic
skill of discomfort index (DI) and indoor wet-bulb globe temperature (WBGTid)
ensemble forecasts, both calculated from the corresponding forecasts of
temperature and dew point temperature. Two different methodological approaches
to calibration are compared. In the first case, we start with joint
post-processing of the temperature and dew point forecasts and then create
calibrated samples of DI and WBGTid using samples from the obtained bivariate
predictive distributions. This approach is compared with direct post-processing
of the heat index ensemble forecasts. For this purpose, a novel ensemble model
output statistics model based on a generalized extreme value distribution is
proposed. The predictive performance of both methods is tested on the
operational temperature and dew point ensemble forecasts of the European Centre
for Medium-Range Weather Forecasts and the corresponding forecasts of DI and
WBGTid. For short lead times (up to day 6), both approaches significantly
improve the forecast skill. Among the competing post-processing methods, direct
calibration of heat indices exhibits the best predictive performance, very
closely followed by the more general approach based on joint calibration of
temperature and dew point temperature. Additionally, a machine learning
approach is tested and shows comparable performance for the case when one is
interested only in forecasting heat index warning level categories.Comment: 29 pages, 12 figure
Generation of scenarios from calibrated ensemble forecasts with a dual ensemble copula coupling approach
Probabilistic forecasts in the form of ensemble of scenarios are required for
complex decision making processes. Ensemble forecasting systems provide such
products but the spatio-temporal structures of the forecast uncertainty is lost
when statistical calibration of the ensemble forecasts is applied for each lead
time and location independently. Non-parametric approaches allow the
reconstruction of spatio-temporal joint probability distributions at a low
computational cost. For example, the ensemble copula coupling (ECC) method
rebuilds the multivariate aspect of the forecast from the original ensemble
forecasts. Based on the assumption of error stationarity, parametric methods
aim to fully describe the forecast dependence structures. In this study, the
concept of ECC is combined with past data statistics in order to account for
the autocorrelation of the forecast error. The new approach, called d-ECC, is
applied to wind forecasts from the high resolution ensemble system COSMO-DE-EPS
run operationally at the German weather service. Scenarios generated by ECC and
d-ECC are compared and assessed in the form of time series by means of
multivariate verification tools and in a product oriented framework.
Verification results over a 3 month period show that the innovative method
d-ECC outperforms or performs as well as ECC in all investigated aspects
Quantile forecast discrimination ability and value
While probabilistic forecast verification for categorical forecasts is well
established, some of the existing concepts and methods have not found their
equivalent for the case of continuous variables. New tools dedicated to the
assessment of forecast discrimination ability and forecast value are introduced
here, based on quantile forecasts being the base product for the continuous
case (hence in a nonparametric framework). The relative user characteristic
(RUC) curve and the quantile value plot allow analysing the performance of a
forecast for a specific user in a decision-making framework. The RUC curve is
designed as a user-based discrimination tool and the quantile value plot
translates forecast discrimination ability in terms of economic value. The
relationship between the overall value of a quantile forecast and the
respective quantile skill score is also discussed. The application of these new
verification approaches and tools is illustrated based on synthetic datasets,
as well as for the case of global radiation forecasts from the high resolution
ensemble COSMO-DE-EPS of the German Weather Service
Statistical postprocessing of ensemble global radiation forecasts with penalized quantile regression
Enhancing COSMO-DE ensemble forecasts by inexpensive techniques
COSMO-DE-EPS, a convection-permitting ensemble prediction system based on the high-resolution numerical weather prediction model COSMO-DE, is pre-operational since December 2010, providing probabilistic forecasts which cover Germany. This ensemble system comprises 20 members based on variations of the lateral boundary conditions, the physics parameterizations and the initial conditions. In order to increase the sample size in a computationally inexpensive way, COSMO-DE-EPS is combined with alternative ensemble techniques: the neighborhood method and the time-lagged approach. Their impact on the quality of the resulting probabilistic forecasts is assessed. Objective verification is performed over a six months period, scores based on the Brier score and its decomposition are shown for June 2011. The combination of the ensemble system with the alternative approaches improves probabilistic forecasts of precipitation in particular for high precipitation thresholds. Moreover, combining COSMO-DE-EPS with only the time-lagged approach improves the skill of area probabilities for precipitation and does not deteriorate the skill of 2 m-temperature and wind gusts forecasts