Verification and post-processing of ensemble weather forecasts for renewable energy applications

The energy transition taking place in Germany encourages a large scale penetration of weather-dependent energy sources into the power grid. The grid integration of intermittent sources increases the need for balancing demand and supply in order to ensure the reliability and safety of the power system. In this context, forecasts are essential for the cost-effective management of reserves and trading activities. Solar and wind power forecasts with a time horizon of few hours up to several days are usually based on outputs of numerical weather prediction systems routinely provided by weather centres. At the German Weather Service, the high-resolution ensemble prediction system COSMO-DE-EPS is called to support renewable energy applications which require dealing with the intermittency and uncertainty in the energy production. In this study, ensemble forecast verification and post-processing are addressed focusing on global radiation, which is the main weather variable affecting solar power production. First, the ensemble forecast performances are assessed from the user’s and developer’s perspectives. New tools are proposed for the verification of quantile forecasts which are probabilistic products appropriate for many renewable energy applications. Forecast discrimination ability and value are assessed considering users with different aversions to under- and over-forecasting. Moreover, a new measure is introduced in order to summarize the added value of the ensemble approach with respect to a single run approach. The new skill score is conditioned on calibration, that is, statistical consistency between the distributional forecasts and observations. Second, an enhanced framework for the post-processing of ensemble forecasts is proposed. The aim is to provide the users with calibrated consistent scenarios which are required for the optimization of complex decision-making processes. Therefore, a two-step procedure is developed starting with the marginal calibration of the forecasts based on quantile regression and the selection of appropriate predictors. Next, consistent scenarios are generated using a dual ensemble copula coupling approach which combines information from past error statistics and the dependence structure in the original ensemble forecast

Improving medium-range ensemble weather forecasts with hierarchical ensemble transformers

Statistical post-processing of global ensemble weather forecasts is revisited by leveraging recent developments in machine learning. Verification of past forecasts is exploited to learn systematic deficiencies of numerical weather predictions in order to boost post-processed forecast performance. Here, we introduce PoET, a post-processing approach based on hierarchical transformers. PoET has 2 major characteristics: 1) the post-processing is applied directly to the ensemble members rather than to a predictive distribution or a functional of it, and 2) the method is ensemble-size agnostic in the sense that the number of ensemble members in training and inference mode can differ. The PoET output is a set of calibrated members that has the same size as the original ensemble but with improved reliability. Performance assessments show that PoET can bring up to 20% improvement in skill globally for 2m temperature and 2% for precipitation forecasts and outperforms the simpler statistical member-by-member method, used here as a competitive benchmark. PoET is also applied to the ENS10 benchmark dataset for ensemble post-processing and provides better results when compared to other deep learning solutions that are evaluated for most parameters. Furthermore, because each ensemble member is calibrated separately, downstream applications should directly benefit from the improvement made on the ensemble forecast with post-processing

WeatherBench 2: A benchmark for the next generation of data-driven global weather models

WeatherBench 2 is an update to the global, medium-range (1-14 day) weather forecasting benchmark proposed by Rasp et al. (2020), designed with the aim to accelerate progress in data-driven weather modeling. WeatherBench 2 consists of an open-source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state-of-the-art models: https://sites.research.google/weatherbench. This paper describes the design principles of the evaluation framework and presents results for current state-of-the-art physical and data-driven weather models. The metrics are based on established practices for evaluating weather forecasts at leading operational weather centers. We define a set of headline scores to provide an overview of model performance. In addition, we also discuss caveats in the current evaluation setup and challenges for the future of data-driven weather forecasting

Forecast verification of a 3D model of the Mediterranean Sea. The use of discrete wavelet transforms and EOFs in the skill assessment of spatial forecasts

The quality assessment of a nested model system of the Mediterranean Sea is realised. The model has two zooms in the Provencal Basin and in the Ligurian Sea, realised with a two-way nesting approach. The experiment lasts for nine weeks, and at each week sea surface temperature (SST) and sea level anomaly are assimilated. The quality assessment of the surface temperature is done in a spatio-temporal approach, to take into account the high complexity of the SST distribution. We focus on the multi-scale nature of oceanic processes using two powerful tools for spatio-temporal analysis, wavelets and Empirical Orthogonal Functions (EOFs). We apply two-dimensional wavelets to decompose the high-resolution model and observed SST into different spatial scales. The Ligurian Sea model results are compared to observations at each of those spatial scales, with special attention on how the assimilation affects the model behaviour. We also use EOFs to assess the similarities between the Mediterranean Sea model and the observed SST. The results show that the assimilation mainly affects the model large-scale features, whereas the small scales show little or no improvement and sometimes, even a decrease in their skill. The multiresolution analysis reveals the connection between large- and small-scale errors, and how the choice of the maximum correlation length of the assimilation scheme affects the distribution of the model error among the different spatial scales. (c) 2006 Elsevier B.V. All rights reserved

Etude de l'effet combiné de l'assimilation de données et de l'emboitement de grilles - application au Golfe du Lion

Modern operational ocean forecasting systems routinely use data assimilation techniques in order to take observations into account in the hydrodynamic model. Moreover, as end users require higher and higher resolution predictions, especially in coastal zones, it is now common to run nested models, where the coastal model gets its open-sea boundary conditions from a low-resolution global model. This configuration is used in the "Mediterranean Forecasting System: Towards environmental predictions" (MFSTEP) project. A global model covering the whole Mediterranean Sea is run weekly, performing 1 week of hindcast and a 10-day forecast. Regional models, using different codes and covering different areas, then use this forecast to implement boundary conditions. Local models in turn use the regional model forecasts for their own boundary conditions. This nested system has proven to be a viable and efficient system to achieve high-resolution weekly forecasts. However, when observations are available in some coastal zone, it remains unclear whether it is better to assimilate them in the global or local model. We perform twin experiments and assimilate observations in the global or in the local model, or in both of them together. We show that, when interested in the local models forecast and provided the global model fields are approximately correct, the best results are obtained when assimilating observations in the local model

WeatherBench 2: A Benchmark for the Next Generation of Data‐Driven Global Weather Models

Abstract WeatherBench 2 is an update to the global, medium‐range (1–14 days) weather forecasting benchmark proposed by (Rasp et al., 2020, https://doi.org/10.1029/2020ms002203), designed with the aim to accelerate progress in data‐driven weather modeling. WeatherBench 2 consists of an open‐source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state‐of‐the‐art models: https://sites.research.google/weatherbench. This paper describes the design principles of the evaluation framework and presents results for current state‐of‐the‐art physical and data‐driven weather models. The metrics are based on established practices for evaluating weather forecasts at leading operational weather centers. We define a set of headline scores to provide an overview of model performance. In addition, we also discuss caveats in the current evaluation setup and challenges for the future of data‐driven weather forecasting