The ESCAPE project : Energy-efficient Scalable Algorithms for Weather Prediction at Exascale

In the simulation of complex multi-scale flows arising in weather and climate modelling, one of the biggest challenges is to satisfy strict service requirements in terms of time to solution and to satisfy budgetary constraints in terms of energy to solution, without compromising the accuracy and stability of the application. These simulations require algorithms that minimise the energy footprint along with the time required to produce a solution, maintain the physically required level of accuracy, are numerically stable, and are resilient in case of hardware failure. The European Centre for Medium-Range Weather Forecasts (ECMWF) led the ESCAPE (Energy-efficient Scalable Algorithms for Weather Prediction at Exascale) project, funded by Horizon 2020 (H2020) under the FET-HPC (Future and Emerging Technologies in High Performance Computing) initiative. The goal of ESCAPE was to develop a sustainable strategy to evolve weather and climate prediction models to next-generation computing technologies. The project partners incorporate the expertise of leading European regional forecasting consortia, university research, experienced high-performance computing centres, and hardware vendors. This paper presents an overview of the ESCAPE strategy: (i) identify domain-specific key algorithmic motifs in weather prediction and climate models (which we term Weather & Climate Dwarfs), (ii) categorise them in terms of computational and communication patterns while (iii) adapting them to different hardware architectures with alternative programming models, (iv) analyse the challenges in optimising, and (v) find alternative algorithms for the same scheme. The participating weather prediction models are the following: IFS (Integrated Forecasting System); ALARO, a combination of AROME (Application de la Recherche a l'Operationnel a Meso-Echelle) and ALADIN (Aire Limitee Adaptation Dynamique Developpement International); and COSMO-EULAG, a combination of COSMO (Consortium for Small-scale Modeling) and EULAG (Eulerian and semi-Lagrangian fluid solver). For many of the weather and climate dwarfs ESCAPE provides prototype implementations on different hardware architectures (mainly Intel Skylake CPUs, NVIDIA GPUs, Intel Xeon Phi, Optalysys optical processor) with different programming models. The spectral transform dwarf represents a detailed example of the co-design cycle of an ESCAPE dwarf. The dwarf concept has proven to be extremely useful for the rapid prototyping of alternative algorithms and their interaction with hardware; e.g. the use of a domain-specific language (DSL). Manual adaptations have led to substantial accelerations of key algorithms in numerical weather prediction (NWP) but are not a general recipe for the performance portability of complex NWP models. Existing DSLs are found to require further evolution but are promising tools for achieving the latter. Measurements of energy and time to solution suggest that a future focus needs to be on exploiting the simultaneous use of all available resources in hybrid CPU-GPU arrangements

Improved ECMWF forecasts of direct normal irradiance: A tool for better operational strategies in concentrating solar power plants

To contribute for improved operational strategies of concentrating solar power plants with accurate forecasts of direct normal irradiance, this work describes the use of several post-processing methods on numerical weather prediction. Focus is given to a multivariate regression model that uses measured irradiance values from previous hours to improve next-hour predictions, which can be used to refine daily strategies based on day-ahead predictions. Short-term forecasts provided by the Integrated Forecasting System, the global model from the European Centre for Medium-Range Weather Forecasts (ECMWF), are used together with measurements in southern Portugal. As a nowcasting tool, the proposed regression model significantly improves hourly predictions with a skill score of z0.84 (i.e. an increase of z27.29% towards the original hourly forecasts). Using previous-day measured availability to improve next-day forecasts, the model shows a skill score of z0.78 (i.e. an increase of z6% towards the original forecasts), being further improved if larger sets of data are used. Through a power plant simulator (i.e. the System Advisor Model), a preliminary economic analysis shows that using improved hourly predictions of electrical energy allows to enhance a power plant’s profit in z0.44 MV/year, as compared with the original forecasts. Operational strategies are proposed accordingly

Land-Focused Changes in the Updated GEOS FP System (Version 5.25)

Many of the changes imposed in the January 2020 upgrade from Version 5.22 to 5.25 of the Goddard Earth Observing System (GEOS) Forward Processing (FP) analysis system were designed to increase the realism of simulated land variables. The changes, which consist of both land model parameter updates and improvements to the physical treatments employed for various land processes, have generally positive or neutral impacts on the character of the FP product, as documented here

Comparison of nonhomogeneous regression models for probabilistic wind speed forecasting

In weather forecasting, nonhomogeneous regression is used to statistically postprocess forecast ensembles in order to obtain calibrated predictive distributions. For wind speed forecasts, the regression model is given by a truncated normal distribution where location and spread are derived from the ensemble. This paper proposes two alternative approaches which utilize the generalized extreme value (GEV) distribution. A direct alternative to the truncated normal regression is to apply a predictive distribution from the GEV family, while a regime switching approach based on the median of the forecast ensemble incorporates both distributions. In a case study on daily maximum wind speed over Germany with the forecast ensemble from the European Centre for Medium-Range Weather Forecasts, all three approaches provide calibrated and sharp predictive distributions with the regime switching approach showing the highest skill in the upper tail

Evaluating Ensemble Post-Processing for Wind Power Forecasts

Capturing the uncertainty in probabilistic wind power forecasts is challenging, especially when uncertain input variables such as the weather, play a role. Since ensemble weather predictions aim to capture the uncertainty in the weather system, they can be used to propagate this uncertainty through to subsequent wind power forecasting models. However, as weather ensemble systems are known to be biased and underdispersed, meteorologists post-process the ensembles. This post-processing can successfully correct the biases in the weather variables but has not been evaluated thoroughly in the context of subsequent forecasts, such as wind power generation forecasts. The present paper evaluates multiple strategies for applying ensemble post-processing to probabilistic wind power forecasts. We use Ensemble Model Output Statistics (EMOS) as the post-processing method and evaluate four possible strategies: only using the raw ensembles without post-processing, a one-step strategy where only the weather ensembles are post-processed, a one-step strategy where we only post-process the power ensembles, and a two-step strategy where we post-process both the weather and power ensembles. Results show that post-processing the final wind power ensemble improves forecast performance regarding both calibration and sharpness, whilst only post-processing the weather ensembles does not necessarily lead to increased forecast performance