6,609 research outputs found
Improving medium-range ensemble weather forecasts with hierarchical ensemble transformers
Statistical post-processing of global ensemble weather forecasts is revisited
by leveraging recent developments in machine learning. Verification of past
forecasts is exploited to learn systematic deficiencies of numerical weather
predictions in order to boost post-processed forecast performance. Here, we
introduce PoET, a post-processing approach based on hierarchical transformers.
PoET has 2 major characteristics: 1) the post-processing is applied directly to
the ensemble members rather than to a predictive distribution or a functional
of it, and 2) the method is ensemble-size agnostic in the sense that the number
of ensemble members in training and inference mode can differ. The PoET output
is a set of calibrated members that has the same size as the original ensemble
but with improved reliability. Performance assessments show that PoET can bring
up to 20% improvement in skill globally for 2m temperature and 2% for
precipitation forecasts and outperforms the simpler statistical
member-by-member method, used here as a competitive benchmark. PoET is also
applied to the ENS10 benchmark dataset for ensemble post-processing and
provides better results when compared to other deep learning solutions that are
evaluated for most parameters. Furthermore, because each ensemble member is
calibrated separately, downstream applications should directly benefit from the
improvement made on the ensemble forecast with post-processing
Precipitation nowcasting with generative diffusion models
In recent years traditional numerical methods for accurate weather prediction
have been increasingly challenged by deep learning methods. Numerous historical
datasets used for short and medium-range weather forecasts are typically
organized into a regular spatial grid structure. This arrangement closely
resembles images: each weather variable can be visualized as a map or, when
considering the temporal axis, as a video. Several classes of generative
models, comprising Generative Adversarial Networks, Variational Autoencoders,
or the recent Denoising Diffusion Models have largely proved their
applicability to the next-frame prediction problem, and is thus natural to test
their performance on the weather prediction benchmarks. Diffusion models are
particularly appealing in this context, due to the intrinsically probabilistic
nature of weather forecasting: what we are really interested to model is the
probability distribution of weather indicators, whose expected value is the
most likely prediction.
In our study, we focus on a specific subset of the ERA-5 dataset, which
includes hourly data pertaining to Central Europe from the years 2016 to 2021.
Within this context, we examine the efficacy of diffusion models in handling
the task of precipitation nowcasting. Our work is conducted in comparison to
the performance of well-established U-Net models, as documented in the existing
literature. Our proposed approach of Generative Ensemble Diffusion (GED)
utilizes a diffusion model to generate a set of possible weather scenarios
which are then amalgamated into a probable prediction via the use of a
post-processing network. This approach, in comparison to recent deep learning
models, substantially outperformed them in terms of overall performance.Comment: 21 pages, 6 figure
Generative ensemble deep learning severe weather prediction from a deterministic convection-allowing model
An ensemble post-processing method is developed for the probabilistic
prediction of severe weather (tornadoes, hail, and wind gusts) over the
conterminous United States (CONUS). The method combines conditional generative
adversarial networks (CGANs), a type of deep generative model, with a
convolutional neural network (CNN) to post-process convection-allowing model
(CAM) forecasts. The CGANs are designed to create synthetic ensemble members
from deterministic CAM forecasts, and their outputs are processed by the CNN to
estimate the probability of severe weather. The method is tested using
High-Resolution Rapid Refresh (HRRR) 1--24 hr forecasts as inputs and Storm
Prediction Center (SPC) severe weather reports as targets. The method produced
skillful predictions with up to 20% Brier Skill Score (BSS) increases compared
to other neural-network-based reference methods using a testing dataset of HRRR
forecasts in 2021. For the evaluation of uncertainty quantification, the method
is overconfident but produces meaningful ensemble spreads that can distinguish
good and bad forecasts. The quality of CGAN outputs is also evaluated. Results
show that the CGAN outputs behave similarly to a numerical ensemble; they
preserved the inter-variable correlations and the contribution of influential
predictors as in the original HRRR forecasts. This work provides a novel
approach to post-process CAM output using neural networks that can be applied
to severe weather prediction
Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correction
Dynamical weather and climate prediction models underpin many studies of the
Earth system and hold the promise of being able to make robust projections of
future climate change based on physical laws. However, simulations from these
models still show many differences compared with observations. Machine learning
has been applied to solve certain prediction problems with great success, and
recently it's been proposed that this could replace the role of
physically-derived dynamical weather and climate models to give better quality
simulations. Here, instead, a framework using machine learning together with
physically-derived models is tested, in which it is learnt how to correct the
errors of the latter from timestep to timestep. This maintains the physical
understanding built into the models, whilst allowing performance improvements,
and also requires much simpler algorithms and less training data. This is
tested in the context of simulating the chaotic Lorenz '96 system, and it is
shown that the approach yields models that are stable and that give both
improved skill in initialised predictions and better long-term climate
statistics. Improvements in long-term statistics are smaller than for single
time-step tendencies, however, indicating that it would be valuable to develop
methods that target improvements on longer time scales. Future strategies for
the development of this approach and possible applications to making progress
on important scientific problems are discussed.Comment: 26p, 7 figures To be published in Journal of Advances in Modeling
Earth System
- …