14 research outputs found
Advancing Parsimonious Deep Learning Weather Prediction using the HEALPix Mesh
We present a parsimonious deep learning weather prediction model on the
Hierarchical Equal Area isoLatitude Pixelization (HEALPix) to forecast seven
atmospheric variables for arbitrarily long lead times on a global approximately
110 km mesh at 3h time resolution. In comparison to state-of-the-art machine
learning weather forecast models, such as Pangu-Weather and GraphCast, our
DLWP-HPX model uses coarser resolution and far fewer prognostic variables. Yet,
at one-week lead times its skill is only about one day behind the
state-of-the-art numerical weather prediction model from the European Centre
for Medium-Range Weather Forecasts. We report successive forecast improvements
resulting from model design and data-related decisions, such as switching from
the cubed sphere to the HEALPix mesh, inverting the channel depth of the U-Net,
and introducing gated recurrent units (GRU) on each level of the U-Net
hierarchy. The consistent east-west orientation of all cells on the HEALPix
mesh facilitates the development of location-invariant convolution kernels that
are successfully applied to propagate global weather patterns across our
planet. Without any loss of spectral power after two days, the model can be
unrolled autoregressively for hundreds of steps into the future to generate
stable and realistic states of the atmosphere that respect seasonal trends, as
showcased in one-year simulations. Our parsimonious DLWP-HPX model is
research-friendly and potentially well-suited for sub-seasonal and seasonal
forecasting
Latent State Inference in a Spatiotemporal Generative Model
Knowledge of the hidden factors that determine particular system dynamics is
crucial for both explaining them and pursuing goal-directed, interventional
actions. The inference of these factors without supervision given time series
data remains an open challenge. Here, we focus on spatio-temporal processes,
including wave propagations and weather dynamics, and assume that universal
causes (e.g. physics) apply throughout space and time. We apply a novel
DIstributed, Spatio-Temporal graph Artificial Neural network Architecture,
DISTANA, which learns a generative model in such domains. DISTANA requires
fewer parameters, and yields more accurate predictions than temporal
convolutional neural networks and other related approaches on a 2D circular
wave prediction task. We show that DISTANA, when combined with a retrospective
latent state inference principle called active tuning, can reliably derive
hidden local causal factors. In a current weather prediction benchmark, DISTANA
infers our planet's land-sea mask solely by observing temperature dynamics and
uses the self inferred information to improve its own prediction of
temperature. We are convinced that the retrospective inference of latent states
in generative RNN architectures will play an essential role in future research
on causal inference and explainable systems.Comment: As submitted to the 35th conference of the Association for the
Advancement of Artificial Intelligence (AAAI-21
Inferring, predicting, and denoising causal wave dynamics
The novel DISTributed Artificial neural Network Architecture (DISTANA) is a
generative, recurrent graph convolution neural network. It implements a grid or
mesh of locally parameterizable laterally connected network modules. DISTANA is
specifically designed to identify the causality behind spatially distributed,
non-linear dynamical processes. We show that DISTANA is very well-suited to
denoise data streams, given that re-occurring patterns are observed,
significantly outperforming alternative approaches, such as temporal
convolution networks and ConvLSTMs, on a complex spatial wave propagation
benchmark. It produces stable and accurate closed-loop predictions even over
hundreds of time steps. Moreover, it is able to effectively filter noise -- an
ability that can be improved further by applying denoising autoencoder
principles or by actively tuning latent neural state activities
retrospectively. Results confirm that DISTANA is ready to model real-world
spatio-temporal dynamics such as brain imaging, supply networks, water flow, or
soil and weather data patterns.Comment: As accepted by the 29th International Conference on Artificial Neural
Networks (ICANN20