33 research outputs found
A Three-dimensional Convolutional-Recurrent Network for Convective Storm Nowcasting
Very short-term convective storm forecasting, termed nowcasting, has long
been an important issue and has attracted substantial interest. Existing
nowcasting methods rely principally on radar images and are limited in terms of
nowcasting storm initiation and growth. Real-time re-analysis of meteorological
data supplied by numerical models provides valuable information about
three-dimensional (3D), atmospheric, boundary layer thermal dynamics, such as
temperature and wind. To mine such data, we here develop a
convolution-recurrent, hybrid deep-learning method with the following
characteristics: (1) the use of cell-based oversampling to increase the number
of training samples; this mitigates the class imbalance issue; (2) the use of
both raw 3D radar data and 3D meteorological data re-analyzed via multi-source
3D convolution without any need for handcraft feature engineering; and (3) the
stacking of convolutional neural networks on a long short-term memory
encoder/decoder that learns the spatiotemporal patterns of convective
processes. Experimental results demonstrated that our method performs better
than other extrapolation methods. Qualitative analysis yielded encouraging
nowcasting results.Comment: 13 pages, 11 figures, accepted by 2019 IEEE International Conference
on Big Knowledge The copyright of this paper has been transferred to the
IEEE, please comply with the copyright of the IEE
A Review on Deep Learning Techniques for Video Prediction
The ability to predict, anticipate and reason about future outcomes is a key component of intelligent decision-making systems. In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction. Defined as a self-supervised learning task, video prediction represents a suitable framework for representation learning, as it demonstrated potential capabilities for extracting meaningful representations of the underlying patterns in natural videos. Motivated by the increasing interest in this task, we provide a review on the deep learning methods for prediction in video sequences. We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets. Next, we carefully analyze existing video prediction models organized according to a proposed taxonomy, highlighting their contributions and their significance in the field. The summary of the datasets and methods is accompanied with experimental results that facilitate the assessment of the state of the art on a quantitative basis. The paper is summarized by drawing some general conclusions, identifying open research challenges and by pointing out future research directions.This work has been funded by the Spanish Government PID2019-104818RB-I00 grant for the MoDeaAS project, supported with Feder funds. This work has also been supported by two Spanish national grants for PhD studies, FPU17/00166, and ACIF/2018/197 respectively
Inductive biases in deep learning models for weather prediction
Deep learning has recently gained immense popularity in the Earth sciences as
it enables us to formulate purely data-driven models of complex Earth system
processes. Deep learning-based weather prediction (DLWP) models have made
significant progress in the last few years, achieving forecast skills
comparable to established numerical weather prediction (NWP) models with
comparatively lesser computational costs. In order to train accurate, reliable,
and tractable DLWP models with several millions of parameters, the model design
needs to incorporate suitable inductive biases that encode structural
assumptions about the data and modelled processes. When chosen appropriately,
these biases enable faster learning and better generalisation to unseen data.
Although inductive biases play a crucial role in successful DLWP models, they
are often not stated explicitly and how they contribute to model performance
remains unclear. Here, we review and analyse the inductive biases of six
state-of-the-art DLWP models, involving a deeper look at five key design
elements: input data, forecasting objective, loss components, layered design of
the deep learning architectures, and optimisation methods. We show how the
design choices made in each of the five design elements relate to structural
assumptions. Given recent developments in the broader DL community, we
anticipate that the future of DLWP will likely see a wider use of foundation
models -- large models pre-trained on big databases with self-supervised
learning -- combined with explicit physics-informed inductive biases that allow
the models to provide competitive forecasts even at the more challenging
subseasonal-to-seasonal scales
Hurricane Forecasting: A Novel Multimodal Machine Learning Framework
This paper describes a machine learning (ML) framework for tropical cyclone
intensity and track forecasting, combining multiple distinct ML techniques and
utilizing diverse data sources. Our framework, which we refer to as Hurricast
(HURR), is built upon the combination of distinct data processing techniques
using gradient-boosted trees and novel encoder-decoder architectures, including
CNN, GRU and Transformers components. We propose a deep-feature extractor
methodology to mix spatial-temporal data with statistical data efficiently. Our
multimodal framework unleashes the potential of making forecasts based on a
wide range of data sources, including historical storm data, and visual data
such as reanalysis atmospheric images. We evaluate our models with current
operational forecasts in North Atlantic and Eastern Pacific basins on 2016-2019
for 24-hour lead time, and show our models consistently outperform
statistical-dynamical models and compete with the best dynamical models, while
computing forecasts in seconds. Furthermore, the inclusion of Hurricast into an
operational forecast consensus model leads to a significant improvement of 5% -
15% over NHC's official forecast, thus highlighting the complementary
properties with existing approaches. In summary, our work demonstrates that
combining different data sources and distinct machine learning methodologies
can lead to superior tropical cyclone forecasting. We hope that this work opens
the door for further use of machine learning in meteorological forecasting.Comment: Under revision by the AMS' Weather and Forecasting journa
ClimaX: A foundation model for weather and climate
Most state-of-the-art approaches for weather and climate modeling are based
on physics-informed numerical models of the atmosphere. These approaches aim to
model the non-linear dynamics and complex interactions between multiple
variables, which are challenging to approximate. Additionally, many such
numerical models are computationally intensive, especially when modeling the
atmospheric phenomenon at a fine-grained spatial and temporal resolution.
Recent data-driven approaches based on machine learning instead aim to directly
solve a downstream forecasting or projection task by learning a data-driven
functional mapping using deep neural networks. However, these networks are
trained using curated and homogeneous climate datasets for specific
spatiotemporal tasks, and thus lack the generality of numerical models. We
develop and demonstrate ClimaX, a flexible and generalizable deep learning
model for weather and climate science that can be trained using heterogeneous
datasets spanning different variables, spatio-temporal coverage, and physical
groundings. ClimaX extends the Transformer architecture with novel encoding and
aggregation blocks that allow effective use of available compute while
maintaining general utility. ClimaX is pre-trained with a self-supervised
learning objective on climate datasets derived from CMIP6. The pre-trained
ClimaX can then be fine-tuned to address a breadth of climate and weather
tasks, including those that involve atmospheric variables and spatio-temporal
scales unseen during pretraining. Compared to existing data-driven baselines,
we show that this generality in ClimaX results in superior performance on
benchmarks for weather forecasting and climate projections, even when
pretrained at lower resolutions and compute budgets. The source code is
available at https://github.com/microsoft/ClimaX.Comment: International Conference on Machine Learning 202
Stacked LSTM sequence-to-sequence autoencoder with feature selection for daily solar radiation prediction: a review and new modeling results
We review the latest modeling techniques and propose new hybrid SAELSTM framework based on Deep Learning (DL) to construct prediction intervals for daily Global Solar Radiation (GSR) using the Manta Ray Foraging Optimization (MRFO) feature selection to select model parameters. Features are employed as potential inputs for Long Short-Term Memory and a seq2seq SAELSTM autoencoder Deep Learning (DL) system in the final GSR prediction. Six solar energy farms in Queensland, Australia are considered to evaluate the method with predictors from Global Climate Models and ground-based observation. Comparisons are carried out among DL models (i.e., Deep Neural Network) and conventional Machine Learning algorithms (i.e., Gradient Boosting Regression, Random Forest Regression, Extremely Randomized Trees, and Adaptive Boosting Regression). The hyperparameters are deduced with grid search, and simulations demonstrate that the DL hybrid SAELSTM model is accurate compared with the other models as well as the persistence methods. The SAELSTM model obtains quality solar energy prediction intervals with high coverage probability and low interval errors. The review and new modelling results utilising an autoencoder deep learning method show that our approach is acceptable to predict solar radiation, and therefore is useful in solar energy monitoring systems to capture the stochastic variations in solar power generation due to cloud cover, aerosols, ozone changes, and other atmospheric attenuation factors