2,427 research outputs found
Bridging the Gap Between Training and Inference for Spatio-Temporal Forecasting
Spatio-temporal sequence forecasting is one of the fundamental tasks in
spatio-temporal data mining. It facilitates many real world applications such
as precipitation nowcasting, citywide crowd flow prediction and air pollution
forecasting. Recently, a few Seq2Seq based approaches have been proposed, but
one of the drawbacks of Seq2Seq models is that, small errors can accumulate
quickly along the generated sequence at the inference stage due to the
different distributions of training and inference phase. That is because
Seq2Seq models minimise single step errors only during training, however the
entire sequence has to be generated during the inference phase which generates
a discrepancy between training and inference. In this work, we propose a novel
curriculum learning based strategy named Temporal Progressive Growing Sampling
to effectively bridge the gap between training and inference for
spatio-temporal sequence forecasting, by transforming the training process from
a fully-supervised manner which utilises all available previous ground-truth
values to a less-supervised manner which replaces some of the ground-truth
context with generated predictions. To do that we sample the target sequence
from midway outputs from intermediate models trained with bigger timescales
through a carefully designed decaying strategy. Experimental results
demonstrate that our proposed method better models long term dependencies and
outperforms baseline approaches on two competitive datasets.Comment: ECAI 2020 Accepted, preprin
Spatiotemporal Observer Design for Predictive Learning of High-Dimensional Data
Although deep learning-based methods have shown great success in
spatiotemporal predictive learning, the framework of those models is designed
mainly by intuition. How to make spatiotemporal forecasting with theoretical
guarantees is still a challenging issue. In this work, we tackle this problem
by applying domain knowledge from the dynamical system to the framework design
of deep learning models. An observer theory-guided deep learning architecture,
called Spatiotemporal Observer, is designed for predictive learning of high
dimensional data. The characteristics of the proposed framework are twofold:
firstly, it provides the generalization error bound and convergence guarantee
for spatiotemporal prediction; secondly, dynamical regularization is introduced
to enable the model to learn system dynamics better during training. Further
experimental results show that this framework could capture the spatiotemporal
dynamics and make accurate predictions in both one-step-ahead and
multi-step-ahead forecasting scenarios.Comment: Under review by IEEE Transactions on Pattern Analysis and Machine
Intelligenc
SimVP: Towards Simple yet Powerful Spatiotemporal Predictive Learning
Recent years have witnessed remarkable advances in spatiotemporal predictive
learning, incorporating auxiliary inputs, elaborate neural architectures, and
sophisticated training strategies. Although impressive, the system complexity
of mainstream methods is increasing as well, which may hinder the convenient
applications. This paper proposes SimVP, a simple spatiotemporal predictive
baseline model that is completely built upon convolutional networks without
recurrent architectures and trained by common mean squared error loss in an
end-to-end fashion. Without introducing any extra tricks and strategies, SimVP
can achieve superior performance on various benchmark datasets. To further
improve the performance, we derive variants with the gated spatiotemporal
attention translator from SimVP that can achieve better performance. We
demonstrate that SimVP has strong generalization and extensibility on
real-world datasets through extensive experiments. The significant reduction in
training cost makes it easier to scale to complex scenarios. We believe SimVP
can serve as a solid baseline to benefit the spatiotemporal predictive learning
community
- …