Test-Time Compensated Representation Learning for Extreme Traffic Forecasting

Traffic forecasting is a challenging task due to the complex spatio-temporal correlations among traffic series. In this paper, we identify an underexplored problem in multivariate traffic series prediction: extreme events. Road congestion and rush hours can result in low correlation in vehicle speeds at various intersections during adjacent time periods. Existing methods generally predict future series based on recent observations and entirely discard training data during the testing phase, rendering them unreliable for forecasting highly nonlinear multivariate time series. To tackle this issue, we propose a test-time compensated representation learning framework comprising a spatio-temporal decomposed data bank and a multi-head spatial transformer model (CompFormer). The former component explicitly separates all training data along the temporal dimension according to periodicity characteristics, while the latter component establishes a connection between recent observations and historical series in the data bank through a spatial attention matrix. This enables the CompFormer to transfer robust features to overcome anomalous events while using fewer computational resources. Our modules can be flexibly integrated with existing forecasting methods through end-to-end training, and we demonstrate their effectiveness on the METR-LA and PEMS-BAY benchmarks. Extensive experimental results show that our method is particularly important in extreme events, and can achieve significant improvements over six strong baselines, with an overall improvement of up to 28.2%.Comment: 13 pages, 10 figures, 5 table

Long Term Predictive Modeling on Big Spatio-Temporal Data

In the era of massive data, one of the most promising research fields involves the analysis of large-scale Spatio-temporal databases to discover exciting and previously unknown but potentially useful patterns from data collected over time and space. A modeling process in this domain must take temporal and spatial correlations into account, but with the dimensionality of the time and space measurements increasing, the number of elements potentially contributing to a target sharply grows, making the target\u27s long-term behavior highly complex, chaotic, highly dynamic, and hard to predict. Therefore, two different considerations are taken into account in this work: one is about how to identify the most relevant and meaningful features from the original Spatio-temporal feature space; the other is about how to model complex space-time dynamics with sensitive dependence on initial and boundary conditions. First, identifying strongly related features and removing the irrelevant or less important features with respect to a target feature from large-scale Spatio-temporal data sets is a critical and challenging issue in many fields, including the evolutionary history of crime hot spots, uncovering weather patterns, predicting floodings, earthquakes, and hurricanes, and determining global warming trends. The optimal sub-feature-set that contains all the valuable information is called the Markov Boundary. Unfortunately, the existing feature selection methods often focus on identifying a single Markov Boundary when real-world data could have many feature subsets that are equally good boundaries. In our work, we design a new multiple-Markov-boundary-based predictive model, Galaxy, to identify the precursors to heavy precipitation event clusters and predict heavy rainfall with a long lead time. We applied Galaxy to an extremely high-dimensional meteorological data set and finally determined 15 Markov boundaries related to heavy rainfall events in the Des Moines River Basin in Iowa. Our model identified the cold surges along the coast of Asia as an essential precursor to the surface weather over the United States, a finding which was later corroborated by climate experts. Second, chaotic behavior exists in many nonlinear Spatio-temporal systems, such as climate dynamics, weather prediction, and the space-time dynamics of virus spread. A reliable solution for these systems must handle their complex space-time dynamics and sensitive dependence on initial and boundary conditions. Deep neural networks\u27 hierarchical feature learning capabilities in both spatial and temporal domains are helpful for nonlinear Spatio-temporal dynamics modeling. However, sensitive dependence on initial and boundary conditions is still challenging for theoretical research and many critical applications. This study proposes a new recurrent architecture, error trajectory tracing, and accompanying training regime, Horizon Forcing, for prediction in chaotic systems. These methods have been validated on real-world Spatio-temporal data sets, including one meteorological dataset, three classics, chaotic systems, and four real-world time series prediction tasks with chaotic characteristics. Experiments\u27 results show that each proposed model could outperform the performance of current baseline approaches

Predicting Spatio-Temporal Time Series Using Dimension Reduced Local States

We present a method for both cross estimation and iterated time series prediction of spatio temporal dynamics based on reconstructed local states, PCA dimension reduction, and local modelling using nearest neighbour methods. The effectiveness of this approach is shown for (noisy) data from a (cubic) Barkley model, the Bueno-Orovio-Cherry-Fenton model, and the Kuramoto-Sivashinsky model

Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data

Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. More recently, as deep learning models have become more common, RNNs have been used to forecast increasingly complicated systems. Dynamical spatio-temporal processes represent a class of complex systems that can potentially benefit from these types of models. Although the RNN literature is expansive and highly developed, uncertainty quantification is often ignored. Even when considered, the uncertainty is generally quantified without the use of a rigorous framework, such as a fully Bayesian setting. Here we attempt to quantify uncertainty in a more formal framework while maintaining the forecast accuracy that makes these models appealing, by presenting a Bayesian RNN model for nonlinear spatio-temporal forecasting. Additionally, we make simple modifications to the basic RNN to help accommodate the unique nature of nonlinear spatio-temporal data. The proposed model is applied to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications

Generalised additive multiscale wavelet models constructed using particle swarm optimisation and mutual information for spatio-temporal evolutionary system representation

A new class of generalised additive multiscale wavelet models (GAMWMs) is introduced for high dimensional spatio-temporal evolutionary (STE) system identification. A novel two-stage hybrid learning scheme is developed for constructing such an additive wavelet model. In the first stage, a new orthogonal projection pursuit (OPP) method, implemented using a particle swarm optimisation(PSO) algorithm, is proposed for successively augmenting an initial coarse wavelet model, where relevant parameters of the associated wavelets are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be a redundant model. In the second stage, a forward orthogonal regression (FOR) algorithm, implemented using a mutual information method, is then applied to refine and improve the initially constructed wavelet model. The proposed two-stage hybrid method can generally produce a parsimonious wavelet model, where a ranked list of wavelet functions, according to the capability of each wavelet to represent the total variance in the desired system output signal is produced. The proposed new modelling framework is applied to real observed images, relative to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, and the associated identification results show that the new modelling framework is applicable and effective for handling high dimensional identification problems of spatio-temporal evolution sytems

A new class of multiscale lattice cell (MLC) models for spatio-temporal evolutionary image representation

Spatio-temporal evolutionary (STE) images are a class of complex dynamical systems that evolve over both space and time. With increased interest in the investigation of nonlinear complex phenomena, especially spatio-temporal behaviour governed by evolutionary laws that are dependent on both spatial and temporal dimensions, there has been an increased need to investigate model identification methods for this class of complex systems. Compared with pure temporal processes, the identification of spatio-temporal models from observed images is much more difficult and quite challenging. Starting with an assumption that there is no apriori information about the true model but only observed data are available, this study introduces a new class of multiscale lattice cell (MLC) models to represent the rules of the associated spatio-temporal evolutionary system. An application to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, is investigated to demonstrate the new modelling framework