259 research outputs found
A Systematic Review for Transformer-based Long-term Series Forecasting
The emergence of deep learning has yielded noteworthy advancements in time
series forecasting (TSF). Transformer architectures, in particular, have
witnessed broad utilization and adoption in TSF tasks. Transformers have proven
to be the most successful solution to extract the semantic correlations among
the elements within a long sequence. Various variants have enabled transformer
architecture to effectively handle long-term time series forecasting (LTSF)
tasks. In this article, we first present a comprehensive overview of
transformer architectures and their subsequent enhancements developed to
address various LTSF tasks. Then, we summarize the publicly available LTSF
datasets and relevant evaluation metrics. Furthermore, we provide valuable
insights into the best practices and techniques for effectively training
transformers in the context of time-series analysis. Lastly, we propose
potential research directions in this rapidly evolving field
Heat Demand Forecasting with Multi-Resolutional Representation of Heterogeneous Temporal Ensemble
One of the primal challenges faced by utility companies is ensuring efficient
supply with minimal greenhouse gas emissions. The advent of smart meters and
smart grids provide an unprecedented advantage in realizing an optimised supply
of thermal energies through proactive techniques such as load forecasting. In
this paper, we propose a forecasting framework for heat demand based on neural
networks where the time series are encoded as scalograms equipped with the
capacity of embedding exogenous variables such as weather, and
holiday/non-holiday. Subsequently, CNNs are utilized to predict the heat load
multi-step ahead. Finally, the proposed framework is compared with other
state-of-the-art methods, such as SARIMAX and LSTM. The quantitative results
from retrospective experiments show that the proposed framework consistently
outperforms the state-of-the-art baseline method with real-world data acquired
from Denmark. A minimal mean error of 7.54% for MAPE and 417kW for RMSE is
achieved with the proposed framework in comparison to all other methods.Comment: https://www.climatechange.ai/papers/neurips2022/4
An autoencoder wavelet based deep neural network with attention mechanism for multi-step prediction of plant growth
Acknowledgements This research was supported as part of SMARTGREEN, an Interreg project supported by the North Sea Programme of the European Regional Development Fund of the European Union. We would like to thank all growers (UK and EU), for providing us with the presented data sets. We also wish to thank the reviewers of the paper. Their valuable feedback, suggestions and comments helped us to improve the quality of this work.Peer reviewedPostprin
Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums
With the fast development of AI-related techniques, the applications of
trajectory prediction are no longer limited to easier scenes and trajectories.
More and more heterogeneous trajectories with different representation forms,
such as 2D or 3D coordinates, 2D or 3D bounding boxes, and even
high-dimensional human skeletons, need to be analyzed and forecasted. Among
these heterogeneous trajectories, interactions between different elements
within a frame of trajectory, which we call the ``Dimension-Wise
Interactions'', would be more complex and challenging. However, most previous
approaches focus mainly on a specific form of trajectories, which means these
methods could not be used to forecast heterogeneous trajectories, not to
mention the dimension-wise interaction. Besides, previous methods mostly treat
trajectory prediction as a normal time sequence generation task, indicating
that these methods may require more work to directly analyze agents' behaviors
and social interactions at different temporal scales. In this paper, we bring a
new ``view'' for trajectory prediction to model and forecast trajectories
hierarchically according to different frequency portions from the spectral
domain to learn to forecast trajectories by considering their frequency
responses. Moreover, we try to expand the current trajectory prediction task by
introducing the dimension from ``another view'', thus extending its
application scenarios to heterogeneous trajectories vertically. Finally, we
adopt the bilinear structure to fuse two factors, including the frequency
response and the dimension-wise interaction, to forecast heterogeneous
trajectories via spectrums hierarchically in a generic way. Experiments show
that the proposed model outperforms most state-of-the-art methods on ETH-UCY,
Stanford Drone Dataset and nuScenes with heterogeneous trajectories, including
2D coordinates, 2D and 3D bounding boxes
Graph Neural Network for spatiotemporal data: methods and applications
In the era of big data, there has been a surge in the availability of data
containing rich spatial and temporal information, offering valuable insights
into dynamic systems and processes for applications such as weather
forecasting, natural disaster management, intelligent transport systems, and
precision agriculture. Graph neural networks (GNNs) have emerged as a powerful
tool for modeling and understanding data with dependencies to each other such
as spatial and temporal dependencies. There is a large amount of existing work
that focuses on addressing the complex spatial and temporal dependencies in
spatiotemporal data using GNNs. However, the strong interdisciplinary nature of
spatiotemporal data has created numerous GNNs variants specifically designed
for distinct application domains. Although the techniques are generally
applicable across various domains, cross-referencing these methods remains
essential yet challenging due to the absence of a comprehensive literature
review on GNNs for spatiotemporal data. This article aims to provide a
systematic and comprehensive overview of the technologies and applications of
GNNs in the spatiotemporal domain. First, the ways of constructing graphs from
spatiotemporal data are summarized to help domain experts understand how to
generate graphs from various types of spatiotemporal data. Then, a systematic
categorization and summary of existing spatiotemporal GNNs are presented to
enable domain experts to identify suitable techniques and to support model
developers in advancing their research. Moreover, a comprehensive overview of
significant applications in the spatiotemporal domain is offered to introduce a
broader range of applications to model developers and domain experts, assisting
them in exploring potential research topics and enhancing the impact of their
work. Finally, open challenges and future directions are discussed
Machine learning and deep learning based methods toward Industry 4.0 predictive maintenance in induction motors: Α state of the art survey
Purpose: Developments in Industry 4.0 technologies and Artificial Intelligence (AI) have enabled data-driven manufacturing. Predictive maintenance (PdM) has therefore become the prominent approach for fault detection and diagnosis (FD/D) of induction motors (IMs). The maintenance and early FD/D of IMs are critical processes, considering that they constitute the main power source in the industrial production environment. Machine learning (ML) methods have enhanced the performance and reliability of PdM. Various deep learning (DL) based FD/D methods have emerged in recent years, providing automatic feature engineering and learning and thereby alleviating drawbacks of traditional ML based methods. This paper presents a comprehensive survey of ML and DL based FD/D methods of IMs that have emerged since 2015. An overview of the main DL architectures used for this purpose is also presented. A discussion of the recent trends is given as well as future directions for research. Design/methodology/approach: A comprehensive survey has been carried out through all available publication databases using related keywords. Classification of the reviewed works has been done according to the main ML and DL techniques and algorithms Findings: DL based PdM methods have been mainly introduced and implemented for IM fault diagnosis in recent years. Novel DL FD/D methods are based on single DL techniques as well as hybrid techniques. DL methods have also been used for signal preprocessing and moreover, have been combined with traditional ML algorithms to enhance the FD/D performance in feature engineering. Publicly available datasets have been mostly used to test the performance of the developed methods, however industrial datasets should become available as well. Multi-agent system (MAS) based PdM employing ML classifiers has been explored. Several methods have investigated multiple IM faults, however, the presence of multiple faults occurring simultaneously has rarely been investigated. Originality/value: The paper presents a comprehensive review of the recent advances in PdM of IMs based on ML and DL methods that have emerged since 2015Peer Reviewe
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection
Time series are the primary data type used to record dynamic system
measurements and generated in great volume by both physical sensors and online
processes (virtual sensors). Time series analytics is therefore crucial to
unlocking the wealth of information implicit in available data. With the recent
advancements in graph neural networks (GNNs), there has been a surge in
GNN-based approaches for time series analysis. Approaches can explicitly model
inter-temporal and inter-variable relationships, which traditional and other
deep neural network-based methods struggle to do. In this survey, we provide a
comprehensive review of graph neural networks for time series analysis
(GNN4TS), encompassing four fundamental dimensions: Forecasting,
classification, anomaly detection, and imputation. Our aim is to guide
designers and practitioners to understand, build applications, and advance
research of GNN4TS. At first, we provide a comprehensive task-oriented taxonomy
of GNN4TS. Then, we present and discuss representative research works and,
finally, discuss mainstream applications of GNN4TS. A comprehensive discussion
of potential future research directions completes the survey. This survey, for
the first time, brings together a vast array of knowledge on GNN-based time
series research, highlighting both the foundations, practical applications, and
opportunities of graph neural networks for time series analysis.Comment: 27 pages, 6 figures, 5 table
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Multivariate time series (MTS) forecasting has shown great importance in
numerous industries. Current state-of-the-art graph neural network (GNN)-based
forecasting methods usually require both graph networks (e.g., GCN) and
temporal networks (e.g., LSTM) to capture inter-series (spatial) dynamics and
intra-series (temporal) dependencies, respectively. However, the uncertain
compatibility of the two networks puts an extra burden on handcrafted model
designs. Moreover, the separate spatial and temporal modeling naturally
violates the unified spatiotemporal inter-dependencies in real world, which
largely hinders the forecasting performance. To overcome these problems, we
explore an interesting direction of directly applying graph networks and
rethink MTS forecasting from a pure graph perspective. We first define a novel
data structure, hypervariate graph, which regards each series value (regardless
of variates or timestamps) as a graph node, and represents sliding windows as
space-time fully-connected graphs. This perspective considers spatiotemporal
dynamics unitedly and reformulates classic MTS forecasting into the predictions
on hypervariate graphs. Then, we propose a novel architecture Fourier Graph
Neural Network (FourierGNN) by stacking our proposed Fourier Graph Operator
(FGO) to perform matrix multiplications in Fourier space. FourierGNN
accommodates adequate expressiveness and achieves much lower complexity, which
can effectively and efficiently accomplish the forecasting. Besides, our
theoretical analysis reveals FGO's equivalence to graph convolutions in the
time domain, which further verifies the validity of FourierGNN. Extensive
experiments on seven datasets have demonstrated our superior performance with
higher efficiency and fewer parameters compared with state-of-the-art methods.Comment: arXiv admin note: substantial text overlap with arXiv:2210.0309
- …