48 research outputs found
An Introduction to Temporal Graphs: An Algorithmic Perspective
A \emph{temporal graph} is, informally speaking, a graph that changes with time. When time is discrete and only the relationships between the participating entities may change and not the entities themselves, a temporal graph may be viewed as a sequence of static graphs over the same (static) set of nodes . Though static graphs have been extensively studied, for their temporal generalization we are still far from having a concrete set of structural and algorithmic principles. Recent research shows that many graph properties and problems become radically different and usually substantially more difficult when an extra time dimension in added to them. Moreover, there is already a rich and rapidly growing set of modern systems and applications that can be naturally modeled and studied via temporal graphs. This, further motivates the need for the development of a temporal extension of graph theory. We survey here recent results on temporal graphs and temporal graph problems that have appeared in the Computer Science community
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection
Time series are the primary data type used to record dynamic system
measurements and generated in great volume by both physical sensors and online
processes (virtual sensors). Time series analytics is therefore crucial to
unlocking the wealth of information implicit in available data. With the recent
advancements in graph neural networks (GNNs), there has been a surge in
GNN-based approaches for time series analysis. Approaches can explicitly model
inter-temporal and inter-variable relationships, which traditional and other
deep neural network-based methods struggle to do. In this survey, we provide a
comprehensive review of graph neural networks for time series analysis
(GNN4TS), encompassing four fundamental dimensions: Forecasting,
classification, anomaly detection, and imputation. Our aim is to guide
designers and practitioners to understand, build applications, and advance
research of GNN4TS. At first, we provide a comprehensive task-oriented taxonomy
of GNN4TS. Then, we present and discuss representative research works and,
finally, discuss mainstream applications of GNN4TS. A comprehensive discussion
of potential future research directions completes the survey. This survey, for
the first time, brings together a vast array of knowledge on GNN-based time
series research, highlighting both the foundations, practical applications, and
opportunities of graph neural networks for time series analysis.Comment: 27 pages, 6 figures, 5 table
Cluster Editing in Multi-Layer and Temporal Graphs
Motivated by the recent rapid growth of research for algorithms to cluster multi-layer and temporal graphs, we study extensions of the classical Cluster Editing problem. In Multi-Layer Cluster Editing we receive a set of graphs on the same vertex set, called layers and aim to transform all layers into cluster graphs (disjoint unions of cliques) that differ only slightly. More specifically, we want to mark at most d vertices and to transform each layer into a cluster graph using at most k edge additions or deletions per layer so that, if we remove the marked vertices, we obtain the same cluster graph in all layers. In Temporal Cluster Editing we receive a sequence of layers and we want to transform each layer into a cluster graph so that consecutive layers differ only slightly. That is, we want to transform each layer into a cluster graph with at most k edge additions or deletions and to mark a distinct set of d vertices in each layer so that each two consecutive layers are the same after removing the vertices marked in the first of the two layers. We study the combinatorial structure of the two problems via their parameterized complexity with respect to the parameters d and k, among others. Despite the similar definition, the two problems behave quite differently: In particular, Multi-Layer Cluster Editing is fixed-parameter tractable with running time k^{O(k + d)} s^{O(1)} for inputs of size s, whereas Temporal Cluster Editing is W[1]-hard with respect to k even if d = 3
Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU
Dynamic graph neural network (DGNN) is becoming increasingly popular because
of its widespread use in capturing dynamic features in the real world. A
variety of dynamic graph neural networks designed from algorithmic perspectives
have succeeded in incorporating temporal information into graph processing.
Despite the promising algorithmic performance, deploying DGNNs on hardware
presents additional challenges due to the model complexity, diversity, and the
nature of the time dependency. Meanwhile, the differences between DGNNs and
static graph neural networks make hardware-related optimizations for static
graph neural networks unsuitable for DGNNs. In this paper, we select eight
prevailing DGNNs with different characteristics and profile them on both CPU
and GPU. The profiling results are summarized and analyzed, providing in-depth
insights into the bottlenecks of DGNNs on hardware and identifying potential
optimization opportunities for future DGNN acceleration. Followed by a
comprehensive survey, we provide a detailed analysis of DGNN performance
bottlenecks on hardware, including temporal data dependency, workload
imbalance, data movement, and GPU warm-up. We suggest several optimizations
from both software and hardware perspectives. This paper is the first to
provide an in-depth analysis of the hardware performance of DGNN Code is
available at https://github.com/sharc-lab/DGNN_analysis.Comment: 14 pages main text, 2 pages appendix, 10 figures, submitted to
IISWC202
A Survey on Graph Representation Learning Methods
Graphs representation learning has been a very active research area in recent
years. The goal of graph representation learning is to generate graph
representation vectors that capture the structure and features of large graphs
accurately. This is especially important because the quality of the graph
representation vectors will affect the performance of these vectors in
downstream tasks such as node classification, link prediction and anomaly
detection. Many techniques are proposed for generating effective graph
representation vectors. Two of the most prevalent categories of graph
representation learning are graph embedding methods without using graph neural
nets (GNN), which we denote as non-GNN based graph embedding methods, and graph
neural nets (GNN) based methods. Non-GNN graph embedding methods are based on
techniques such as random walks, temporal point processes and neural network
learning methods. GNN-based methods, on the other hand, are the application of
deep learning on graph data. In this survey, we provide an overview of these
two categories and cover the current state-of-the-art methods for both static
and dynamic graphs. Finally, we explore some open and ongoing research
directions for future work
Advances in Learning and Understanding with Graphs through Machine Learning
Graphs have increasingly become a crucial way of representing large, complex and disparate datasets from a range of domains, including many scientific disciplines. Graphs are particularly useful at capturing complex relationships or interdependencies within or even between datasets, and enable unique insights which are not possible with other data formats. Over recent years, significant improvements in the ability of machine learning approaches to automatically learn from and identify patterns in datasets have been made.
However due to the unique nature of graphs, and the data they are used to represent, employing machine learning with graphs has thus far proved challenging. A review of relevant literature has revealed that key challenges include issues arising with macro-scale graph learning, interpretability of machine learned representations and a failure to incorporate the temporal dimension present in many datasets. Thus, the work and contributions presented in this thesis primarily investigate how modern machine learning techniques can be adapted to tackle key graph mining tasks, with a particular focus on optimal macro-level representation, interpretability and incorporating temporal dynamics into the learning process. The majority of methods employed are novel approaches centered around attempting to use artificial neural networks in order to learn from graph datasets.
Firstly, by devising a novel graph fingerprint technique, it is demonstrated that this can successfully be applied to two different tasks whilst out-performing established baselines, namely graph comparison and classification. Secondly, it is shown that a mapping can be found between certain topological features and graph embeddings. This, for perhaps the the first time, suggests that it is possible that machines are learning something analogous to human knowledge acquisition, thus bringing interpretability to the graph embedding process. Thirdly, in exploring two new models for incorporating temporal information into the graph learning process, it is found that including such information is crucial to predictive performance in certain key tasks, such as link prediction, where state-of-the-art baselines are out-performed.
The overall contribution of this work is to provide greater insight into and explanation of the ways in which machine learning with respect to graphs is emerging as a crucial set of techniques for understanding complex datasets. This is important as these techniques can potentially be applied to a broad range of scientific disciplines. The thesis concludes with an assessment of limitations and recommendations for future research
Probabilistic Parametric Curves for Sequence Modeling
Repräsentationen sequenzieller Daten basieren in der Regel auf der Annahme, dass beobachtete Sequenzen Realisierungen eines unbekannten zugrundeliegenden stochastischen Prozesses sind. Die Bestimmung einer solchen Repräsentation wird üblicherweise als Lernproblem ausgelegt und ergibt ein Sequenzmodell. Das Modell muss in diesem Zusammenhang in der Lage sein, die multimodale Natur der Daten zu erfassen, ohne einzelne Modi zu vermischen. Zur Modellierung eines zugrundeliegenden stochastischen Prozesses lernen häufig verwendete, auf neuronalen Netzen basierende Ansätze entweder eine Wahrscheinlichkeitsverteilung zu parametrisieren oder eine implizite Repräsentation unter Verwendung stochastischer Eingaben oder Neuronen. Dabei integrieren diese Modelle in der Regel Monte Carlo Verfahren oder andere Näherungslösungen, um die Parameterschätzung und probabilistische Inferenz zu ermöglichen. Dies gilt sogar für regressionsbasierte Ansätze basierend auf Mixture Density Netzwerken, welche ebenso Monte Carlo Simulationen zur multi-modalen Inferenz benötigen. Daraus ergibt sich eine Forschungslücke für vollständig regressionsbasierte Ansätze zur Parameterschätzung und probabilistischen Inferenz.
Infolgedessen stellt die vorliegende Arbeit eine probabilistische Erweiterung für Bézierkurven (-Kurven) als Basis für die Modellierung zeitkontinuierlicher stochastischer Prozesse mit beschränkter Indexmenge vor. Das vorgestellte Modell, bezeichnet als -Kurven - Modell, basiert auf Mixture Density Netzwerken (MDN) und Bézierkurven, welche Kurvenkontrollpunkte als normalverteilt annehmen. Die Verwendung eines MDN-basierten Ansatzes steht im Einklang mit aktuellen Versuchen, Unsicherheitsschätzung als Regressionsproblem auszulegen, und ergibt ein generisches Modell, welches allgemein als Basismodell für die probabilistische Sequenzmodellierung einsetzbar ist. Ein wesentlicher Vorteil des Modells ist unter anderem die Möglichkeit glatte, multi-modale Vorhersagen in einem einzigen Inferenzschritt zu generieren, ohne dabei Monte Carlo Simulationen zu benötigen. Durch die Verwendung von Bézierkurven als Basis, kann das Modell außerdem theoretisch für beliebig hohe Datendimensionen verwendet werden, indem die Kontrollpunkte in einen hochdimensionalen Raum eingebettet werden. Um die durch den Fokus auf beschränkte Indexmengen existierenden theoretischen Einschränkungen aufzuheben, wird zusätzlich eine konzeptionelle Erweiterung für das -Kurven - Modell vorgestellt, mit der unendliche stochastische Prozesse modelliert werden können. Wesentliche Eigenschaften des vorgestellten Modells und dessen Erweiterung werden auf verschiedenen Beispielen zur Sequenzsynthese gezeigt.
Aufgrund der hinreichenden Anwendbarkeit des -Kurven - Modells auf die meisten Anwendungsfälle, wird dessen Tauglichkeit umfangreich auf verschiedenen Mehrschrittprädiktionsaufgaben unter Verwendung realer Daten evaluiert. Zunächst wird das Modell gegen häufig verwendete probabilistische Sequenzmodelle im Kontext der Vorhersage von Fußgängertrajektorien evaluiert, wobei es sämtliche Vergleichsmodelle übertrifft. In einer qualitativen Auswertung wird das Verhalten des Modells in einem Vorhersagekontext untersucht. Außerdem werden Schwierigkeiten bei der Bewertung probabilistischer Sequenzmodelle in einem multimodalen Setting diskutiert. Darüber hinaus wird das Modell im Kontext der Vorhersage menschlicher Bewegungen angewendet, um die angestrebte Skalierbarkeit des Modells auf höherdimensionale Daten zu bewerten. Bei dieser Aufgabe übertrifft das Modell allgemein verwendete einfache und auf neuronalen Netzen basierende Grundmodelle und ist in verschiedenen Situationen auf Augenhöhe mit verschiedenen State-of-the-Art-Modellen, was die Einsetzbarkeit in diesem höherdimensionalen Beispiel zeigt. Des Weiteren werden Schwierigkeiten bei der Kovarianzschätzung und die Glättungseigenschaften des -Kurven - Modells diskutiert
Probabilistic Parametric Curves for Sequence Modeling
This work proposes a probabilistic extension to BĂ©zier curves as a basis for effectively modeling stochastic processes with a bounded index set. The proposed stochastic process model is based on Mixture Density Networks and BĂ©zier curves with Gaussian random variables as control points. A key advantage of this model is given by the ability to generate multi-mode predictions in a single inference step, thus avoiding the need for Monte Carlo simulation
A Diffusion model for POI recommendation
Next Point-of-Interest (POI) recommendation is a critical task in
location-based services that aim to provide personalized suggestions for the
user's next destination. Previous works on POI recommendation have laid focused
on modeling the user's spatial preference. However, existing works that
leverage spatial information are only based on the aggregation of users'
previous visited positions, which discourages the model from recommending POIs
in novel areas. This trait of position-based methods will harm the model's
performance in many situations. Additionally, incorporating sequential
information into the user's spatial preference remains a challenge. In this
paper, we propose Diff-POI: a Diffusion-based model that samples the user's
spatial preference for the next POI recommendation. Inspired by the wide
application of diffusion algorithm in sampling from distributions, Diff-POI
encodes the user's visiting sequence and spatial character with two
tailor-designed graph encoding modules, followed by a diffusion-based sampling
strategy to explore the user's spatial visiting trends. We leverage the
diffusion process and its reversed form to sample from the posterior
distribution and optimized the corresponding score function. We design a joint
training and inference framework to optimize and evaluate the proposed
Diff-POI. Extensive experiments on four real-world POI recommendation datasets
demonstrate the superiority of our Diff-POI over state-of-the-art baseline
methods. Further ablation and parameter studies on Diff-POI reveal the
functionality and effectiveness of the proposed diffusion-based sampling
strategy for addressing the limitations of existing methods
MegaCRN: Meta-Graph Convolutional Recurrent Network for Spatio-Temporal Modeling
Spatio-temporal modeling as a canonical task of multivariate time series
forecasting has been a significant research topic in AI community. To address
the underlying heterogeneity and non-stationarity implied in the graph streams,
in this study, we propose Spatio-Temporal Meta-Graph Learning as a novel Graph
Structure Learning mechanism on spatio-temporal data. Specifically, we
implement this idea into Meta-Graph Convolutional Recurrent Network (MegaCRN)
by plugging the Meta-Graph Learner powered by a Meta-Node Bank into GCRN
encoder-decoder. We conduct a comprehensive evaluation on two benchmark
datasets (METR-LA and PEMS-BAY) and a large-scale spatio-temporal dataset that
contains a variaty of non-stationary phenomena. Our model outperformed the
state-of-the-arts to a large degree on all three datasets (over 27% MAE and 34%
RMSE). Besides, through a series of qualitative evaluations, we demonstrate
that our model can explicitly disentangle locations and time slots with
different patterns and be robustly adaptive to different anomalous situations.
Codes and datasets are available at https://github.com/deepkashiwa20/MegaCRN.Comment: Preprint submitted to Artificial Intelligence. arXiv admin note:
substantial text overlap with arXiv:2211.1470