39 research outputs found
Graph Embedding Techniques, Applications, and Performance: A Survey
Graphs, such as social networks, word co-occurrence networks, and
communication networks, occur naturally in various real-world applications.
Analyzing them yields insight into the structure of society, language, and
different patterns of communication. Many approaches have been proposed to
perform the analysis. Recently, methods which use the representation of graph
nodes in vector space have gained traction from the research community. In this
survey, we provide a comprehensive and structured analysis of various graph
embedding techniques proposed in the literature. We first introduce the
embedding task and its challenges such as scalability, choice of
dimensionality, and features to be preserved, and their possible solutions. We
then present three categories of approaches based on factorization methods,
random walks, and deep learning, with examples of representative algorithms in
each category and analysis of their performance on various tasks. We evaluate
these state-of-the-art methods on a few common datasets and compare their
performance against one another. Our analysis concludes by suggesting some
potential applications and future directions. We finally present the
open-source Python library we developed, named GEM (Graph Embedding Methods,
available at https://github.com/palash1992/GEM), which provides all presented
algorithms within a unified interface to foster and facilitate research on the
topic.Comment: Submitted to Knowledge Based Systems for revie
Deep Neural Networks for Optimal Team Composition
Cooperation is a fundamental social mechanism, whose effects on human
performance have been investigated in several environments. Online games are
modern-days natural settings in which cooperation strongly affects human
behavior. Every day, millions of players connect and play together in
team-based games: the patterns of cooperation can either foster or hinder
individual skill learning and performance. This work has three goals: (i)
identifying teammates' influence on players' performance in the short and long
term, (ii) designing a computational framework to recommend teammates to
improve players' performance, and (iii) setting to demonstrate that such
improvements can be predicted via deep learning. We leverage a large dataset
from Dota 2, a popular Multiplayer Online Battle Arena game. We generate a
directed co-play network, whose links' weights depict the effect of teammates
on players' performance. Specifically, we propose a measure of network
influence that captures skill transfer from player to player over time. We then
use such framing to design a recommendation system to suggest new teammates
based on a modified deep neural autoencoder and we demonstrate its
state-of-the-art recommendation performance. We finally provide insights into
skill transfer effects: our experimental results demonstrate that such dynamics
can be predicted using deep neural networks
dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning
Learning graph representations is a fundamental task aimed at capturing
various properties of graphs in vector space. The most recent methods learn
such representations for static networks. However, real world networks evolve
over time and have varying dynamics. Capturing such evolution is key to
predicting the properties of unseen networks. To understand how the network
dynamics affect the prediction performance, we propose an embedding approach
which learns the structure of evolution in dynamic graphs and can predict
unseen links with higher precision. Our model, dyngraph2vec, learns the
temporal transitions in the network using a deep architecture composed of dense
and recurrent layers. We motivate the need of capturing dynamics for prediction
on a toy data set created using stochastic block models. We then demonstrate
the efficacy of dyngraph2vec over existing state-of-the-art methods on two real
world data sets. We observe that learning dynamics can improve the quality of
embedding and yield better performance in link prediction
Capturing Edge Attributes via Network Embedding
Network embedding, which aims to learn low-dimensional representations of
nodes, has been used for various graph related tasks including visualization,
link prediction and node classification. Most existing embedding methods rely
solely on network structure. However, in practice we often have auxiliary
information about the nodes and/or their interactions, e.g., content of
scientific papers in co-authorship networks, or topics of communication in
Twitter mention networks. Here we propose a novel embedding method that uses
both network structure and edge attributes to learn better network
representations. Our method jointly minimizes the reconstruction error for
higher-order node neighborhood, social roles and edge attributes using a deep
architecture that can adequately capture highly non-linear interactions. We
demonstrate the efficacy of our model over existing state-of-the-art methods on
a variety of real-world networks including collaboration networks, and social
networks. We also observe that using edge attributes to inform network
embedding yields better performance in downstream tasks such as link prediction
and node classification
Hierarchical Class-Based Curriculum Loss
Classification algorithms in machine learning often assume a flat label
space. However, most real world data have dependencies between the labels,
which can often be captured by using a hierarchy. Utilizing this relation can
help develop a model capable of satisfying the dependencies and improving model
accuracy and interpretability. Further, as different levels in the hierarchy
correspond to different granularities, penalizing each label equally can be
detrimental to model learning. In this paper, we propose a loss function,
hierarchical curriculum loss, with two properties: (i) satisfy hierarchical
constraints present in the label space, and (ii) provide non-uniform weights to
labels based on their levels in the hierarchy, learned implicitly by the
training paradigm. We theoretically show that the proposed loss function is a
tighter bound of 0-1 loss compared to any other loss satisfying the
hierarchical constraints. We test our loss function on real world image data
sets, and show that it significantly substantially outperforms multiple
baselines
DynamicGEM: A Library for Dynamic Graph Embedding Methods
DynamicGEM is an open-source Python library for learning node representations
of dynamic graphs. It consists of state-of-the-art algorithms for defining
embeddings of nodes whose connections evolve over time. The library also
contains the evaluation framework for four downstream tasks on the network:
graph reconstruction, static and temporal link prediction, node classification,
and temporal visualization. We have implemented various metrics to evaluate the
state-of-the-art methods, and examples of evolving networks from various
domains. We have easy-to-use functions to call and evaluate the methods and
have extensive usage documentation. Furthermore, DynamicGEM provides a template
to add new algorithms with ease to facilitate further research on the topic
Tracking Temporal Evolution of Graphs using Non-Timestamped Data
Datasets to study the temporal evolution of graphs are scarce. To encourage
the research of novel dynamic graph learning algorithms we introduce
YoutubeGraph-Dyn (available at https://github.com/palash1992/YoutubeGraph-Dyn),
an evolving graph dataset generated from YouTube real-world interactions.
YoutubeGraph-Dyn provides intra-day time granularity (with 416 snapshots taken
every 6 hours for a period of 104 days), multi-modal relationships that capture
different aspects of the data, multiple attributes including timestamped,
non-timestamped, word embeddings, and integers. Our data collection methodology
emphasizes the creation of time evolving graphs from non-timestamped data. In
this paper, we provide various graph statistics of YoutubeGraph-Dyn and test
state-of-the-art graph clustering algorithms to detect community migration, and
time series analysis and recurrent neural network algorithms to forecast
non-timestamped data
Cross-modal Learning for Multi-modal Video Categorization
Multi-modal machine learning (ML) models can process data in multiple
modalities (e.g., video, audio, text) and are useful for video content analysis
in a variety of problems (e.g., object detection, scene understanding, activity
recognition). In this paper, we focus on the problem of video categorization
using a multi-modal ML technique. In particular, we have developed a novel
multi-modal ML approach that we call "cross-modal learning", where one modality
influences another but only when there is correlation between the modalities --
for that, we first train a correlation tower that guides the main multi-modal
video categorization tower in the model. We show how this cross-modal principle
can be applied to different types of models (e.g., RNN, Transformer, NetVLAD),
and demonstrate through experiments how our proposed multi-modal video
categorization models with cross-modal learning out-perform strong
state-of-the-art baseline models
DynGEM: Deep Embedding Method for Dynamic Graphs
Embedding large graphs in low dimensional spaces has recently attracted
significant interest due to its wide applications such as graph visualization,
link prediction and node classification. Existing methods focus on computing
the embedding for static graphs. However, many graphs in practical applications
are dynamic and evolve constantly over time. Naively applying existing
embedding algorithms to each snapshot of dynamic graphs independently usually
leads to unsatisfactory performance in terms of stability, flexibility and
efficiency. In this work, we present an efficient algorithm DynGEM based on
recent advances in deep autoencoders for graph embeddings, to address this
problem. The major advantages of DynGEM include: (1) the embedding is stable
over time, (2) it can handle growing dynamic graphs, and (3) it has better
running time than using static embedding methods on each snapshot of a dynamic
graph. We test DynGEM on a variety of tasks including graph visualization,
graph reconstruction, link prediction and anomaly detection (on both synthetic
and real datasets). Experimental results demonstrate the superior stability and
scalability of our approach.Comment: 3rd International Workshop on Representation Learning for Graphs
(ReLiG), IJCAI 201
Exploiting Temporal Coherence for Multi-modal Video Categorization
Multimodal ML models can process data in multiple modalities (e.g., video,
images, audio, text) and are useful for video content analysis in a variety of
problems (e.g., object detection, scene understanding). In this paper, we focus
on the problem of video categorization by using a multimodal approach. We have
developed a novel temporal coherence-based regularization approach, which
applies to different types of models (e.g., RNN, NetVLAD, Transformer). We
demonstrate through experiments how our proposed multimodal video
categorization models with temporal coherence out-perform strong
state-of-the-art baseline models