15,998 research outputs found
Continuous-Time Relationship Prediction in Dynamic Heterogeneous Information Networks
Online social networks, World Wide Web, media and technological networks, and
other types of so-called information networks are ubiquitous nowadays. These
information networks are inherently heterogeneous and dynamic. They are
heterogeneous as they consist of multi-typed objects and relations, and they
are dynamic as they are constantly evolving over time. One of the challenging
issues in such heterogeneous and dynamic environments is to forecast those
relationships in the network that will appear in the future. In this paper, we
try to solve the problem of continuous-time relationship prediction in dynamic
and heterogeneous information networks. This implies predicting the time it
takes for a relationship to appear in the future, given its features that have
been extracted by considering both heterogeneity and temporal dynamics of the
underlying network. To this end, we first introduce a feature extraction
framework that combines the power of meta-path-based modeling and recurrent
neural networks to effectively extract features suitable for relationship
prediction regarding heterogeneity and dynamicity of the networks. Next, we
propose a supervised non-parametric approach, called Non-Parametric Generalized
Linear Model (NP-GLM), which infers the hidden underlying probability
distribution of the relationship building time given its features. We then
present a learning algorithm to train NP-GLM and an inference method to answer
time-related queries. Extensive experiments conducted on synthetic data and
three real-world datasets, namely Delicious, MovieLens, and DBLP, demonstrate
the effectiveness of NP-GLM in solving continuous-time relationship prediction
problem vis-a-vis competitive baselinesComment: To appear in ACM Transactions on Knowledge Discovery from Dat
A Comprehensive Survey on Graph Neural Networks
Deep learning has revolutionized many machine learning tasks in recent years,
ranging from image classification and video processing to speech recognition
and natural language understanding. The data in these tasks are typically
represented in the Euclidean space. However, there is an increasing number of
applications where data are generated from non-Euclidean domains and are
represented as graphs with complex relationships and interdependency between
objects. The complexity of graph data has imposed significant challenges on
existing machine learning algorithms. Recently, many studies on extending deep
learning approaches for graph data have emerged. In this survey, we provide a
comprehensive overview of graph neural networks (GNNs) in data mining and
machine learning fields. We propose a new taxonomy to divide the
state-of-the-art graph neural networks into four categories, namely recurrent
graph neural networks, convolutional graph neural networks, graph autoencoders,
and spatial-temporal graph neural networks. We further discuss the applications
of graph neural networks across various domains and summarize the open source
codes, benchmark data sets, and model evaluation of graph neural networks.
Finally, we propose potential research directions in this rapidly growing
field.Comment: Minor revision (updated tables and references
T-EDGE: Temporal WEighted MultiDiGraph Embedding for Ethereum Transaction Network Analysis
Recently, graph embedding techniques have been widely used in the analysis of
various networks, but most of the existing embedding methods omit the network
dynamics and the multiplicity of edges, so it is difficult to accurately
describe the detailed characteristics of the transaction networks. Ethereum is
a blockchain-based platform supporting smart contracts. The open nature of
blockchain makes the transaction data on Ethereum completely public, and also
brings unprecedented opportunities for the transaction network analysis. By
taking the realistic rules and features of transaction networks into
consideration, we first model the Ethereum transaction network as a Temporal
Weighted Multidigraph (TWMDG), where each node is a unique Ethereum account and
each edge represents a transaction weighted by amount and assigned with
timestamp. Then we define the problem of Temporal Weighted Multidigraph
Embedding (T-EDGE) by incorporating both temporal and weighted information of
the edges, the purpose being to capture more comprehensive properties of
dynamic transaction networks. To evaluate the effectiveness of the proposed
embedding method, we conduct experiments of node classification on real-world
transaction data collected from Ethereum. Experimental results demonstrate that
T-EDGE outperforms baseline embedding methods, indicating that time-dependent
walks and multiplicity characteristic of edges are informative and essential
for time-sensitive transaction networks.Comment: 12 page
DyLink2Vec: Effective Feature Representation for Link Prediction in Dynamic Networks
The temporal dynamics of a complex system such as a social network or a
communication network can be studied by understanding the patterns of link
appearance and disappearance over time. A critical task along this
understanding is to predict the link state of the network at a future time
given a collection of link states at earlier time points. In existing
literature, this task is known as link prediction in dynamic networks. Solving
this task is more difficult than its counterpart in static networks because an
effective feature representation of node-pair instances for the case of dynamic
network is hard to obtain. To overcome this problem, we propose a novel method
for metric embedding of node-pair instances of a dynamic network. The proposed
method models the metric embedding task as an optimal coding problem where the
objective is to minimize the reconstruction error, and it solves this
optimization task using a gradient descent method. We validate the
effectiveness of the learned feature representation by utilizing it for link
prediction in various real-life dynamic networks. Specifically, we show that
our proposed link prediction model, which uses the extracted feature
representation for the training instances, outperforms several existing methods
that use well-known link prediction features
A Survey on Embedding Dynamic Graphs
Embedding static graphs in low-dimensional vector spaces plays a key role in
network analytics and inference, supporting applications like node
classification, link prediction, and graph visualization. However, many
real-world networks present dynamic behavior, including topological evolution,
feature evolution, and diffusion. Therefore, several methods for embedding
dynamic graphs have been proposed to learn network representations over time,
facing novel challenges, such as time-domain modeling, temporal features to be
captured, and the temporal granularity to be embedded. In this survey, we
overview dynamic graph embedding, discussing its fundamentals and the recent
advances developed so far. We introduce the formal definition of dynamic graph
embedding, focusing on the problem setting and introducing a novel taxonomy for
dynamic graph embedding input and output. We further explore different dynamic
behaviors that may be encompassed by embeddings, classifying by topological
evolution, feature evolution, and processes on networks. Afterward, we describe
existing techniques and propose a taxonomy for dynamic graph embedding
techniques based on algorithmic approaches, from matrix and tensor
factorization to deep learning, random walks, and temporal point processes. We
also elucidate main applications, including dynamic link prediction, anomaly
detection, and diffusion prediction, and we further state some promising
research directions in the area.Comment: 41 pages, 10 figure
Cognitive computation with autonomously active neural networks: an emerging field
The human brain is autonomously active. To understand the functional role of
this self-sustained neural activity, and its interplay with the sensory data
input stream, is an important question in cognitive system research and we
review here the present state of theoretical modelling.
This review will start with a brief overview of the experimental efforts,
together with a discussion of transient vs. self-sustained neural activity in
the framework of reservoir computing. The main emphasis will be then on two
paradigmal neural network architectures showing continuously ongoing
transient-state dynamics: saddle point networks and networks of attractor
relics.
Self-active neural networks are confronted with two seemingly contrasting
demands: a stable internal dynamical state and sensitivity to incoming stimuli.
We show, that this dilemma can be solved by networks of attractor relics based
on competitive neural dynamics, where the attractor relics compete on one side
with each other for transient dominance, and on the other side with the
dynamical influence of the input signals. Unsupervised and local Hebbian-style
online learning then allows the system to build up correlations between the
internal dynamical transient states and the sensory input stream. An emergent
cognitive capability results from this set-up. The system performs online, and
on its own, a non-linear independent component analysis of the sensory data
stream, all the time being continuously and autonomously active. This process
maps the independent components of the sensory input onto the attractor relics,
which acquire in this way a semantic meaning.Comment: keynote review. Cognitive Computation (in press, 2009
Learning Dynamic Embeddings from Temporal Interactions
Modeling a sequence of interactions between users and items (e.g., products,
posts, or courses) is crucial in domains such as e-commerce, social networking,
and education to predict future interactions. Representation learning presents
an attractive solution to model the dynamic evolution of user and item
properties, where each user/item can be embedded in a euclidean space and its
evolution can be modeled by dynamic changes in embedding. However, existing
embedding methods either generate static embeddings, treat users and items
independently, or are not scalable.
Here we present JODIE, a coupled recurrent model to jointly learn the dynamic
embeddings of users and items from a sequence of user-item interactions. JODIE
has three components. First, the update component updates the user and item
embedding from each interaction using their previous embeddings with the two
mutually-recursive Recurrent Neural Networks. Second, a novel projection
component is trained to forecast the embedding of users at any future time.
Finally, the prediction component directly predicts the embedding of the item
in a future interaction. For models that learn from a sequence of interactions,
traditional training data batching cannot be done due to complex user-user
dependencies. Therefore, we present a novel batching algorithm called t-Batch
that generates time-consistent batches of training data that can run in
parallel, giving massive speed-up.
We conduct six experiments on two prediction tasks---future interaction
prediction and state change prediction---using four real-world datasets. We
show that JODIE outperforms six state-of-the-art algorithms in these tasks by
up to 22.4%. Moreover, we show that JODIE is highly scalable and up to 9.2x
faster than comparable models. As an additional experiment, we illustrate that
JODIE can predict student drop-out from courses five interactions in advance
Knowledge Graph Embeddings and Explainable AI
Knowledge graph embeddings are now a widely adopted approach to knowledge
representation in which entities and relationships are embedded in vector
spaces. In this chapter, we introduce the reader to the concept of knowledge
graph embeddings by explaining what they are, how they can be generated and how
they can be evaluated. We summarize the state-of-the-art in this field by
describing the approaches that have been introduced to represent knowledge in
the vector space. In relation to knowledge representation, we consider the
problem of explainability, and discuss models and methods for explaining
predictions obtained via knowledge graph embeddings.Comment: Federico Bianchi, Gaetano Rossiello, Luca Costabello, Matteo
Plamonari, Pasquale Minervini, Knowledge Graph Embeddings and Explainable AI.
In: Ilaria Tiddi, Freddy Lecue, Pascal Hitzler (eds.), Knowledge Graphs for
eXplainable AI -- Foundations, Applications and Challenges. Studies on the
Semantic Web, IOS Press, Amsterdam, 202
Predicting the evolution of complex networks via local information
Almost all real-world networks are subject to constant evolution, and plenty
of evolving networks have been investigated to uncover the underlying
mechanisms for a deeper understanding of the organization and development of
them. Compared with the rapid expansion of the empirical studies about
evolution mechanisms exploration, the future links prediction methods
corresponding to the evolution mechanisms are deficient. Real-world information
always contain hints of what would happen next, which is also the case in the
observed evolving networks. In this paper, we firstly propose a
structured-dependent index to strengthen the robustness of link prediction
methods. Then we treat the observed links and their timestamps in evolving
networks as known information. We envision evolving networks as dynamic systems
and model the evolutionary dynamics of nodes similarity. Based on the iterative
updating of nodes' network position, the potential trend of evolving networks
is uncovered, which improves the accuracy of future links prediction.
Experiments on various real-world networks show that the proposed index
performs better than baseline methods and the spatial-temporal position drift
model performs well in real-world evolving networks
Improving confidence while predicting trends in temporal disease networks
For highly sensitive real-world predictive analytic applications such as
healthcare and medicine, having good prediction accuracy alone is often not
enough. These kinds of applications require a decision making process which
uses uncertainty estimation as input whenever possible. Quality of uncertainty
estimation is a subject of over or under confident prediction, which is often
not addressed in many models. In this paper we show several extensions to the
Gaussian Conditional Random Fields model, which aim to provide higher quality
uncertainty estimation. These extensions are applied to the temporal disease
graph built from the State Inpatient Database (SID) of California, acquired
from the HCUP. Our experiments demonstrate benefits of using graph information
in modeling temporal disease properties as well as improvements in uncertainty
estimation provided by given extensions of the Gaussian Conditional Random
Fields method.Comment: Proceedings of the 4th Workshop on Data Mining for Medicine and
Healthcare, 2015 SIAM International Conference on Data Mining, Vancouver,
Canada, April 30 - May 02, 201
- …