45,481 research outputs found
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
There has been a surge of recent interest in learning representations for
graph-structured data. Graph representation learning methods have generally
fallen into three main categories, based on the availability of labeled data.
The first, network embedding (such as shallow graph embedding or graph
auto-encoders), focuses on learning unsupervised representations of relational
structure. The second, graph regularized neural networks, leverages graphs to
augment neural network losses with a regularization objective for
semi-supervised learning. The third, graph neural networks, aims to learn
differentiable functions over discrete topologies with arbitrary structure.
However, despite the popularity of these areas there has been surprisingly
little work on unifying the three paradigms. Here, we aim to bridge the gap
between graph neural networks, network embedding and graph regularization
models. We propose a comprehensive taxonomy of representation learning methods
for graph-structured data, aiming to unify several disparate bodies of work.
Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which
generalizes popular algorithms for semi-supervised learning on graphs (e.g.
GraphSage, Graph Convolutional Networks, Graph Attention Networks), and
unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc)
into a single consistent approach. To illustrate the generality of this
approach, we fit over thirty existing methods into this framework. We believe
that this unifying view both provides a solid foundation for understanding the
intuition behind these methods, and enables future research in the area
Machine Learning for Survival Analysis: A Survey
Accurately predicting the time of occurrence of an event of interest is a
critical problem in longitudinal data analysis. One of the main challenges in
this context is the presence of instances whose event outcomes become
unobservable after a certain time point or when some instances do not
experience any event during the monitoring period. Such a phenomenon is called
censoring which can be effectively handled using survival analysis techniques.
Traditionally, statistical approaches have been widely developed in the
literature to overcome this censoring issue. In addition, many machine learning
algorithms are adapted to effectively handle survival data and tackle other
challenging problems that arise in real-world data. In this survey, we provide
a comprehensive and structured review of the representative statistical methods
along with the machine learning techniques used in survival analysis and
provide a detailed taxonomy of the existing methods. We also discuss several
topics that are closely related to survival analysis and illustrate several
successful applications in various real-world application domains. We hope that
this paper will provide a more thorough understanding of the recent advances in
survival analysis and offer some guidelines on applying these approaches to
solve new problems that arise in applications with censored data
Discovering time-varying aeroelastic models of a long-span suspension bridge from field measurements by sparse identification of nonlinear dynamical systems
We develop data-driven dynamical models of the nonlinear aeroelastic effects
on a long-span suspension bridge from sparse, noisy sensor measurements which
monitor the bridge. Using the {\em sparse identification of nonlinear dynamics}
(SINDy) algorithm, we are able to identify parsimonious, time-varying dynamical
systems that capture vortex-induced vibration (VIV) events in the bridge. Thus
we are able to posit new, data-driven models highlighting the aeroelastic
interaction of the bridge structure with VIV events. The bridge dynamics are
shown to have distinct, time-dependent modes of behavior, thus requiring
parametric models to account for the diversity of dynamics. Our method
generates hitherto unknown bridge-wind interaction models that go beyond
current theoretical and computational descriptions. Our proposed method for
real-time monitoring and model discovery allow us to move our model predictions
beyond lab theory to practical engineering design, which has the potential to
assess bad engineering configurations that are susceptible to deleterious
bridge-wind interactions. With the rise of real-time sensor networks on major
bridges, our model discovery methods can enhance an engineers ability to assess
the nonlinear aeroelastic interactions of the bridge with its wind environment.Comment: 14 pages, 12 figure
Using Embeddings to Correct for Unobserved Confounding in Networks
We consider causal inference in the presence of unobserved confounding. We
study the case where a proxy is available for the unobserved confounding in the
form of a network connecting the units. For example, the link structure of a
social network carries information about its members. We show how to
effectively use the proxy to do causal inference. The main idea is to reduce
the causal estimation problem to a semi-supervised prediction of both the
treatments and outcomes. Networks admit high-quality embedding models that can
be used for this semi-supervised prediction. We show that the method yields
valid inferences under suitable (weak) conditions on the quality of the
predictive model. We validate the method with experiments on a semi-synthetic
social network dataset. Code is available at
github.com/vveitch/causal-network-embeddings.Comment: An earlier version also addressed the use of text embeddings. That
material has been expanded and moved to arxiv:1905.12741, "Using Text
Embeddings for Causal Inference
Joint Modeling of Event Sequence and Time Series with Attentional Twin Recurrent Neural Networks
A variety of real-world processes (over networks) produce sequences of data
whose complex temporal dynamics need to be studied. More especially, the event
timestamps can carry important information about the underlying network
dynamics, which otherwise are not available from the time-series evenly sampled
from continuous signals. Moreover, in most complex processes, event sequences
and evenly-sampled times series data can interact with each other, which
renders joint modeling of those two sources of data necessary. To tackle the
above problems, in this paper, we utilize the rich framework of (temporal)
point processes to model event data and timely update its intensity function by
the synergic twin Recurrent Neural Networks (RNNs). In the proposed
architecture, the intensity function is synergistically modulated by one RNN
with asynchronous events as input and another RNN with time series as input.
Furthermore, to enhance the interpretability of the model, the attention
mechanism for the neural point process is introduced. The whole model with
event type and timestamp prediction output layers can be trained end-to-end and
allows a black-box treatment for modeling the intensity. We substantiate the
superiority of our model in synthetic data and three real-world benchmark
datasets.Comment: 14 page
Topological based classification of paper domains using graph convolutional networks
The main approaches for node classification in graphs are information
propagation and the association of the class of the node with external
information. State of the art methods merge these approaches through Graph
Convolutional Networks. We here use the association of topological features of
the nodes with their class to predict this class. Moreover, combining
topological information with information propagation improves classification
accuracy on the standard CiteSeer and Cora paper classification task.
Topological features and information propagation produce results almost as good
as text-based classification, without no textual or content information. We
propose to represent the topology and information propagation through a GCN
with the neighboring training node classification as an input and the current
node classification as output. Such a formalism outperforms state of the art
methods
Continuous-Time Relationship Prediction in Dynamic Heterogeneous Information Networks
Online social networks, World Wide Web, media and technological networks, and
other types of so-called information networks are ubiquitous nowadays. These
information networks are inherently heterogeneous and dynamic. They are
heterogeneous as they consist of multi-typed objects and relations, and they
are dynamic as they are constantly evolving over time. One of the challenging
issues in such heterogeneous and dynamic environments is to forecast those
relationships in the network that will appear in the future. In this paper, we
try to solve the problem of continuous-time relationship prediction in dynamic
and heterogeneous information networks. This implies predicting the time it
takes for a relationship to appear in the future, given its features that have
been extracted by considering both heterogeneity and temporal dynamics of the
underlying network. To this end, we first introduce a feature extraction
framework that combines the power of meta-path-based modeling and recurrent
neural networks to effectively extract features suitable for relationship
prediction regarding heterogeneity and dynamicity of the networks. Next, we
propose a supervised non-parametric approach, called Non-Parametric Generalized
Linear Model (NP-GLM), which infers the hidden underlying probability
distribution of the relationship building time given its features. We then
present a learning algorithm to train NP-GLM and an inference method to answer
time-related queries. Extensive experiments conducted on synthetic data and
three real-world datasets, namely Delicious, MovieLens, and DBLP, demonstrate
the effectiveness of NP-GLM in solving continuous-time relationship prediction
problem vis-a-vis competitive baselinesComment: To appear in ACM Transactions on Knowledge Discovery from Dat
An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning
We present a novel hybrid algorithm for Bayesian network structure learning,
called Hybrid HPC (H2PC). It first reconstructs the skeleton of a Bayesian
network and then performs a Bayesian-scoring greedy hill-climbing search to
orient the edges. It is based on a subroutine called HPC, that combines ideas
from incremental and divide-and-conquer constraint-based methods to learn the
parents and children of a target variable. We conduct an experimental
comparison of H2PC against Max-Min Hill-Climbing (MMHC), which is currently the
most powerful state-of-the-art algorithm for Bayesian network structure
learning, on several benchmarks with various data sizes. Our extensive
experiments show that H2PC outperforms MMHC both in terms of goodness of fit to
new data and in terms of the quality of the network structure itself, which is
closer to the true dependence structure of the data. The source code (in R) of
H2PC as well as all data sets used for the empirical tests are publicly
available.Comment: arXiv admin note: text overlap with arXiv:1101.5184 by other authors.
Lecture notes in computer science, springer, 2012, Machine Learning and
Knowledge Discovery in Databases, 7523, pp.58-7
Machine Learning for the Geosciences: Challenges and Opportunities
Geosciences is a field of great societal relevance that requires solutions to
several urgent problems facing our humanity and the planet. As geosciences
enters the era of big data, machine learning (ML) -- that has been widely
successful in commercial domains -- offers immense potential to contribute to
problems in geosciences. However, problems in geosciences have several unique
challenges that are seldom found in traditional applications, requiring novel
problem formulations and methodologies in machine learning. This article
introduces researchers in the machine learning (ML) community to these
challenges offered by geoscience problems and the opportunities that exist for
advancing both machine learning and geosciences. We first highlight typical
sources of geoscience data and describe their properties that make it
challenging to use traditional machine learning techniques. We then describe
some of the common categories of geoscience problems where machine learning can
play a role, and discuss some of the existing efforts and promising directions
for methodological development in machine learning. We conclude by discussing
some of the emerging research themes in machine learning that are applicable
across all problems in the geosciences, and the importance of a deep
collaboration between machine learning and geosciences for synergistic
advancements in both disciplines.Comment: Under review at IEEE Transactions on Knowledge and Data Engineerin
Modeling Event Propagation via Graph Biased Temporal Point Process
Temporal point process is widely used for sequential data modeling. In this
paper, we focus on the problem of modeling sequential event propagation in
graph, such as retweeting by social network users, news transmitting between
websites, etc. Given a collection of event propagation sequences, conventional
point process model consider only the event history, i.e. embed event history
into a vector, not the latent graph structure. We propose a Graph Biased
Temporal Point Process (GBTPP) leveraging the structural information from graph
representation learning, where the direct influence between nodes and indirect
influence from event history is modeled respectively. Moreover, the learned
node embedding vector is also integrated into the embedded event history as
side information. Experiments on a synthetic dataset and two real-world
datasets show the efficacy of our model compared to conventional methods and
state-of-the-art.Comment: 9 pages, 6 figures, 2 table
- …