2,105 research outputs found
Regression and Singular Value Decomposition in Dynamic Graphs
Most of real-world graphs are {\em dynamic}, i.e., they change over time.
However, while problems such as regression and Singular Value Decomposition
(SVD) have been studied for {\em static} graphs, they have not been
investigated for {\em dynamic} graphs, yet. In this paper, we introduce,
motivate and study regression and SVD over dynamic graphs. First, we present
the notion of {\em update-efficient matrix embedding} that defines the
conditions sufficient for a matrix embedding to be used for the dynamic graph
regression problem (under norm). We prove that given an
update-efficient matrix embedding (e.g., adjacency matrix), after an update
operation in the graph, the optimal solution of the graph regression problem
for the revised graph can be computed in time. We also study dynamic
graph regression under least absolute deviation. Then, we characterize a class
of matrix embeddings that can be used to efficiently update SVD of a dynamic
graph. For adjacency matrix and Laplacian matrix, we study those graph update
operations for which SVD (and low rank approximation) can be updated
efficiently
Adversarial Permutation Guided Node Representations for Link Prediction
After observing a snapshot of a social network, a link prediction (LP)
algorithm identifies node pairs between which new edges will likely materialize
in future. Most LP algorithms estimate a score for currently non-neighboring
node pairs, and rank them by this score. Recent LP systems compute this score
by comparing dense, low dimensional vector representations of nodes. Graph
neural networks (GNNs), in particular graph convolutional networks (GCNs), are
popular examples. For two nodes to be meaningfully compared, their embeddings
should be indifferent to reordering of their neighbors. GNNs typically use
simple, symmetric set aggregators to ensure this property, but this design
decision has been shown to produce representations with limited expressive
power. Sequence encoders are more expressive, but are permutation sensitive by
design. Recent efforts to overcome this dilemma turn out to be unsatisfactory
for LP tasks. In response, we propose PermGNN, which aggregates neighbor
features using a recurrent, order-sensitive aggregator and directly minimizes
an LP loss while it is `attacked' by adversarial generator of neighbor
permutations. By design, PermGNN{} has more expressive power compared to
earlier symmetric aggregators. Next, we devise an optimization framework to map
PermGNN's node embeddings to a suitable locality-sensitive hash, which speeds
up reporting the top- most likely edges for the LP task. Our experiments on
diverse datasets show that \our outperforms several state-of-the-art link
predictors by a significant margin, and can predict the most likely edges fast.Comment: Rectified an error in evaluation in earlier 60-40 split
Recommended from our members
Unsupervised Representation Learning with Correlations
Unsupervised representation learning algorithms have been playing important roles in machine learning and related fields. However, due to optimization intractability or lack of consideration in given data correlation structures, some unsupervised representation learning algorithms still cannot well discover the inherent features from the data, under certain circumstances. This thesis extends these algorithms, and improves over the above issues by taking data correlations into consideration.
We study three different aspects of improvements on unsupervised representation learning algorithms by utilizing correlation information, via the following three tasks respectively:
1. Using estimated correlations between data points to provide smart optimization initializations, for multi-way matching (Chapter 2). In this work, we define a correlation score between pairs of data points as metrics for correlations, and initialize all the permutation matrices along a maximum spanning tree of the undirected graph with these metrics as the weights.
2. Faster optimization by utilizing the correlations in the observations, for variational inference (Chapter 3). We construct a positive definite matrix from the negative Hessian of the log-likelihood part of the objective that can capture the influence of the observation correlations on the parameter vector. We then use the inverse of this matrix to rescale the gradient.
3. Utilizing additional side-information on data correlation structures to explicitly learn correlations between data points, for extensions of Variational Auto-Encoders (VAEs) (Chapters 4 and 5). Consider the case where we know a correlation graph G of the data points. Instead of placing an i.i.d. prior as in the most common setting, we adopt correlated priors and/or correlated variational distributions on the latent variables through utilizing the graph G.
Empirical results on these tasks show the success of the proposed methods in improving the performances of unsupervised representation learning algorithms. We compare our methods with multiple recent advanced algorithms on various tasks, on both synthetic and real datasets. We also provide theoretical analysis for some of the proposed methods, showing their advantages under certain situations.
The proposed methods have wide ranges of applications. For examples, image compression (via smart initializations for multi-way matching), link prediction (by VAEs with correlations), etc
- …