2 research outputs found
Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
Since the invention of word2vec, the skip-gram model has significantly
advanced the research of network embedding, such as the recent emergence of the
DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of
the aforementioned models with negative sampling can be unified into the matrix
factorization framework with closed forms. Our analysis and proofs reveal that:
(1) DeepWalk empirically produces a low-rank transformation of a network's
normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk
when the size of vertices' context is set to one; (3) As an extension of LINE,
PTE can be viewed as the joint factorization of multiple networks' Laplacians;
(4) node2vec is factorizing a matrix related to the stationary distribution and
transition probability tensor of a 2nd-order random walk. We further provide
the theoretical connections between skip-gram based network embedding
algorithms and the theory of graph Laplacian. Finally, we present the NetMF
method as well as its approximation algorithm for computing network embedding.
Our method offers significant improvements over DeepWalk and LINE for
conventional network mining tasks. This work lays the theoretical foundation
for skip-gram based network embedding methods, leading to a better
understanding of latent network representation learning.Comment: 9 pages, published in WSDM 2018 proceeding
Deep Representation Learning for Social Network Analysis
Social network analysis is an important problem in data mining. A fundamental
step for analyzing social networks is to encode network data into
low-dimensional representations, i.e., network embeddings, so that the network
topology structure and other attribute information can be effectively
preserved. Network representation leaning facilitates further applications such
as classification, link prediction, anomaly detection and clustering. In
addition, techniques based on deep neural networks have attracted great
interests over the past a few years. In this survey, we conduct a comprehensive
review of current literature in network representation learning utilizing
neural network models. First, we introduce the basic models for learning node
representations in homogeneous networks. Meanwhile, we will also introduce some
extensions of the base models in tackling more complex scenarios, such as
analyzing attributed networks, heterogeneous networks and dynamic networks.
Then, we introduce the techniques for embedding subgraphs. After that, we
present the applications of network representation learning. At the end, we
discuss some promising research directions for future work