Deep Generation of Heterogeneous Networks

Heterogeneous graphs are ubiquitous data structures that can inherently capture multi-type and multi-modal interactions between objects. In recent years, research on encoding heterogeneous graph into latent representations have enjoyed a rapid increase. However, its reverse process, namely how to construct heterogeneous graphs from underlying representations and distributions have not been well explored due to several challenges in 1) modeling the local heterogeneous semantic distribution; 2) preserving the graph-structured distributions over the local semantics; and 3) characterizing the global heterogeneous graph distributions. To address these challenges, we propose a novel framework for heterogeneous graph generation (HGEN) that jointly captures the semantic, structural, and global distributions of heterogeneous graphs. Specifically, we propose a heterogeneous walk generator that hierarchically generates meta-paths and their path instances. In addition, a novel heterogeneous graph assembler is developed that can sample and combine the generated meta-path instances (e.g., walks) into heterogeneous graphs in a stratified manner. Theoretical analysis on the preservation of heterogeneous graph patterns by the proposed generation process has been performed. Extensive experiments on multiple real-world and synthetic heterogeneous graph datasets demonstrate the effectiveness of the proposed HGEN in generating realistic heterogeneous graphs.Comment: 10 pages, accepted by 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 202

A Systematic Survey on Deep Generative Models for Graph Generation

Graphs are important data representations for describing objects and their relationships, which appear in a wide diversity of real-world scenarios. As one of a critical problem in this area, graph generation considers learning the distributions of given graphs and generating more novel graphs. Owing to its wide range of applications, generative models for graphs have a rich history, which, however, are traditionally hand-crafted and only capable of modeling a few statistical properties of graphs. Recent advances in deep generative models for graph generation is an important step towards improving the fidelity of generated graphs and paves the way for new kinds of applications. This article provides an extensive overview of the literature in the field of deep generative models for the graph generation. Firstly, the formal definition of deep generative models for the graph generation as well as preliminary knowledge is provided. Secondly, two taxonomies of deep generative models for unconditional, and conditional graph generation respectively are proposed; the existing works of each are compared and analyzed. After that, an overview of the evaluation metrics in this specific domain is provided. Finally, the applications that deep graph generation enables are summarized and five promising future research directions are highlighted

Graph Neural Networks for Natural Language Processing: A Survey

Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP). Although text inputs are typically represented as a sequence of tokens, there isa rich variety of NLP problems that can be best expressed with a graph structure. As a result, thereis a surge of interests in developing new deep learning techniques on graphs for a large numberof NLP tasks. In this survey, we present a comprehensive overview onGraph Neural Networks(GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, whichsystematically organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models. We further introducea large number of NLP applications that are exploiting the power of GNNs and summarize thecorresponding benchmark datasets, evaluation metrics, and open-source codes. Finally, we discussvarious outstanding challenges for making the full use of GNNs for NLP as well as future researchdirections. To the best of our knowledge, this is the first comprehensive overview of Graph NeuralNetworks for Natural Language Processing.Comment: 127 page

Network alignment on big networks

In the age of big data, multiple networks naturally appear in a variety of domains, such as social network analysis, bioinformatics, finance, infrastructure and so on. Network alignment, which aims to find the node correspondences across different networks, can integrate multiple networks from different sources into a world-view network. By mining such a world-view network, one may gain considerable insights that are invisible if mining different networks separably. Networks as one common data type, share the well-known 4Vs characteristics of big data, including variety, veracity, velocity and volume, each of which brings unique challenges to the big network alignment task. Specifically, the variety characteristic of big networks depicts the rich information associated with multiple networks. Many prior network alignment methods find the node correspondences merely based on network structures while inevitably ignoring the rich node and/or edge attributes of networks. In the meanwhile, conventional methods often assume the alignment consistency among the neighboring node pairs, which could be easily violated due to the disparity among various networks. Despite the emergence of the sites and tools that enable to link entities, there still exist the bottlenecks of collecting the networked data, such as the privacy issues in social networks. Thus, real-world networks are often noisy and incomplete with missing edges. However, it still remains a daunting task on how to deal with the incompleteness and analyze the robustness of network alignment owing to the veracity characteristic. The velocity of big networks indicates that real-world networks are often dynamically changing. The dynamics behind multiple networks may benefit network alignment from the temporal information of nodes and edges in addition to the static structural information of networks. Yet, how to design the dynamic alignment model still remains an open problem. Given the sheer volume of large-scale networks but relatively limited computational resources, the at least quadratic complexity of many prior network alignment methods is not scalable especially when aligning networks with a large number of nodes and edges. In this way, the efficiency issue has become a fundamental challenge of big network alignment. The theme of my Ph.D. research is to address the above challenges associated with the 4Vs characteristics and align big networks. Note that we consider volume as an overarching goal so we can align big networks efficiently. First (for variety), to leverage attribute information of networks, we develop a family of algorithms FINAL that optimize the alignment consistency in terms of network structures and attributes and achieve an up to 30% improvement in terms of the alignment accuracy over the comparison methods without attributes. We also develop a novel alignment method that displace node representations to be more comparable through non-rigid point set registration. Moreover, to address network disparity issue, we design an encoder-decoder architecture NetTrans that learns network transformation functions in a hierarchical manner. Besides, we design a relational graph convolutional network based model with an adaptive negative sampling strategy to strike a balance between alignment consistency and disparity. This developed method named NextAlign achieves an at least 3% performance improvement over the best competitor. Second (for veracity), we hypothesize that network alignment and network completion mutually benefit each other and develop an effective algorithm based on multiplicative update that outperforms baseline methods on incomplete networks. In addition, we provide a robustness analysis of network alignment against structural noise. Last (for velocity), we design a representation learning model on dynamic network of networks which can leverage temporal information underlying networks and is applied for dynamic network alignment task