Graph-embedding Enhanced Attention Adversarial Autoencoder

When dealing with the graph data in real problems, only part of the nodes in the graph are labeled and the rest are not. A core problem is how to use this information to extend the labeling so that all nodes are assigned a label (or labels). Intuitively we can learn the patterns (or extract some representations) from those labeled nodes and then apply the patterns to determine the membership for those unknown nodes. A majority of previous related studies focus on extracting the local information representations and may suffer from lack of additional constraints which are necessary for improving the robustness of representation. In this work, we presented Graph- embedding enhanced attention Adversarial Autoencoder Networks (Great AAN), a new scalable generalized framework for graph-structured data representation learning and node classification. In our framework, we firstly introduce the attention layers and provide insights on the self-attention mechanism with multi-heads. Moreover, the shortest path length between nodes is incorporated into the self-attention mechanism to enhance the embedding of the node’s structural spatial information. Then a generative adversarial autoencoder is proposed to encode both global and local information and enhance the robustness of the embedded data distribution. Due to the scalability of our approach, it has efficient and various applications, including node classification, a recommendation system, and graph link prediction. We applied this Great AAN on multiple datasets (including PPI, Cora, Citeseer, Pubmed and Alipay) from social science and biomedical science. The experimental results demonstrated that our new framework significantly outperforms several popular methods

Knowledge-Enhanced Top-K Recommendation in Poincar\'e Ball

Personalized recommender systems are increasingly important as more content and services become available and users struggle to identify what might interest them. Thanks to the ability for providing rich information, knowledge graphs (KGs) are being incorporated to enhance the recommendation performance and interpretability. To effectively make use of the knowledge graph, we propose a recommendation model in the hyperbolic space, which facilitates the learning of the hierarchical structure of knowledge graphs. Furthermore, a hyperbolic attention network is employed to determine the relative importances of neighboring entities of a certain item. In addition, we propose an adaptive and fine-grained regularization mechanism to adaptively regularize items and their neighboring representations. Via a comparison using three real-world datasets with state-of-the-art methods, we show that the proposed model outperforms the best existing models by 2-16% in terms of NDCG@K on Top-K recommendation.Comment: Accepted by the 35th AAAI Conference on Artificial Intelligence (AAAI 2021

Receptive fields optimization in deep learning for enhanced interpretability, diversity, and resource efficiency.

In both supervised and unsupervised learning settings, deep neural networks (DNNs) are known to perform hierarchical and discriminative representation of data. They are capable of automatically extracting excellent hierarchy of features from raw data without the need for manual feature engineering. Over the past few years, the general trend has been that DNNs have grown deeper and larger, amounting to huge number of final parameters and highly nonlinear cascade of features, thus improving the flexibility and accuracy of resulting models. In order to account for the scale, diversity and the difficulty of data DNNs learn from, the architectural complexity and the excessive number of weights are often deliberately built in into their design. This flexibility and performance usually come with high computational and memory demands both during training and inference. In addition, insight into the mappings DNN models perform and human ability to understand them still remain very limited. This dissertation addresses some of these limitations by balancing three conflicting objectives: computational/ memory demands, interpretability, and accuracy. This dissertation first introduces some unsupervised feature learning methods in a broader context of dictionary learning. It also sets the tone for deep autoencoder learning and constraints for data representations in light of removing some of the aforementioned bottlenecks such as the feature interpretability of deep learning models with nonnegativity constraints on receptive fields. In addition, the two main classes of solution to the drawbacks associated with overparameterization/ over-complete representation in deep learning models are also presented. Subsequently, two novel methods, one for each solution class, are presented to address the problems resulting from over-complete representation exhibited by most deep learning models. The first method is developed to achieve inference-cost-efficient models via elimination of redundant features with negligible deterioration of prediction accuracy. This is important especially for deploying deep learning models into resource-limited portable devices. The second method aims at diversifying the features of DNNs in the learning phase to improve their performance without undermining their size and capacity. Lastly, feature diversification is considered to stabilize adversarial learning and extensive experimental outcomes show that these methods have the potential of advancing the current state-of-the-art on different learning tasks and benchmark datasets

Network Representation Learning: From Traditional Feature Learning to Deep Learning

Network representation learning (NRL) is an effective graph analytics technique and promotes users to deeply understand the hidden characteristics of graph data. It has been successfully applied in many real-world tasks related to network science, such as social network data processing, biological information processing, and recommender systems. Deep Learning is a powerful tool to learn data features. However, it is non-trivial to generalize deep learning to graph-structured data since it is different from the regular data such as pictures having spatial information and sounds having temporal information. Recently, researchers proposed many deep learning-based methods in the area of NRL. In this survey, we investigate classical NRL from traditional feature learning method to the deep learning-based model, analyze relationships between them, and summarize the latest progress. Finally, we discuss open issues considering NRL and point out the future directions in this field