GoGNN: Graph of Graphs Neural Network for Predicting Structured Entity Interactions

Entity interaction prediction is essential in many important applications such as chemistry, biology, material science, and medical science. The problem becomes quite challenging when each entity is represented by a complex structure, namely structured entity, because two types of graphs are involved: local graphs for structured entities and a global graph to capture the interactions between structured entities. We observe that existing works on structured entity interaction prediction cannot properly exploit the unique graph of graphs model. In this paper, we propose a Graph of Graphs Neural Network, namely GoGNN, which extracts the features in both structured entity graphs and the entity interaction graph in a hierarchical way. We also propose the dual-attention mechanism that enables the model to preserve the neighbor importance in both levels of graphs. Extensive experiments on real-world datasets show that GoGNN outperforms the state-of-the-art methods on two representative structured entity interaction prediction tasks: chemical-chemical interaction prediction and drug-drug interaction prediction. Our code is available at Github.Comment: Accepted by IJCAI 202

Memory efficient location recommendation through proximity-aware representation

Sequential location recommendation plays a huge role in modern life, which can enhance user experience, bring more profit to businesses and assist in government administration. Although methods for location recommendation have evolved significantly thanks to the development of recommendation systems, there is still limited utilization of geographic information, along with the ongoing challenge of addressing data sparsity. In response, we introduce a Proximity-aware based region representation for Sequential Recommendation (PASR for short), built upon the Self-Attention Network architecture. We tackle the sparsity issue through a novel loss function employing importance sampling, which emphasizes informative negative samples during optimization. Moreover, PASR enhances the integration of geographic information by employing a self-attention-based geography encoder to the hierarchical grid and proximity grid at each GPS point. To further leverage geographic information, we utilize the proximity-aware negative samplers to enhance the quality of negative samples. We conducted evaluations using three real-world Location-Based Social Networking (LBSN) datasets, demonstrating that PASR surpasses state-of-the-art sequential location recommendation method

Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss

Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevailing in semi-supervised top-K recommendations. The basic idea is to routinely conduct heuristic-based data augmentation and apply contrastive losses (e.g., InfoNCE) on the augmented views. Yet, some CF-tailored challenges make this adoption suboptimal, such as the issue of out-of-distribution, the risk of false negatives, and the nature of top-K evaluation. They necessitate the CL-based CF scheme to focus more on mining hard negatives and distinguishing false negatives from the vast unlabeled user-item interactions, for informative contrast signals. Worse still, there is limited understanding of contrastive loss in CF methods, especially w.r.t. its generalization ability. To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods. AdvInfoNCE adaptively explores and assigns hardness to each negative instance in an adversarial fashion and further utilizes a fine-grained hardness-aware ranking criterion to empower the recommender's generalization ability. Training CF models with AdvInfoNCE, we validate the effectiveness of AdvInfoNCE on both synthetic and real-world benchmark datasets, thus showing its generalization ability to mitigate out-of-distribution problems. Given the theoretical guarantees and empirical superiority of AdvInfoNCE over most contrastive loss functions, we advocate its adoption as a standard loss in recommender systems, particularly for the out-of-distribution tasks. Codes are available at https://github.com/LehengTHU/AdvInfoNCE.Comment: Accepted to NeurIPS 202

Effective graph representation learning for ranking-based recommendation

Ranking-based recommender systems are designed to generate a personalised ranking list of items for a given user to address the information overload problem. An effective and efficient ranking-based recommender system can benefit users by providing them with items of interest as well as service providers by increasing their exposure and profits. Since more and more users and providers of items have been increasingly interacting with online platforms, the underlying recommendation algorithms are facing more challenges. For example, traditional collaborative filtering-based recommender systems cannot generate effective recommendations to cold-start users due to the lack of sufficient interactions. In addition, although recommender systems can leverage deep learning-based techniques to enhance their effectiveness, they are not robust enough against variances in the models’ initialisations, which can degrade the users’ satisfaction. Furthermore, when incorporating these complex deep models, the training phases of recommender systems become less efficient, which might slower the online platforms from quickly capturing the users’ interests. Graph representation learning includes techniques that can leverage graph-structured data and generate latent representations for the nodes, graphs/sub-graphs and edges between nodes. Since the user-item interaction matrix is in fact a bipartite graph, we can use these graph-based techniques to leverage the interaction matrix and generate more effective node representations for the users and items. Therefore, this thesis aims to enhance the ranking-based recommendations by proposing novel recommender systems based on graph representation learning. In particular, this thesis uses heterogeneous graph representation learning, graph pre-training and graph contrastive learning to improve the effectiveness of ranking-based recommendations while alleviating the aforementioned cold-start problem as well as the low-robustness and low training-efficiency issues. To enhance the effectiveness of ranking-based recommendations and alleviate the cold-start problem, we propose to use the heterogeneous graph representation learning technique to encode the typical side information of the users and items, which are usually defined as the attributes of users and the descriptions of items. For example, a user-item interaction matrix, social relations are one of the most naturally available relations that can be used to enrich such an interaction matrix. Therefore, we choose the social relations among different types of side information to build the heterogeneous graph. We propose a novel recommender system, the Social-aware Gaussian Pre-trained model (SGP), which encodes the user social relations and interaction data using the heterogeneous graph representation learning technique. Next, in the subsequent fine-tuning stage, our SGP model adopts a Gaussian Mixture Model (GMM) to factorise these pre-trained embeddings for further training. Our extensive experiments on three public datasets show that SGP can alleviate the cold-start problem while also ensuring effective recommendations for regular users. To alleviate the low-robustness issue and enhance the recommendation effectiveness, we propose to leverage multiple types of side information using the graph pre-training technique. In particular, we aim to generalise the pre-training technique used by SGP for multiple types of side information associated with both users and items. Specifically, we propose two novel pre-training schemes, namely Single-P and Multi-P, to leverage side information such as the ages and occupations of users and the textual reviews and categories of items. Instead of jointly training with two objectives, our pre-training schemes first pre-train a representation model under the users and items’ multi/single relational graphs constructed by their side information and then fine-tune their embeddings under an existing general representation-based recommendation model. Extensive experiments on three public datasets show that the graph pre-training technique can effectively enhance the effectiveness of ranking-based recommender systems and alleviates the cold-start problem. In addition, our pre-training schemes can provide more ef-fective initialisations for both the users and items; hence the robustness of fine-tuning models namely MF, NCF, NGCF and LightGCN, can be improved. Finally, to enhance the training efficiency of graph-based recommenders while ensuring their effectiveness, we propose to use the graph contrastive learning technique to improve the traditional random negative sampling approach. In particular, we propose a dynamic negative sampling (DNS) approach that leverages the graph contrastive learning technique to replace the randomly sampled negative items with more informative negative items. Our experiments show that DNS can improve the recommendation effectiveness of four competitive recommenders. Next, we further propose a novel graph-based model, i.e. MLP-CGRec, that leverages a multiple sampling approach to enhance the training efficiency of the graph-based recommender system. In particular, MLP-CGRec uses DNS to sample contrastive negative items and an efficient graph-based sampling method to select pseudo-positive samples. Experimental results on three public datasets show that MLP-CGRec can maintain competitive effectiveness and achieve the best efficiency compared with state-of-the-art recommender systems