8 research outputs found
T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillation
Graph Neural Networks (GNNs) have been a prevailing technique for tackling
various analysis tasks on graph data. A key premise for the remarkable
performance of GNNs relies on complete and trustworthy initial graph
descriptions (i.e., node features and graph structure), which is often not
satisfied since real-world graphs are often incomplete due to various
unavoidable factors. In particular, GNNs face greater challenges when both node
features and graph structure are incomplete at the same time. The existing
methods either focus on feature completion or structure completion. They
usually rely on the matching relationship between features and structure, or
employ joint learning of node representation and feature (or structure)
completion in the hope of achieving mutual benefit. However, recent studies
confirm that the mutual interference between features and structure leads to
the degradation of GNN performance. When both features and structure are
incomplete, the mismatch between features and structure caused by the missing
randomness exacerbates the interference between the two, which may trigger
incorrect completions that negatively affect node representation. To this end,
in this paper we propose a general GNN framework based on teacher-student
distillation to improve the performance of GNNs on incomplete graphs, namely
T2-GNN. To avoid the interference between features and structure, we separately
design feature-level and structure-level teacher models to provide targeted
guidance for student model (base GNNs, such as GCN) through distillation. Then
we design two personalized methods to obtain well-trained feature and structure
teachers. To ensure that the knowledge of the teacher model is comprehensively
and effectively distilled to the student model, we further propose a dual
distillation mode to enable the student to acquire as much expert knowledge as
possible.Comment: Accepted by AAAI2
Genetic Meta-Structure Search for Recommendation on Heterogeneous Information Network
In the past decade, the heterogeneous information network (HIN) has become an
important methodology for modern recommender systems. To fully leverage its
power, manually designed network templates, i.e., meta-structures, are
introduced to filter out semantic-aware information. The hand-crafted
meta-structure rely on intense expert knowledge, which is both laborious and
data-dependent. On the other hand, the number of meta-structures grows
exponentially with its size and the number of node types, which prohibits
brute-force search. To address these challenges, we propose Genetic
Meta-Structure Search (GEMS) to automatically optimize meta-structure designs
for recommendation on HINs. Specifically, GEMS adopts a parallel genetic
algorithm to search meaningful meta-structures for recommendation, and designs
dedicated rules and a meta-structure predictor to efficiently explore the
search space. Finally, we propose an attention based multi-view graph
convolutional network module to dynamically fuse information from different
meta-structures. Extensive experiments on three real-world datasets suggest the
effectiveness of GEMS, which consistently outperforms all baseline methods in
HIN recommendation. Compared with simplified GEMS which utilizes hand-crafted
meta-paths, GEMS achieves over performance gain on most evaluation
metrics. More importantly, we conduct an in-depth analysis on the identified
meta-structures, which sheds light on the HIN based recommender system design.Comment: Published in Proceedings of the 29th ACM International Conference on
Information and Knowledge Management (CIKM '20
ATBRG: Adaptive Target-Behavior Relational Graph Network for Effective Recommendation
Recommender system (RS) devotes to predicting user preference to a given item
and has been widely deployed in most web-scale applications. Recently,
knowledge graph (KG) attracts much attention in RS due to its abundant
connective information. Existing methods either explore independent meta-paths
for user-item pairs over KG, or employ graph neural network (GNN) on whole KG
to produce representations for users and items separately. Despite
effectiveness, the former type of methods fails to fully capture structural
information implied in KG, while the latter ignores the mutual effect between
target user and item during the embedding propagation. In this work, we propose
a new framework named Adaptive Target-Behavior Relational Graph network (ATBRG
for short) to effectively capture structural relations of target user-item
pairs over KG. Specifically, to associate the given target item with user
behaviors over KG, we propose the graph connect and graph prune techniques to
construct adaptive target-behavior relational graph. To fully distill
structural information from the sub-graph connected by rich relations in an
end-to-end fashion, we elaborate on the model design of ATBRG, equipped with
relation-aware extractor layer and representation activation layer. We perform
extensive experiments on both industrial and benchmark datasets. Empirical
results show that ATBRG consistently and significantly outperforms
state-of-the-art methods. Moreover, ATBRG has also achieved a performance
improvement of 5.1% on CTR metric after successful deployment in one popular
recommendation scenario of Taobao APP.Comment: Accepted by SIGIR 2020, full paper with 10 pages and 5 figure
Federated Heterogeneous Graph Neural Network for Privacy-preserving Recommendation
The heterogeneous information network (HIN), which contains rich semantics
depicted by meta-paths, has emerged as a potent tool for mitigating data
sparsity in recommender systems. Existing HIN-based recommender systems operate
under the assumption of centralized storage and model training. However,
real-world data is often distributed due to privacy concerns, leading to the
semantic broken issue within HINs and consequent failures in centralized
HIN-based recommendations. In this paper, we suggest the HIN is partitioned
into private HINs stored on the client side and shared HINs on the server.
Following this setting, we propose a federated heterogeneous graph neural
network (FedHGNN) based framework, which facilitates collaborative training of
a recommendation model using distributed HINs while protecting user privacy.
Specifically, we first formalize the privacy definition for HIN-based federated
recommendation (FedRec) in the light of differential privacy, with the goal of
protecting user-item interactions within private HIN as well as users'
high-order patterns from shared HINs. To recover the broken meta-path based
semantics and ensure proposed privacy measures, we elaborately design a
semantic-preserving user interactions publishing method, which locally perturbs
user's high-order patterns and related user-item interactions for publishing.
Subsequently, we introduce an HGNN model for recommendation, which conducts
node- and semantic-level aggregations to capture recovered semantics. Extensive
experiments on four datasets demonstrate that our model outperforms existing
methods by a substantial margin (up to 34% in HR@10 and 42% in NDCG@10) under a
reasonable privacy budget.Comment: Accepted by WWW 202