Graph Convolutional Matrix Completion

We consider matrix completion for recommender systems from the point of view of link prediction on graphs. Interaction data such as movie ratings can be represented by a bipartite user-item graph with labeled edges denoting observed ratings. Building on recent progress in deep learning on graph-structured data, we propose a graph auto-encoder framework based on differentiable message passing on the bipartite interaction graph. Our model shows competitive performance on standard collaborative filtering benchmarks. In settings where complimentary feature information or structured data such as a social network is available, our framework outperforms recent state-of-the-art methods.Comment: 9 pages, 3 figures, updated with additional experimental evaluatio

Deep Learning based Recommender System: A Survey and New Perspectives

With the ever-growing volume of online information, recommender systems have been an effective strategy to overcome such information overload. The utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. Evidently, the field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the field.Comment: The paper has been accepted by ACM Computing Surveys. https://doi.acm.org/10.1145/328502

A Survey on Bayesian Deep Learning

A comprehensive artificial intelligence system needs to not only perceive the environment with different `senses' (e.g., seeing and hearing) but also infer the world's conditional (or even causal) relations and corresponding uncertainty. The past decade has seen major advances in many perception tasks such as visual object recognition and speech recognition using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. In recent years, Bayesian deep learning has emerged as a unified probabilistic framework to tightly integrate deep learning and Bayesian models. In this general framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in turn, the feedback from the inference process is able to enhance the perception of text or images. This survey provides a comprehensive introduction to Bayesian deep learning and reviews its recent applications on recommender systems, topic models, control, etc. Besides, we also discuss the relationship and differences between Bayesian deep learning and other related topics such as Bayesian treatment of neural networks.Comment: To appear in ACM Computing Surveys (CSUR) 202

그래프 협업 필터링을 위한 강력한 부트스트래핑 기반 자기 지도 학습

학위논문(석사) -- 서울대학교대학원 : 공과대학 컴퓨터공학부, 2023. 2. 권태경.대조 학습 기반 모델은 원시 데이터에서 자체 감독 신호를 추출하는 기능이 데이터 희소성 문제를 해결하기 위한 추천 시스템의 요구 사항과 일치하기 때문에 추천 연구에서 주목을 받고 있다. 이러한 효율성에도 불구하고 대조 학습 기반 모델에는 중요한 한계점이 있다. 바로 네거티브 샘플링이다. 네거티브 샘플링 방식을 사용하면 사용자의 취향에 맞는 항목이지만 상호작용이 관찰되지 않은 사용자-항목 쌍을 네거티브로 선택할 수 있다. 이를 해결하기 위해 네거티브 샘플링이 필요하지 않은 부트스트래핑 기반의 자기 지도 학습 방법이 제안되었다. 그러나 이 방법에도 한계점이 있다. 관찰된 샘플만 사용하기 때문에 노이즈가 있는 상호 작용에 취약하다. 또한 실제 데이터 셋에는 희소성 문제가 있다. 위의 문제를 해결하기 위해 그래프 협업 필터링을 위한 강력한 부트스트래핑 기반 자기 지도 학습 모델, RBS를 소개한다. RBS는 그래프 노이즈 제거 모듈과 자가 지도 학습 모듈의 두 가지 모듈로 구성된다. 그래프 노이즈 제거 모듈은 잡음이 있는 상호 작용의 영향을 줄이기 위해 설계되었다. 자기 지도 학습 모듈은 온라인 인코더와 타깃 인코더로 구성된다. RBS는 타깃 인코더의 표현을 예측하도록 온라인 인코더를 학습하는 반면, 타깃 인코더는 온라인 인코더를 천천히 근사하여 일관된 타깃을 제공한다. 또한 RBS는 인코더 입력에 노이즈를 추가하여 데이터 희소성 문제를 효과적으로 완화한다. 3가지 벤치마크 데이터 셋에 대한 포괄적인 경험적 연구는 RBS가 모든 기준 모델을 일관되고 크게 능가한다는 것을 보여준다.Contrastive learning (CL) based models are gaining traction in recommendation research, since their ability to extract self-supervised signals from raw data matches the requirements of recommender systems to solve the data sparsity issue. Despite their effectiveness, CL-based models have an important limitation: negative sampling. A negative sampling scheme allows positive but unobserved pairs to be selected as negative. To solve this problem, a bootstrapping-based self-supervised learning method that does not require negative sampling has been proposed. However, this method also has limitations. Because only positive samples are used, it is vulnerable to noisy interactions. Also, there is a sparsity issue in real-world data sets. To tackle the above issues, we introduce a Robust Bootstrapping-based Self-supervised learning model for graph collaborative filtering, named RBS. RBS consists of two modules: a graph denoising module and a self-supervised learning module. The graph denoising module is designed to reduce the influence of noisy interactions. The self-supervised learning module consists of an online encoder and a target encoder. RBS trains its online encoder to predict the target encoders representation, while the target encoder provides consistent targets by slowly approximating the online encoder. In addition, RBS effectively alleviates the data sparsity issue, by adding noises to encoder inputs. A comprehensive empirical study on three benchmark datasets demonstrates that RBS consistently and significantly outperforms all baseline methods.Chapter 1. Introduction 1 Chapter 2. Related Work 3 2.1. Graph Neural Networks 3 2.2. Graph Collaborative Filtering 3 2.3. Self-supervised Learning 4 Chapter 3. Methodology 6 3.1. Overview 6 3.2. Problem Definition 6 3.3. Graph Denoising Module 7 3.4. Self-supervised Learning Module 9 3.5. Prediction 11 Chapter 4. Experiments 12 4.1. Datasets 12 4.2. Baselines 13 4.3. Evaluation Metrics 13 4.4. Implementation Details 14 4.5. Overall Performance 14 4.6. Ablation Study 18 Chapter 5. Conclusion 21 Bibliography 22 초록 27석

TransNets: Learning to Transform for Recommendation

Recently, deep learning methods have been shown to improve the performance of recommender systems over traditional methods, especially when review text is available. For example, a recent model, DeepCoNN, uses neural nets to learn one latent representation for the text of all reviews written by a target user, and a second latent representation for the text of all reviews for a target item, and then combines these latent representations to obtain state-of-the-art performance on recommendation tasks. We show that (unsurprisingly) much of the predictive value of review text comes from reviews of the target user for the target item. We then introduce a way in which this information can be used in recommendation, even when the target user's review for the target item is not available. Our model, called TransNets, extends the DeepCoNN model by introducing an additional latent layer representing the target user-target item pair. We then regularize this layer, at training time, to be similar to another latent representation of the target user's review of the target item. We show that TransNets and extensions of it improve substantially over the previous state-of-the-art.Comment: Accepted for publication in the 11th ACM Conference on Recommender Systems (RecSys 2017

Joint Deep Modeling of Users and Items Using Reviews for Recommendation

A large amount of information exists in reviews written by users. This source of information has been ignored by most of the current recommender systems while it can potentially alleviate the sparsity problem and improve the quality of recommendations. In this paper, we present a deep model to learn item properties and user behaviors jointly from review text. The proposed model, named Deep Cooperative Neural Networks (DeepCoNN), consists of two parallel neural networks coupled in the last layers. One of the networks focuses on learning user behaviors exploiting reviews written by the user, and the other one learns item properties from the reviews written for the item. A shared layer is introduced on the top to couple these two networks together. The shared layer enables latent factors learned for users and items to interact with each other in a manner similar to factorization machine techniques. Experimental results demonstrate that DeepCoNN significantly outperforms all baseline recommender systems on a variety of datasets.Comment: WSDM 201

A deep learning-based hybrid model for recommendation generation and ranking

A recommender system plays a vital role in information filtering and retrieval, and its application is omnipresent in many domains. There are some drawbacks such as the cold-start and the data sparsity problems which affect the performance of the recommender model. Various studies help with drastically improving the performance of recommender systems via unique methods, such as the traditional way of performing matrix factorization (MF) and also applying deep learning (DL) techniques in recent years. By using DL in the recommender system, we can overcome the difficulties of collaborative filtering. DL now focuses mainly on modeling content descriptions, but those models ignore the main factor of user–item interaction. In the proposed hybrid Bayesian stacked auto-denoising encoder (HBSADE) model, it recognizes the latent interests of the user and analyzes contextual reviews that are performed through the MF method. The objective of the model is to identify the user’s point of interest, recommending products/services based on the user’s latent interests. The proposed two-stage novel hybrid deep learning-based collaborative filtering method explores the user’s point of interest, captures the communications between items and users and provides better recommendations in a personalized way. We used a multilayer neural network to manipulate the nonlinearities between the user and item communication from data. Experiments were to prove that our HBSADE outperforms existing methodologies over Amazon-b and Book-Crossing datasets

Collaborative Deep Learning for Recommender Systems

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recent advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art