Preference Networks: Probabilistic Models for Recommendation Systems

Recommender systems are important to help users select relevant and personalised information over massive amounts of data available. We propose an unified framework called Preference Network (PN) that jointly models various types of domain knowledge for the task of recommendation. The PN is a probabilistic model that systematically combines both content-based filtering and collaborative filtering into a single conditional Markov random field. Once estimated, it serves as a probabilistic database that supports various useful queries such as rating prediction and top-$N$ recommendation. To handle the challenging problem of learning large networks of users and items, we employ a simple but effective pseudo-likelihood with regularisation. Experiments on the movie rating data demonstrate the merits of the PN.Comment: In Proc. of 6th Australasian Data Mining Conference (AusDM), Gold Coast, Australia, pages 195--202, 200

Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms

학위논문(석사)--서울대학교 대학원 :융합과학기술대학원 융합과학부(디지털정보융합전공),2019. 8. 서봉원.Since Matrix Factorization based linear models have been dominant in the Collaborative Filtering context for a long time in the past, Neural Network based CF Models for recommendation have started to gain attention recently. One branch of research is based on using deep generative models to model user preferences and Variational Autoencoders where shown to give state-of-the-art results. However, there are some potentially problematic characteristics of the current Variational Autoencoder for CF. The first is the too simplistic prior VAEs incorporate for learning the latent representations of user preference, which may be restricting the model from learning more expressive and richer latent variables that could boost recommendation performance. The other is the models inability to learn deeper representations with more than one hidden layer. Our goal is to incorporate appropriate techniques in order to mitigate the aforementioned problems of Variational Autoencoder CF and further improve the recommendation performance of VAE based Collaborative Fil-tering. We bring the VampPrior, which successfully made improvements for image generation to tackle the restrictive prior problem. We also adopt Gat-ed Linear Units (GLUs) which were used in stacked convolutions for lan-guage modeling to control information flow in the easily deepening auto-encoder framework. We show that such simple priors (in original VAEs) may be too restric-tive to fully model user preferences and setting a more flexible prior gives significant gains. We also show that VAMP priors coupled with gating mechanisms outperform SOTA results including the Variational Autoencoder for Collaborative Filtering by meaningful margins on 4 benchmark datasets (MovieLens, Netflix, Pinterest, Melon).최근 뉴럴넷 기반 협업필터링 추천알고리즘이 주목을 받고 있다. 그 중 한 갈래의 연구는 깊은 생성모형 (Deep Generative Model)을 이용해 사용자들의 선호를 모델링하는 방법이다. 이중 Variational Autoencoder를 (VAE) 이용한 방법이 최근 state-of-the-art (SOTA) 성능을 보여주었다. 그러나 VAE를 이용한 협업필터링 알고리즘은 현재 몇 가지의 잠재적인 문제점들을 지니고 있다. 첫 번째는 사용자 선호를 압축하는 잠재변수를 학습하는 과정에서 매우 단순한 사전분포를 사용한다는 것이다. 또 다른 문제점은 모델이 현재 여러 단을 이용한 깊은 인코더와 디코더를 사용하지 못하고 있다는 것이다. 본 연구는 최신기술들을 활용하여 앞선 문제점들을 해결하고 VAE를 이용한 협업필터링 알고리즘의 추천성능을 더욱 높이는 것이 목표이다. 본 연구는 협업필터링 문제에 더 복잡한 사전분포 (Flexible Prior)를 적용한 첫 연구로서, 기존의 단순한 사전분포가 모델의 표현력을 제한할 수 있으며 더 복잡한 사전분포를 정의함으로써 모델의 성능을 더욱 높일 수 있음을 보였다. 이를 위해 이미지 생성 문제에서 좋은 결과를 보인 VampPrior를 이용해 실험을 진행하였다. 또한 VampPrior를 Gating Mechanisim과 함께 사용하였을 때 기존 SOTA를 넘어서는 성능을 보임을 추천알고리즘에서 사용되는 대표적인 데이터셋들을 통해 보여준다.1 INTRODUCTION 1 1.1 Background and Motivation 1 1.2 Research Goal 3 1.3 Enhancing VAEs for Collaborative Filtering 3 1.4 Experiments 5 1.5 Contributions 5 2 RELATED WORK 7 2.1 Collaborative Filtering 7 2.1.1 Traditional methods & Matrix-Factorization based CF 8 2.1.2 Autoencoders for CF 12 2.2 Deep Generative Models (VAE) 17 2.2.1 Variational Bayes 18 2.2.2 Variational Autoencoder 18 2.3 Variational Autoencoder for Collaborative Filtering 20 2.3.1 VAE for CF 21 2.4 Recent research in Computer Vision & Deep Learning 24 2.4.1 VampPrior 24 2.4.2 Gated CNN 25 3 METHOD 28 3.1 Flexible Prior 29 3.1.1 Motivation 29 3.1.2 VampPrior 30 3.1.3 Hierarchical Stochastic Units 31 3.2 Gating Mechanism 32 3.2.1 Motivation 32 3.2.2 Gated Linear Units 34 4 EXPERIMENT 35 4.1 Setup 35 4.1.1 Baseline Models 35 4.1.2 Proposed Models 37 4.1.3 Strong Generalization 37 4.1.4 Evaluation Metrics 38 4.2 Datasets 38 4.3 Configurations 39 4.4 Results 40 4.4.1 Model Performance 40 4.4.5 Further Analysis on the Effect of Gating 44 5 CONCLUSION 45 Bibliography 47 국문초록 51Maste

Deep Learning for Recommender Systems

The widespread adoption of the Internet has led to an explosion in the number of choices available to consumers. Users begin to expect personalized content in modern E-commerce, entertainment and social media platforms. Recommender Systems (RS) provide a critical solution to this problem by maintaining user engagement and satisfaction with personalized content. Traditional RS techniques are often linear limiting the expressivity required to model complex user-item interactions and require extensive handcrafted features from domain experts. Deep learning demonstrated significant breakthroughs in solving problems that have alluded the artificial intelligence community for many years advancing state-of-the-art results in domains such as computer vision and natural language processing. The recommender domain consists of heterogeneous and semantically rich data such as unstructured text (e.g. product descriptions), categorical attributes (e.g. genre of a movie), and user-item feedback (e.g. purchases). Deep learning can automatically capture the intricate structure of user preferences by encoding learned feature representations from high dimensional data. In this thesis, we explore five novel applications of deep learning-based techniques to address top-n recommendation. First, we propose Collaborative Memory Network, which unifies the strengths of the latent factor model and neighborhood-based methods inspired by Memory Networks to address collaborative filtering with implicit feedback. Second, we propose Neural Semantic Personalized Ranking, a novel probabilistic generative modeling approach to integrate deep neural network with pairwise ranking for the item cold-start problem. Third, we propose Attentive Contextual Denoising Autoencoder augmented with a context-driven attention mechanism to integrate arbitrary user and item attributes. Fourth, we propose a flexible encoder-decoder architecture called Neural Citation Network, embodying a powerful max time delay neural network encoder augmented with an attention mechanism and author networks to address context-aware citation recommendation. Finally, we propose a generic framework to perform conversational movie recommendations which leverages transfer learning to infer user preferences from natural language. Comprehensive experiments validate the effectiveness of all five proposed models against competitive baseline methods and demonstrate the successful adaptation of deep learning-based techniques to the recommendation domain

Mining User Personality from Music Listening Behavior in Online Platforms Using Audio Attributes

Music and emotions are inherently intertwined. Humans leave hints of their personality everywhere, and particularly their music listening behavior shows conscious and unconscious diametric tendencies and influences. So, what could be more elegant than finding the underlying character given the attributes of a certain music piece and, as such, identifying the likelihood that music preference is also imprinted or at least resonating with its listener? This thesis focuses on the music audio attributes or the latent song features to determine human personality. Based on unsupervised learning, we cluster several large music datasets using multiple clustering techniques known to us. This analysis led us to classify song genres based on audio attributes, which can be deemed a novel contribution in the intersection of Music Information Retrieval (MIR) and human psychology studies. Existing research found a relationship between Myers-Briggs personality models and music genres. Our goal was to correlate audio attributes with the music genre, which will ultimately help us to determine user personality based on their music listening behavior from online music platforms. This target has been achieved as we showed the users’ spectral personality traits from the audio feature values of the songs they listen to online and verified our decision process with the help of a customized Music Recommendation System (MRS). Our model performs genre classification and personality detection with 78% and 74% accuracy, respectively. The results are promising compared to competitor approaches as they are explainable via statistics and visualizations. Furthermore, the RS completes and validates our pursuit through 81.3% accurate song suggestions. We believe the outcome of this thesis will work as an inspiration and assistance for fellow researchers in this arena to come up with more personalized song suggestions. As music preferences will shape specific user personality parameters, it is expected that more such elements will surface that would portray the daily activities of individuals and their underlying mentality

Extracting information from informal communication

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (leaves 89-93).This thesis focuses on the problem of extracting information from informal communication. Textual informal communication, such as e-mail, bulletin boards and blogs, has become a vast information resource. However, such information is poorly organized and difficult for a computer to understand due to lack of editing and structure. Thus, techniques which work well for formal text, such as newspaper articles, may be considered insufficient on informal text. One focus of ours is to attempt to advance the state-of-the-art for sub-problems of the information extraction task. We make contributions to the problems of named entity extraction, co-reference resolution and context tracking. We channel our efforts toward methods which are particularly applicable to informal communication. We also consider a type of information which is somewhat unique to informal communication: preferences and opinions. Individuals often expression their opinions on products and services in such communication. Others' may read these "reviews" to try to predict their own experiences. However, humans do a poor job of aggregating and generalizing large sets of data. We develop techniques that can perform the job of predicting unobserved opinions.(cont.) We address both the single-user case where information about the items is known, and the multi-user case where we can generalize opinions without external information. Experiments on large-scale rating data sets validate our approach.by Jason D.M. Rennie.Ph.D