Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms

학위논문(석사)--서울대학교 대학원 :융합과학기술대학원 융합과학부(디지털정보융합전공),2019. 8. 서봉원.Since Matrix Factorization based linear models have been dominant in the Collaborative Filtering context for a long time in the past, Neural Network based CF Models for recommendation have started to gain attention recently. One branch of research is based on using deep generative models to model user preferences and Variational Autoencoders where shown to give state-of-the-art results. However, there are some potentially problematic characteristics of the current Variational Autoencoder for CF. The first is the too simplistic prior VAEs incorporate for learning the latent representations of user preference, which may be restricting the model from learning more expressive and richer latent variables that could boost recommendation performance. The other is the models inability to learn deeper representations with more than one hidden layer. Our goal is to incorporate appropriate techniques in order to mitigate the aforementioned problems of Variational Autoencoder CF and further improve the recommendation performance of VAE based Collaborative Fil-tering. We bring the VampPrior, which successfully made improvements for image generation to tackle the restrictive prior problem. We also adopt Gat-ed Linear Units (GLUs) which were used in stacked convolutions for lan-guage modeling to control information flow in the easily deepening auto-encoder framework. We show that such simple priors (in original VAEs) may be too restric-tive to fully model user preferences and setting a more flexible prior gives significant gains. We also show that VAMP priors coupled with gating mechanisms outperform SOTA results including the Variational Autoencoder for Collaborative Filtering by meaningful margins on 4 benchmark datasets (MovieLens, Netflix, Pinterest, Melon).최근 뉴럴넷 기반 협업필터링 추천알고리즘이 주목을 받고 있다. 그 중 한 갈래의 연구는 깊은 생성모형 (Deep Generative Model)을 이용해 사용자들의 선호를 모델링하는 방법이다. 이중 Variational Autoencoder를 (VAE) 이용한 방법이 최근 state-of-the-art (SOTA) 성능을 보여주었다. 그러나 VAE를 이용한 협업필터링 알고리즘은 현재 몇 가지의 잠재적인 문제점들을 지니고 있다. 첫 번째는 사용자 선호를 압축하는 잠재변수를 학습하는 과정에서 매우 단순한 사전분포를 사용한다는 것이다. 또 다른 문제점은 모델이 현재 여러 단을 이용한 깊은 인코더와 디코더를 사용하지 못하고 있다는 것이다. 본 연구는 최신기술들을 활용하여 앞선 문제점들을 해결하고 VAE를 이용한 협업필터링 알고리즘의 추천성능을 더욱 높이는 것이 목표이다. 본 연구는 협업필터링 문제에 더 복잡한 사전분포 (Flexible Prior)를 적용한 첫 연구로서, 기존의 단순한 사전분포가 모델의 표현력을 제한할 수 있으며 더 복잡한 사전분포를 정의함으로써 모델의 성능을 더욱 높일 수 있음을 보였다. 이를 위해 이미지 생성 문제에서 좋은 결과를 보인 VampPrior를 이용해 실험을 진행하였다. 또한 VampPrior를 Gating Mechanisim과 함께 사용하였을 때 기존 SOTA를 넘어서는 성능을 보임을 추천알고리즘에서 사용되는 대표적인 데이터셋들을 통해 보여준다.1 INTRODUCTION 1 1.1 Background and Motivation 1 1.2 Research Goal 3 1.3 Enhancing VAEs for Collaborative Filtering 3 1.4 Experiments 5 1.5 Contributions 5 2 RELATED WORK 7 2.1 Collaborative Filtering 7 2.1.1 Traditional methods & Matrix-Factorization based CF 8 2.1.2 Autoencoders for CF 12 2.2 Deep Generative Models (VAE) 17 2.2.1 Variational Bayes 18 2.2.2 Variational Autoencoder 18 2.3 Variational Autoencoder for Collaborative Filtering 20 2.3.1 VAE for CF 21 2.4 Recent research in Computer Vision & Deep Learning 24 2.4.1 VampPrior 24 2.4.2 Gated CNN 25 3 METHOD 28 3.1 Flexible Prior 29 3.1.1 Motivation 29 3.1.2 VampPrior 30 3.1.3 Hierarchical Stochastic Units 31 3.2 Gating Mechanism 32 3.2.1 Motivation 32 3.2.2 Gated Linear Units 34 4 EXPERIMENT 35 4.1 Setup 35 4.1.1 Baseline Models 35 4.1.2 Proposed Models 37 4.1.3 Strong Generalization 37 4.1.4 Evaluation Metrics 38 4.2 Datasets 38 4.3 Configurations 39 4.4 Results 40 4.4.1 Model Performance 40 4.4.5 Further Analysis on the Effect of Gating 44 5 CONCLUSION 45 Bibliography 47 국문초록 51Maste

PocketCare: Tracking the Flu with Mobile Phones using Partial Observations of Proximity and Symptoms

Mobile phones provide a powerful sensing platform that researchers may adopt to understand proximity interactions among people and the diffusion, through these interactions, of diseases, behaviors, and opinions. However, it remains a challenge to track the proximity-based interactions of a whole community and then model the social diffusion of diseases and behaviors starting from the observations of a small fraction of the volunteer population. In this paper, we propose a novel approach that tries to connect together these sparse observations using a model of how individuals interact with each other and how social interactions happen in terms of a sequence of proximity interactions. We apply our approach to track the spreading of flu in the spatial-proximity network of a 3000-people university campus by mobilizing 300 volunteers from this population to monitor nearby mobile phones through Bluetooth scanning and to daily report flu symptoms about and around them. Our aim is to predict the likelihood for an individual to get flu based on how often her/his daily routine intersects with those of the volunteers. Thus, we use the daily routines of the volunteers to build a model of the volunteers as well as of the non-volunteers. Our results show that we can predict flu infection two weeks ahead of time with an average precision from 0.24 to 0.35 depending on the amount of information. This precision is six to nine times higher than with a random guess model. At the population level, we can predict infectious population in a two-week window with an r-squared value of 0.95 (a random-guess model obtains an r-squared value of 0.2). These results point to an innovative approach for tracking individuals who have interacted with people showing symptoms, allowing us to warn those in danger of infection and to inform health researchers about the progression of contact-induced diseases