Deep learning-based implicit feedback recommendation

Recommender systems are of vital importance, in the era of the Web, to address the problem of information overload. It can benefit both users by recommending personalized interesting items and service providers by increasing their site traffic. Plenty of use cases have emerged as applied recommender systems, including but not limited to multimedia recommendation (e.g., news, movies, music, and videos) and e-commerce recommendation. A recommendation agent can be trained from user-item interaction data which can be categorized as explicit feedback and implicit feedback. Compared with explicit ratings which depict the user preference explicitly, implicit feedback data like clicks, purchases, and dwell time is more prevalent in the real-world scenario. On the other hand, deep learning has achieved great success recently due to the high model expressiveness and fidelity. In this thesis, we investigate deep learning techniques for recommendation from implicit feedback data. We focus on two learning perspectives: deep supervised learning and deep reinforcement learning. Supervised learning tries to infer knowledge from implicit historical interactions. From this perspective, two models namely Convolutional Factorization Machines (CFM) and Relational Collaborative Filtering (RCF) are proposed. CFM tackles the implicit user-item interactions with side information as feature vectors and utilizes convolutional neural networks to learn high-order interaction signals. RCF considers multiple item relations into the recommendation model and tackles the implicit feedback as relation-enriched data. The two models investigate deep learning techniques for recommendation by tackling the data as two different structures: feature vectors and relations. Experimental results demonstrate that the proposed deep learning models are effective to improve the recommendation accuracy. Besides, RCF also helps to provide explainable recommendation and get a better comprehension of user behaviors. Reinforcement learning is reward-driven and focuses on long-term optimization in a whole interaction session, which conforms more with the objective of recommender systems. From this perspective, we first formulate the next-item recommendation task from implicit feedback data as a Markov Decision Process (MDP). Then we analyzed that directly utilizing reinforcement learning algorithms for recommendation is infeasible due to the challenge of pure off-policy setting and lack of negative reward signals. To address the problems, we proposed Self-Supervised Q-learning (SQN) and Self-Supervised Actor-Critic (SAC). The key insight is to combine reinforcement learning with supervised learning and perform knowledge transfer between the two components. Based on SQN and SAC, we further proposed Self-Supervised Negative Q-learning (SNQN) and Self-Supervised Advantage Actor-Critic (SA2C) to introduce the negative sampling strategy to enhance the learning of the reinforcement component. Experimental results demonstrate that the proposed learning frameworks are effective when integrated with different existing base models. Moreover, we show that combining supervised learning and reinforcement learning is a promising direction for future recommender systems. In that case, reinforcement learning introduces reward-driven objectives and long-term optimization perspectives into supervised learning while supervised learning helps to improve the data efficiency for reinforcement learning

Fast Matrix Factorization for Online Recommendation with Implicit Feedback

This paper contributes improvements on both the effectiveness and efficiency of Matrix Factorization (MF) methods for implicit feedback. We highlight two critical issues of existing works. First, due to the large space of unobserved feedback, most existing works resort to assign a uniform weight to the missing data to reduce computational complexity. However, such a uniform assumption is invalid in real-world settings. Second, most methods are also designed in an offline setting and fail to keep up with the dynamic nature of online data. We address the above two issues in learning MF models from implicit feedback. We first propose to weight the missing data based on item popularity, which is more effective and flexible than the uniform-weight assumption. However, such a non-uniform weighting poses efficiency challenge in learning the model. To address this, we specifically design a new learning algorithm based on the element-wise Alternating Least Squares (eALS) technique, for efficiently optimizing a MF model with variably-weighted missing data. We exploit this efficiency to then seamlessly devise an incremental update strategy that instantly refreshes a MF model given new feedback. Through comprehensive experiments on two public datasets in both offline and online protocols, we show that our eALS method consistently outperforms state-of-the-art implicit MF methods. Our implementation is available at https://github.com/hexiangnan/sigir16-eals.Comment: 10 pages, 8 figure