Effective neural architectures for context-aware venue recommendation

Users in Location-Based Social Networks (LBSNs), such as Yelp and Foursquare, can search for interesting venues such as restaurants and museums to visit, or share their location with their friends by making an implicit feedback (e.g. checking in at venues they have visited). The users can also leave explicit feedback on the venues they have visited by providing rat- ings and/or comments. Such explicit and implicit feedback by the users provide rich infor- mation about both users and venues, and thus can be leveraged to study the users’ movement in urban cities, as well as enhance the quality of personalised venue recommendations. Un- like traditional recommendation systems (e.g. book and movie recommendation systems), making effective venue recommendations is more challenging because we need to take into account the users’ current context (e.g. time of the day, user’s current location as well as his recently visited venues). Two common techniques that are widely used in the literature for venue recommen- dation systems are Matrix Factorisation (MF) and Bayesian Personalised Ranking (BPR). MF is a popular Collaborative Filtering (CF) technique that can leverage the users’ explicit feedback (e.g. the numerical ratings) to predict the users’ ratings on the venues and hence relevant venues can be suggested to the users based on these predicted ratings. On the other hand, BPR is a pairwise ranking-based model that can leverage implicit feedback to generate effective top-K venue recommendations. In this thesis, based upon MF and BPR models, we aim to generate effective context-aware venue recommendation that a user may wish to visit based on the user’s historical explicit and implicit feedbacks, the user’s contextual informa- tion (e.g. the user’s current location and time of the day) and additional information (e.g. the geographical location of venues and users’ social relationships). To achieve this goal, we need to address the following challenges: namely (C1) modelling the users’ preferences and the characteristic of venues, (C2) capturing the complex structure of user-venue inter- actions in a Collaborative Filtering manner, (C3) modelling the users’ short-term (dynamic) preferences from the sequential order of user’s observed feedback as well as the contextual information associated with the successive feedback, (C4) generating accurate top-K venue recommendations based on the users’ preferences using a pairwise ranking-based model and (C5) appropriately sampling potential negative instances to train a ranking-based model. First, to address challenge C1, we leverage the users’ explicit feedback (e.g. their rat- ings and the textual content of the comments) and additional information (e.g. users’ social relationships) to effectively model the users’ preferences and the characteristics of venues. In particular, we propose a novel regularisation technique and a factorisation-based model that leverages the users’ explicit feedback and the additional information to improve the rat- ing prediction accuracy of the traditional MF model. Experiments conducted on a large scale rating dataset on LBSN demonstrate that the textual content of comments plays an important role in enhancing the accuracy of rating prediction. Second, we investigate how to leverage the users’ implicit feedback and additional in- formation such as the users’ social relationship and the geographical location of venues to improve the quality of top-K venue recommendations. We argue that the potential negative instances can be effectively sampled based on the social correlations between users and their friends as well as the geographical influences between the users’ and venues’ geographi- cal location. In particular, to address challenges C4 and C5, we propose a novel pairwise ranking-based framework for top-K venue recommendations that can incorporate multiple sources of additional information (e.g. the users’ social relationship and the geographical location of venues) to effectively sample the potential negative instances. Experimental re- sults on three large scale checkin and rating datasets from LBSNs demonstrate that the social correlations and the geographical influences play an important role to the quality of sampled negative instances and hence can improve the quality of top-K venue recommendations. Finally, to address challenges C2 and C3, we propose a framework for context-aware venue recommendations that exploits Deep Neural Network (DNN) models to effectively capture the complex structure of user-venue interactions and the users’ long-term (dynamic) preferences from their sequential order of checkins. In particular, within the framework, we propose a novel Recurrent Neural Network (RNN) architecture that can effectively in- corporate the contextual information associated with the successive implicit feedback (e.g. the time interval and the geographical distance between two successive checkins) to gener- ate high quality context-aware venue recommendations. Experimental results on three large scale checkin and rating datasets from LBSNs demonstrate the effectiveness and robustness of our proposed framework for context-aware venue recommendations. In particular, the results demonstrate that the sequential order of users’ implicit feedback can be leveraged to effectively improve the effectiveness of context-aware venue recommendation system. In addition, the time intervals and the geographical distances between two successive checkins play an important role in capturing the users’ short-term preferences

Contextual Attention Recurrent Architecture for Context-aware Venue Recommendation

Venue recommendation systems aim to effectively rank a list of interesting venues users should visit based on their historical feedback (e.g. checkins). Such systems are increasingly deployed by Location-based Social Networks (LBSNs) such as Foursquare and Yelp to enhance their usefulness to users. Recently, various RNN architectures have been proposed to incorporate contextual information associated with the users' sequence of checkins (e.g. time of the day, location of venues) to effectively capture the users' dynamic preferences. However, these architectures assume that different types of contexts have an identical impact on the users' preferences, which may not hold in practice. For example, an ordinary context such as the time of the day reflects the user's current contextual preferences, whereas a transition context - such as a time interval from their last visited venue - indicates a transition effect from past behaviour to future behaviour. To address these challenges, we propose a novel Contextual Attention Recurrent Architecture (CARA) that leverages both sequences of feedback and contextual information associated with the sequences to capture the users' dynamic preferences. Our proposed recurrent architecture consists of two types of gating mechanisms, namely 1) a contextual attention gate that controls the influence of the ordinary context on the users' contextual preferences and 2) a time- and geo-based gate that controls the influence of the hidden state from the previous checkin based on the transition context. Thorough experiments on three large checkin and rating datasets from commercial LBSNs demonstrate the effectiveness of our proposed CARA architecture by significantly outperforming many state-of-the-art RNN architectures and factorisation approaches

Recommendations for item set completion: On the semantics of item co-occurrence with data sparsity, input size, and input modalities

We address the problem of recommending relevant items to a user in order to "complete" a partial set of items already known. We consider the two scenarios of citation and subject label recommendation, which resemble different semantics of item co-occurrence: relatedness for co-citations and diversity for subject labels. We assess the influence of the completeness of an already known partial item set on the recommender performance. We also investigate data sparsity through a pruning parameter and the influence of using additional metadata. As recommender models, we focus on different autoencoders, which are particularly suited for reconstructing missing items in a set. We extend autoencoders to exploit a multi-modal input of text and structured data. Our experiments on six real-world datasets show that supplying the partial item set as input is helpful when item co-occurrence resembles relatedness, while metadata are effective when co-occurrence implies diversity. This outcome means that the semantics of item co-occurrence is an important factor. The simple item co-occurrence model is a strong baseline for citation recommendation. However, autoencoders have the advantage to enable exploiting additional metadata besides the partial item set as input and achieve comparable performance. For the subject label recommendation task, the title is the most important attribute. Adding more input modalities sometimes even harms the result. In conclusion, it is crucial to consider the semantics of the item co-occurrence for the choice of an appropriate recommendation model and carefully decide which metadata to exploit

Machine Learning Models for Context-Aware Recommender Systems

The mass adoption of the internet has resulted in the exponential growth of products and services on the world wide web. An individual consumer, faced with this data deluge, is expected to make reasonable choices saving time and money. Organizations are facing increased competition, and they are looking for innovative ways to increase revenue and customer loyalty. A business wants to target the right product or service to an individual consumer, and this drives personalized recommendation. Recommender systems, designed to provide personalized recommendations, initially focused only on the user-item interaction. However, these systems evolved to provide a context-aware recommendations. Context-aware recommender systems utilize additional context, such as genre for movie recommendation, while recommending items to users. Latent factor methods have been a popular choice for recommender systems. With the resurgence of neural networks, there has also been a trend towards applying deep learning methods to recommender systems. This study proposes a novel contextual latent factor model that is capable of utilizing the context from a dual-perspective of both users and items. The proposed model, known as the Group-Aware Latent Factor Model (GLFM), is applied to the event recommendation task. The GLFM model is extensible, and it allows other contextual attributes to be easily be incorporated into the model. While latent-factor models have been extremely popular for recommender systems, they are unable to model the complex non-linear user-item relationships. This has resulted in the interest in applying deep learning methods to recommender systems. This study also proposes another novel method based on the denoising autoencoder architecture, which is referred to as the Attentive Contextual Denoising Autoencoder (ACDA). The ACDA model augments the basic denoising autoencoder with a context-driven attention mechanism to provide personalized recommendation. The ACDA model is applied to the event and movie recommendation tasks. The effectiveness of the proposed models is demonstrated against real-world datasets from Meetup and Movielens, and the results are compared against the current state-of-the-art baseline methods

Deep Learning for Recommender Systems

The widespread adoption of the Internet has led to an explosion in the number of choices available to consumers. Users begin to expect personalized content in modern E-commerce, entertainment and social media platforms. Recommender Systems (RS) provide a critical solution to this problem by maintaining user engagement and satisfaction with personalized content. Traditional RS techniques are often linear limiting the expressivity required to model complex user-item interactions and require extensive handcrafted features from domain experts. Deep learning demonstrated significant breakthroughs in solving problems that have alluded the artificial intelligence community for many years advancing state-of-the-art results in domains such as computer vision and natural language processing. The recommender domain consists of heterogeneous and semantically rich data such as unstructured text (e.g. product descriptions), categorical attributes (e.g. genre of a movie), and user-item feedback (e.g. purchases). Deep learning can automatically capture the intricate structure of user preferences by encoding learned feature representations from high dimensional data. In this thesis, we explore five novel applications of deep learning-based techniques to address top-n recommendation. First, we propose Collaborative Memory Network, which unifies the strengths of the latent factor model and neighborhood-based methods inspired by Memory Networks to address collaborative filtering with implicit feedback. Second, we propose Neural Semantic Personalized Ranking, a novel probabilistic generative modeling approach to integrate deep neural network with pairwise ranking for the item cold-start problem. Third, we propose Attentive Contextual Denoising Autoencoder augmented with a context-driven attention mechanism to integrate arbitrary user and item attributes. Fourth, we propose a flexible encoder-decoder architecture called Neural Citation Network, embodying a powerful max time delay neural network encoder augmented with an attention mechanism and author networks to address context-aware citation recommendation. Finally, we propose a generic framework to perform conversational movie recommendations which leverages transfer learning to infer user preferences from natural language. Comprehensive experiments validate the effectiveness of all five proposed models against competitive baseline methods and demonstrate the successful adaptation of deep learning-based techniques to the recommendation domain

Target Apps Selection: Towards a Unified Search Framework for Mobile Devices

With the recent growth of conversational systems and intelligent assistants such as Apple Siri and Google Assistant, mobile devices are becoming even more pervasive in our lives. As a consequence, users are getting engaged with the mobile apps and frequently search for an information need in their apps. However, users cannot search within their apps through their intelligent assistants. This requires a unified mobile search framework that identifies the target app(s) for the user's query, submits the query to the app(s), and presents the results to the user. In this paper, we take the first step forward towards developing unified mobile search. In more detail, we introduce and study the task of target apps selection, which has various potential real-world applications. To this aim, we analyze attributes of search queries as well as user behaviors, while searching with different mobile apps. The analyses are done based on thousands of queries that we collected through crowdsourcing. We finally study the performance of state-of-the-art retrieval models for this task and propose two simple yet effective neural models that significantly outperform the baselines. Our neural approaches are based on learning high-dimensional representations for mobile apps. Our analyses and experiments suggest specific future directions in this research area.Comment: To appear at SIGIR 201