Diverse personalized recommendations with uncertainty from implicit preference data with the Bayesian Mallows Model

Clicking data, which exists in abundance and contains objective user preference information, is widely used to produce personalized recommendations in web-based applications. Current popular recommendation algorithms, typically based on matrix factorizations, often have high accuracy and achieve good clickthrough rates. However, diversity of the recommended items, which can greatly enhance user experiences, is often overlooked. Moreover, most algorithms do not produce interpretable uncertainty quantifications of the recommendations. In this work, we propose the Bayesian Mallows for Clicking Data (BMCD) method, which augments clicking data into compatible full ranking vectors by enforcing all the clicked items to be top-ranked. User preferences are learned using a Mallows ranking model. Bayesian inference leads to interpretable uncertainties of each individual recommendation, and we also propose a method to make personalized recommendations based on such uncertainties. With a simulation study and a real life data example, we demonstrate that compared to state-of-the-art matrix factorization, BMCD makes personalized recommendations with similar accuracy, while achieving much higher level of diversity, and producing interpretable and actionable uncertainty estimation.Comment: 27 page

JoVA-hinge: joint variational autoencoders for personalized recommendation with implicit feedback

Recently, Variational Autoencoders (VAEs) have shown remarkable performance in collaborative filtering (CF) with implicit feedback. These existing recommendation models learn user representations to reconstruct or predict user preferences. However, existing VAE-based recommendation models learn user and item representations separately. This thesis introduces joint variational autoencoders (JoVA). JoVA, as an ensemble of two VAEs, simultaneously and jointly learns both user-user and item-item correlations and collectively reconstructs and predicts user preferences. Moreover, a variant of JoVA, referred to as JoVA-Hinge, is introduced to improve recommendation quality. JoVA-Hinge incorporates pairwise ranking loss to VAE's losses. Extensive experiments on multiple real-world datasets show that our model can outperform state-of-the-art under a variety of commonly-used metrics. Our empirical experiments also confirm that JoVA-Hinge offers better results than existing methods for cold-start users with limited training data

JoVA-hinge: joint variational autoencoders for personalized recommendation with implicit feedback

Recently, Variational Autoencoders (VAEs) have shown remarkable performance in collaborative filtering (CF) with implicit feedback. These existing recommendation models learn user representations to reconstruct or predict user preferences. However, existing VAE-based recommendation models learn user and item representations separately. This thesis introduces joint variational autoencoders (JoVA). JoVA, as an ensemble of two VAEs, simultaneously and jointly learns both user-user and item-item correlations and collectively reconstructs and predicts user preferences. Moreover, a variant of JoVA, referred to as JoVA-Hinge, is introduced to improve recommendation quality. JoVA-Hinge incorporates pairwise ranking loss to VAE's losses. Extensive experiments on multiple real-world datasets show that our model can outperform state-of-the-art under a variety of commonly-used metrics. Our empirical experiments also confirm that JoVA-Hinge offers better results than existing methods for cold-start users with limited training data

Sequential Inference with the Mallows Model

The Mallows model is a widely used probabilistic model for analysing rank data. It assumes that a collection of n items can be ranked by each assessor and then summarised as a permutation of size n. The associated probability distribution is defined on the permutation space of these items. A hierarchical Bayesian framework for the Mallows model, named the Bayesian Mallows model, has been developed recently to perform inference and to provide uncertainty estimates of the model parameters. This framework typically uses Markov chain Monte Carlo (MCMC) methods to simulate from the target posterior distribution. However, MCMC can be considerably slow when additional computational effort is presented in the form of new ranking data. It can therefore be difficult to update the Bayesian Mallows model in real time. This thesis extends the Bayesian Mallows model to allow for sequential updates of its posterior estimates each time a collection of new preference data is observed. The posterior is updated over a sequence of discrete time steps with fixed computational complexity. This can be achieved using Sequential Monte Carlo (SMC) methods. SMC offers a standard alternative to MCMC by constructing a sequence of posterior distributions using a set of weighted samples. The samples are propagated via a combination of importance sampling, resampling and moving steps. We propose an SMC framework that can perform sequential updates for the posterior distribution for both a single Mallows model and a Mallows mixture each time we observe new full rankings in an online setting. We also construct a framework to conduct SMC with partial rankings for a single Mallows model. We propose an alternative proposal distribution for data augmentation in partial rankings that incorporates the current posterior estimates of the Mallows model parameters in each SMC iteration. We also extend the framework to consider how the posterior is updated when known assessors provide additional information in their partial ranking. We show how these corrections in the latent information are performed to account for the changes in the posterior

Network Structures, Concurrency, and Interpretability: Lessons from the Development of an AI Enabled Graph Database System

This thesis describes the development of the SmartGraph, an AI enabled graph database. The need for such a system has been independently recognized in the isolated fields of graph databases, graph computing, and computational graph deep learning systems, such as TensorFlow. Though prior works have investigated some relationships between these fields, we believe that the SmartGraph is the first system designed from conception to incorporate the most significant and useful characteristics of each. Examples include the ability to store graph structured data, run analytics natively on this data, and run gradient descent algorithms. It is the synergistic aspects of combining these fields that provide the most novel results presented in this dissertation. Key among them is how the notion of “graph querying” as used in graph databases can be used to solve a problem that has plagued deep learning systems since their inception; rather than attempting to embed graph structured datasets into restrictive vector spaces, we instead allow the deep learning functionality of the system to natively perform graph querying in memory during optimization as a way of interpreting (and learning) the graph. This results in a concept of natural and interpretable processing of graph structured data. Graph computing systems have traditionally used distributed computing across multiple compute nodes (e.g. separate machines connected via Ethernet or internet) to deal with large-scale datasets whilst working sequentially on problems over entire datasets. In this dissertation, we outline a distributed graph computing methodology that facilitates all the above capabilities (even in an environment consisting of a single physical machine) while allowing for a workflow more typical of a graph database than a graph computing system; massive concurrent access allowing for arbitrarily asynchronous execution of queries and analytics across the entire system. Further, we demonstrate how this methodology is key to the artificial intelligence capabilities of the system

Intelligent, Item-Based Stereotype Recommender System

Recommender systems (RS) have become key components driving the success of e-commerce, and other platforms where revenue and customer satisfaction is dependent on the user’s ability to discover desirable items in large catalogues. As the number of users and items on a platform grows, the computational complexity, the vastness of the data, and the sparsity problem constitute important challenges for any recommendation algorithm. In addition, the most widely studied filtering-based RS, while effective in providing suggestions for established users and items, are known for their poor performance for the new user and new item (cold start) problems. Stereotypical modelling of users and items is a promising approach to solving these problems. A stereotype represents an aggregation of the characteristics of the items or users which can be used to create general user or item classes. This work propose a set of methodologies for the automatic generation of stereotypes during the cold-starts. The novelty of the proposed approach rests on the findings that stereotypes built independently of the user-to-item ratings improve both recommendation metrics and computational performance during cold-start phases. The resulting RS can be used with any machine learning algorithm as a solver, and the improved performance gains due to rate-agnostic stereotypes are orthogonal to the gains obtained using more sophisticated solvers. Recommender Systems using the primitive metadata features (baseline systems) as well as factorisation-based systems are used as benchmarks for state-of-the-art methodologies to assess the results of the proposed approach under a wide range of recommendation quality metrics. The results demonstrate how such generic groupings of the metadata features, when performed in a manner that is unaware and independent of the user’s community preferences, may greatly reduce the dimension of the recommendation model, and provide a framework that improves the quality of recommendations in the cold start