Regression and Learning to Rank Aggregation for User Engagement Evaluation

User engagement refers to the amount of interaction an instance (e.g., tweet, news, and forum post) achieves. Ranking the items in social media websites based on the amount of user participation in them, can be used in different applications, such as recommender systems. In this paper, we consider a tweet containing a rating for a movie as an instance and focus on ranking the instances of each user based on their engagement, i.e., the total number of retweets and favorites it will gain. For this task, we define several features which can be extracted from the meta-data of each tweet. The features are partitioned into three categories: user-based, movie-based, and tweet-based. We show that in order to obtain good results, features from all categories should be considered. We exploit regression and learning to rank methods to rank the tweets and propose to aggregate the results of regression and learning to rank methods to achieve better performance. We have run our experiments on an extended version of MovieTweeting dataset provided by ACM RecSys Challenge 2014. The results show that learning to rank approach outperforms most of the regression models and the combination can improve the performance significantly.Comment: In Proceedings of the 2014 ACM Recommender Systems Challenge, RecSysChallenge '1

A Hybrid Multi-strategy Recommender System Using Linked Open Data

In this paper, we discuss the development of a hybrid multi-strategy book recommendation system using Linked Open Data. Our approach builds on training individual base recommenders and using global popularity scores as generic recommenders. The results of the individual recommenders are combined using stacking regression and rank aggregation. We show that this approach delivers very good results in different recommendation settings and also allows for incorporating diversity of recommendations

Predicting User Engagement in Twitter with Collaborative Ranking

Collaborative Filtering (CF) is a core component of popular web-based services such as Amazon, YouTube, Netflix, and Twitter. Most applications use CF to recommend a small set of items to the user. For instance, YouTube presents to a user a list of top-n videos she would likely watch next based on her rating and viewing history. Current methods of CF evaluation have been focused on assessing the quality of a predicted rating or the ranking performance for top-n recommended items. However, restricting the recommender system evaluation to these two aspects is rather limiting and neglects other dimensions that could better characterize a well-perceived recommendation. In this paper, instead of optimizing rating or top-n recommendation, we focus on the task of predicting which items generate the highest user engagement. In particular, we use Twitter as our testbed and cast the problem as a Collaborative Ranking task where the rich features extracted from the metadata of the tweets help to complement the transaction information limited to user ids, item ids, ratings and timestamps. We learn a scoring function that directly optimizes the user engagement in terms of nDCG@10 on the predicted ranking. Experiments conducted on an extended version of the MovieTweetings dataset, released as part of the RecSys Challenge 2014, show the effectiveness of our approach.Comment: RecSysChallenge'14 at RecSys 2014, October 10, 2014, Foster City, CA, US

Interactive Content Diversity and User Exploration in Online Movie Recommenders: A Field Experiment

Recommender systems often struggle to strike a balance between matching users' tastes and providing unexpected recommendations. When recommendations are too narrow and fail to cover the full range of users' preferences, the system is perceived as useless. Conversely, when the system suggests too many items that users don't like, it is considered impersonal or ineffective. To better understand user sentiment about the breadth of recommendations given by a movie recommender, we conducted interviews and surveys and found out that many users considered narrow recommendations to be useful, while a smaller number explicitly wanted greater breadth. Additionally, we designed and ran an online field experiment with a larger user group, evaluating two new interfaces designed to provide users with greater access to broader recommendations. We looked at user preferences and behavior for two groups of users: those with higher initial movie diversity and those with lower diversity. Among our findings, we discovered that different level of exploration control and users' subjective preferences on interfaces are more predictive of their satisfaction with the recommender.Comment: International Journal of Human Computer Interactio

Data-Driven Recommender Systems: Sequences of recommendations

This document is about some scalable and reliable methods for recommender systems from a machine learner point of view. In particular it adresses some difficulties from the non stationary case

Inferring User Needs & Tasks from App Usage Interactions

Mobile devices have become an increasingly ubiquitous part of our everyday life, which are not only used for basic communication. Nowadays, the need for mobile services arises from a broad range of requirements including both single app usage (e.g., check on the weather) and complex task completion (such as planning vacation) which may lead to lengthy operations within distinct apps. Understanding how users interact with apps could provide us with great signals for profiling users and help service providers/app developers/smartphone manufacturers to improve user experience and retention. Therefore, in this thesis, we present work towards inferring user needs and tasks from their app usage interactions. Firstly, we aim to better understand users' behaviour on using one particular app under different contexts. There have been many researchers proposed models for recommending the app user would use next proactively. However, less work has been conducted to enhance the app usage prediction when a new user comes whose information is insufficient for learning. Additionally, besides predicting which app users would use, we aim to further investigate if the app dwell time could also be modelled based on various user characteristics and contextual information. By conducting the comprehensive analysis and experiments, we demonstrate that users' next app and the time spent could be effectively predicted at the same time. Other than effectively serving the individual apps that correspond to users' simple needs, we aim to further understand the high-level tasks within users' minds while engaging with different apps. We focus on identifying and characterizing tasks from app usage behaviour and then leveraging the extracted task information for improving mobile services. We first present an automatic method that accurately determines mobile tasks from users' app usage logs based on a set of features. Given the extracted tasks, we further investigate if there are common patterns that exist among all the complex mobile tasks. Finally, we demonstrate that the extracted task information could benefit user profiling in demographics prediction and next task prediction, especially when compared to the traditional app-based methods. To summarize, in this thesis, we conduct a more comprehensive study on modelling users app usage behaviour. Additionally, we propose to set the stage for evaluating mobile apps usage, not on a per-app basis, but on the basis of users' tasks. Finally, we provide the initial steps in shaping future research on investigating whether and how the extracted tasks could be applied for improving mobile services

Inferring User Needs & Tasks from App Usage Interactions

Mobile devices have become an increasingly ubiquitous part of our everyday life, which are not only used for basic communication. Nowadays, the need for mobile services arises from a broad range of requirements including both single app usage (e.g., check on the weather) and complex task completion (such as planning vacation) which may lead to lengthy operations within distinct apps. Understanding how users interact with apps could provide us with great signals for profiling users and help service providers/app developers/smartphone manufacturers to improve user experience and retention. Therefore, in this thesis, we present work towards inferring user needs and tasks from their app usage interactions. Firstly, we aim to better understand users' behaviour on using one particular app under different contexts. There have been many researchers proposed models for recommending the app user would use next proactively. However, less work has been conducted to enhance the app usage prediction when a new user comes whose information is insufficient for learning. Additionally, besides predicting which app users would use, we aim to further investigate if the app dwell time could also be modelled based on various user characteristics and contextual information. By conducting the comprehensive analysis and experiments, we demonstrate that users' next app and the time spent could be effectively predicted at the same time. Other than effectively serving the individual apps that correspond to users' simple needs, we aim to further understand the high-level tasks within users' minds while engaging with different apps. We focus on identifying and characterizing tasks from app usage behaviour and then leveraging the extracted task information for improving mobile services. We first present an automatic method that accurately determines mobile tasks from users' app usage logs based on a set of features. Given the extracted tasks, we further investigate if there are common patterns that exist among all the complex mobile tasks. Finally, we demonstrate that the extracted task information could benefit user profiling in demographics prediction and next task prediction, especially when compared to the traditional app-based methods. To summarize, in this thesis, we conduct a more comprehensive study on modelling users app usage behaviour. Additionally, we propose to set the stage for evaluating mobile apps usage, not on a per-app basis, but on the basis of users' tasks. Finally, we provide the initial steps in shaping future research on investigating whether and how the extracted tasks could be applied for improving mobile services

Exploring Tweet Engagement in the RecSys 2014 Data Challenge

8th ACM Conference on Recommender Systems, Foster City, Silicon Valley, USA, 6-10 October 2014While much recommender system research has been driven by the rating prediction task, there is an emphasis in recent research on exploring new methods to evaluate the effectiveness of a recommendation. The Recommender Systems Challenge 2014 takes up this theme by challenging re-searchers to explore engagement as an evaluation criterion.In this paper we discuss how predicting engagement differs from the traditional rating prediction task and motivate the rationale behind our approach to the challenge. We show that standard matrix factorization recommender algorithms do not perform well on the task. Our solution depends on clustering items according to their time-dependent profile to distinguish topical movies from other movies. Our pre-diction engine also exploits the observation that extreme ratings are more likely to attract engagement.Science Foundation IrelandInsight Research Centr