Benchmarking News Recommendations in a Living Lab

Most user-centric studies of information access systems in literature suffer from unrealistic settings or limited numbers of users who participate in the study. In order to address this issue, the idea of a living lab has been promoted. Living labs allow us to evaluate research hypotheses using a large number of users who satisfy their information need in a real context. In this paper, we introduce a living lab on news recommendation in real time. The living lab has first been organized as News Recommendation Challenge at ACM RecSys’13 and then as campaign-style evaluation lab NEWSREEL at CLEF’14. Within this lab, researchers were asked to provide news article recommendations to millions of users in real time. Different from user studies which have been performed in a laboratory, these users are following their own agenda. Consequently, laboratory bias on their behavior can be neglected. We outline the living lab scenario and the experimental setup of the two benchmarking events. We argue that the living lab can serve as reference point for the implementation of living labs for the evaluation of information access systems

Social Navigation in a Location-Based Information System

Much of contextaware application research has dealt with the technical aspects of context capturing and how to interpret the context of a user. Little effort has been spent on the experience and usage of these systems. This thesis will present the general aspects of social awareness and present an example on how these concepts can be implemented into a location-based information system to help users navigate a potential information overload. This thesis also states that giving the users an experience of not being alone in the system increases the pleasure of using such a system. However this implies a decrease in privacy. To demonstrate these ideas I will describe a locationbased information system, GeoNotes, built by a group of researchers at SICS, the Swedish Institute of Computer Science. I will state a set of interaction requirements for how to extend the GeoNotes system with functionality for social awareness. Furthermore I will set up functional requirements for those interaction requirements to after implementation be able to conclude which interaction requirements I have been able to implement for. I will also give suggestions on how to position users in a WLAN. The deliverable from this project is a locationbased information system with functionality for social awareness. However, it was not within this project to test the system on true users. Therefore the statement that this functionality can help users to navigate a potential information overload is still just a hypothesis. To retrieve the position of a user in a W-LAN a packet is sent to all base stations in the network. In the first returning packet the mac address of contacting base station is extracted. Each base station is therefore a unique position. Triangulation was discarded due to its sensitivity to noise and weather circumstances, although a system that uses triangulation would have offered a much higher granularity

Fast Differentially Private Matrix Factorization

Differentially private collaborative filtering is a challenging task, both in terms of accuracy and speed. We present a simple algorithm that is provably differentially private, while offering good performance, using a novel connection of differential privacy to Bayesian posterior sampling via Stochastic Gradient Langevin Dynamics. Due to its simplicity the algorithm lends itself to efficient implementation. By careful systems design and by exploiting the power law behavior of the data to maximize CPU cache bandwidth we are able to generate 1024 dimensional models at a rate of 8.5 million recommendations per second on a single PC

A personalized and context-aware news offer for mobile devices

For classical domains, such as movies, recommender systems have proven their usefulness. But recommending news is more challenging due to the short life span of news content and the demand for up-to-date recommendations. This paper presents a news recommendation service with a content-based algorithm that uses features of a search engine for content processing and indexing, and a collaborative filtering algorithm for serendipity. The extension towards a context-aware algorithm is made to assess the information value of context in a mobile environment through a user study. Analyzing interaction behavior and feedback of users on three recommendation approaches shows that interaction with the content is crucial input for user modeling. Context-aware recommendations using time and device type as context data outperform traditional recommendations with an accuracy gain dependent on the contextual situation. These findings demonstrate that the user experience of news services can be improved by a personalized context-aware news offer

Budget-Constrained Item Cold-Start Handling in Collaborative Filtering Recommenders via Optimal Design

It is well known that collaborative filtering (CF) based recommender systems provide better modeling of users and items associated with considerable rating history. The lack of historical ratings results in the user and the item cold-start problems. The latter is the main focus of this work. Most of the current literature addresses this problem by integrating content-based recommendation techniques to model the new item. However, in many cases such content is not available, and the question arises is whether this problem can be mitigated using CF techniques only. We formalize this problem as an optimization problem: given a new item, a pool of available users, and a budget constraint, select which users to assign with the task of rating the new item in order to minimize the prediction error of our model. We show that the objective function is monotone-supermodular, and propose efficient optimal design based algorithms that attain an approximation to its optimum. Our findings are verified by an empirical study using the Netflix dataset, where the proposed algorithms outperform several baselines for the problem at hand.Comment: 11 pages, 2 figure

Evolution of Ego-networks in Social Media with Link Recommendations

Ego-networks are fundamental structures in social graphs, yet the process of their evolution is still widely unexplored. In an online context, a key question is how link recommender systems may skew the growth of these networks, possibly restraining diversity. To shed light on this matter, we analyze the complete temporal evolution of 170M ego-networks extracted from Flickr and Tumblr, comparing links that are created spontaneously with those that have been algorithmically recommended. We find that the evolution of ego-networks is bursty, community-driven, and characterized by subsequent phases of explosive diameter increase, slight shrinking, and stabilization. Recommendations favor popular and well-connected nodes, limiting the diameter expansion. With a matching experiment aimed at detecting causal relationships from observational data, we find that the bias introduced by the recommendations fosters global diversity in the process of neighbor selection. Last, with two link prediction experiments, we show how insights from our analysis can be used to improve the effectiveness of social recommender systems.Comment: Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM 2017), Cambridge, UK. 10 pages, 16 figures, 1 tabl

Shedding light on a living lab: the CLEF NEWSREEL open recommendation platform

In the CLEF NEWSREEL lab, participants are invited to evaluate news recommendation techniques in real-time by providing news recommendations to actual users that visit commercial news portals to satisfy their information needs. A central role within this lab is the communication between participants and the users. This is enabled by The Open Recommendation Platform (ORP), a web-based platform which distributes users' impressions of news articles to the participants and returns their recommendations to the readers. In this demo, we illustrate the platform and show how requests are handled to provide relevant news articles in real-time

Transfer Learning for Content-Based Recommender Systems using Tree Matching

In this paper we present a new approach to content-based transfer learning for solving the data sparsity problem in cases when the users' preferences in the target domain are either scarce or unavailable, but the necessary information on the preferences exists in another domain. We show that training a system to use such information across domains can produce better performance. Specifically, we represent users' behavior patterns based on topological graph structures. Each behavior pattern represents the behavior of a set of users, when the users' behavior is defined as the items they rated and the items' rating values. In the next step we find a correlation between behavior patterns in the source domain and behavior patterns in the target domain. This mapping is considered a bridge between the two domains. Based on the correlation and content-attributes of the items, we train a machine learning model to predict users' ratings in the target domain. When we compare our approach to the popularity approach and KNN-cross-domain on a real world dataset, the results show that on an average of 83$$ of the cases our approach outperforms both methods