Replicable Evaluation of Recommender Systems

This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in RecSys '15 Proceedings of the 9th ACM Conference on Recommender Systems, http://dx.doi.org/10.1145/2792838.2792841.Recommender systems research is by and large based on comparisons of recommendation algorithms’ predictive accuracies: the better the evaluation metrics (higher accuracy scores or lower predictive errors), the better the recommendation algorithm. Comparing the evaluation results of two recommendation approaches is however a difficult process as there are very many factors to be considered in the implementation of an algorithm, its evaluation, and how datasets are processed and prepared. This tutorial shows how to present evaluation results in a clear and concise manner, while ensuring that the results are comparable, replicable and unbiased. These insights are not limited to recommender systems research alone, but are also valid for experiments with other types of personalized interactions and contextual information access.Supported in part by the Ministerio de Educación y Ciencia (TIN2013-47090-C3-2)

Clemson University’s Teacher Learning Progression Program: Personalized Advanced Credentials for Teachers

This chapter provides an overview of Clemson University\u27s Teacher Learning Progression program, which offers participating middle school science, technology, engineering, and/or mathematics (STEM) teachers with personalized advanced credentials. In contrast to typical professional development (PD) approaches, this program identifies individualized pathways for PD based on teachers\u27 unique interests and needs and offers PD options through the use of a “recommender system”—a system providing context-specific recommendations to guide teachers toward the identification of preferred PD pathways and content. In this chapter, the authors introduce the program and highlight (1) the data collection and instrumentation needed to make personalized PD recommendations, (2) the recommender system, and (3) the personalized advanced credential options. The authors also discuss lessons learned through initial stages of project implementation and consider future directions for the use of recommender systems to support teacher PD, considering both research and applied implications and settings

QUARE: 1st Workshop on Measuring the Quality of Explanations in Recommender Systems

QUARE - measuring the QUality of explAnations in REcommender systems - is the first workshop that aims to promote discussion upon future research and practice directions around evaluation methodologies for explanations in recommender systems. To that end, we bring together researchers and practitioners from academia and industry to facilitate discussions about the main issues and best practices in the respective areas, identify possible synergies, and outline priorities regarding future research directions. Additionally, we want to stimulate reflections around methods to systematically and holistically assess explanation approaches, impact, and goals, at the interplay between organisational and human values. The homepage of the workshop is available at: https: //sites.google.com/view/quare-2022/

Beyond Personalization: Research Directions in Multistakeholder Recommendation

Recommender systems are personalized information access applications; they are ubiquitous in today's online environment, and effective at finding items that meet user needs and tastes. As the reach of recommender systems has extended, it has become apparent that the single-minded focus on the user common to academic research has obscured other important aspects of recommendation outcomes. Properties such as fairness, balance, profitability, and reciprocity are not captured by typical metrics for recommender system evaluation. The concept of multistakeholder recommendation has emerged as a unifying framework for describing and understanding recommendation settings where the end user is not the sole focus. This article describes the origins of multistakeholder recommendation, and the landscape of system designs. It provides illustrative examples of current research, as well as outlining open questions and research directions for the field.Comment: 64 page