31 research outputs found
Auto-Surprise: An Automated Recommender-System (AutoRecSys) Library with Tree of Parzens Estimator (TPE) Optimization
We introduce Auto-Surprise, an Automated Recommender System library.
Auto-Surprise is an extension of the Surprise recommender system library and
eases the algorithm selection and configuration process. Compared to
out-of-the-box Surprise library, Auto-Surprise performs better when evaluated
with MovieLens, Book Crossing and Jester Datasets. It may also result in the
selection of an algorithm with significantly lower runtime. Compared to
Surprise's grid search, Auto-Surprise performs equally well or slightly better
in terms of RMSE, and is notably faster in finding the optimum hyperparameters.Comment: To be presented at RecSys '20 Fourteenth ACM Conference on
Recommender Systems, September 21-26, 2020, Virtual Even
Improving accountability in recommender systems research through reproducibility
Reproducibility is a key requirement for scientific progress. It allows the reproduction of the works of others, and, as a consequence, to fully trust the reported claims and results. In this work, we argue that, by facilitating reproducibility of recommender systems experimentation, we indirectly address the issues of accountability and transparency in recommender systems research from the perspectives of practitioners, designers, and engineers aiming to assess the capabilities of published research works. These issues have become increasingly prevalent in recent literature. Reasons for this include societal movements around intelligent systems and artificial intelligence striving toward fair and objective use of human behavioral data (as in Machine Learning, Information Retrieval, or Human–Computer Interaction). Society has grown to expect explanations and transparency standards regarding the underlying algorithms making automated decisions for and around us. This work surveys existing definitions of these concepts and proposes a coherent terminology for recommender systems research, with the goal to connect reproducibility to accountability. We achieve this by introducing several guidelines and steps that lead to reproducible and, hence, accountable experimental workflows and research. We additionally analyze several instantiations of recommender system implementations available in the literature and discuss the extent to which they fit in the introduced framework. With this work, we aim to shed light on this important problem and facilitate progress in the field by increasing the accountability of researchThis work has been funded by the Ministerio de Ciencia, Innovación y Universidades (reference: PID2019-108965GB-I00
The Potential of AutoML for Recommender Systems
Automated Machine Learning (AutoML) has greatly advanced applications of
Machine Learning (ML) including model compression, machine translation, and
computer vision. Recommender Systems (RecSys) can be seen as an application of
ML. Yet, AutoML has found little attention in the RecSys community; nor has
RecSys found notable attention in the AutoML community. Only few and relatively
simple Automated Recommender Systems (AutoRecSys) libraries exist that adopt
AutoML techniques. However, these libraries are based on student projects and
do not offer the features and thorough development of AutoML libraries. We set
out to determine how AutoML libraries perform in the scenario of an
inexperienced user who wants to implement a recommender system. We compared the
predictive performance of 60 AutoML, AutoRecSys, ML, and RecSys algorithms from
15 libraries, including a mean predictor baseline, on 14 explicit feedback
RecSys datasets. To simulate the perspective of an inexperienced user, the
algorithms were evaluated with default hyperparameters. We found that AutoML
and AutoRecSys libraries performed best. AutoML libraries performed best for
six of the 14 datasets (43%), but it was not always the same AutoML library
performing best. The single-best library was the AutoRecSys library
Auto-Surprise, which performed best on five datasets (36%). On three datasets
(21%), AutoML libraries performed poorly, and RecSys libraries with default
parameters performed best. Although, while obtaining 50% of all placements in
the top five per dataset, RecSys algorithms fall behind AutoML on average. ML
algorithms generally performed the worst
Recommended from our members
Controlling the Fairness / Accuracy Tradeoff in Recommender Systems
Recommender systems are one of the most pervasive applications of machine learning. They play a pivotal role in helping users find items tailored to their taste. Although these systems intend to assist people in their information needs, they can cause implicit or explicit discrimination against individuals or groups. There are several ways that different biases can creep into recommender systems. Reflection of societal and historical prejudices in datasets and during the data collection process, lack of sufficient data on minority groups, lack of suitable evaluation methods and model designs to detect these biases and lessen the unfairness caused by them are among the many reasons for unfairness in these systems. A system needs to defend against the biases in recommendation output to prevent harm and unfairness. However, integrating the goal of fairness with accuracy in recommender systems is challenging, primarily because of this goal's significant trade-offs with accuracy. Accuracy in recommender systems is the ability of that system to predict users' needs and interests accurately. On the other hand, fairness is a complicated concept with a variety of definitions. To use fairness as an objective, we need to define it based on the application area and the context of a problem. Additionally, we need to specify the fairness concerns of the different stakeholders involved in the recommender systems and the fairness priorities of a system. Any of these aspects might disagree with the goal of accuracy. For example, if fairness for content providers is more exposure to users, increasing it might cause a reduction in accuracy. Therefore, controlling the trade-off between accuracy and fairness becomes essential. Throughout this dissertation, several recommendation models and re-ranking approaches are presented that aim to address this problem using in- and post- processing methods. These approaches show promising results, but it is worth mentioning that they have intrinsic limitations and, therefore, shouldn't be considered ultimate solutions
Understanding and Mitigating Multi-sided Exposure Bias in Recommender Systems
Fairness is a critical system-level objective in recommender systems that has
been the subject of extensive recent research. It is especially important in
multi-sided recommendation platforms where it may be crucial to optimize
utilities not just for the end user, but also for other actors such as item
sellers or producers who desire a fair representation of their items. Existing
solutions do not properly address various aspects of multi-sided fairness in
recommendations as they may either solely have one-sided view (i.e. improving
the fairness only for one side), or do not appropriately measure the fairness
for each actor involved in the system. In this thesis, I aim at first
investigating the impact of unfair recommendations on the system and how these
unfair recommendations can negatively affect major actors in the system. Then,
I seek to propose solutions to tackle the unfairness of recommendations. I
propose a rating transformation technique that works as a pre-processing step
before building the recommendation model to alleviate the inherent popularity
bias in the input data and consequently to mitigate the exposure unfairness for
items and suppliers in the recommendation lists. Also, as another solution, I
propose a general graph-based solution that works as a post-processing approach
after recommendation generation for mitigating the multi-sided exposure bias in
the recommendation results. For evaluation, I introduce several metrics for
measuring the exposure fairness for items and suppliers, and show that these
metrics better capture the fairness properties in the recommendation results. I
perform extensive experiments to evaluate the effectiveness of the proposed
solutions. The experiments on different publicly-available datasets and
comparison with various baselines confirm the superiority of the proposed
solutions in improving the exposure fairness for items and suppliers.Comment: Doctoral thesi
Fairness of Exposure in Dynamic Recommendation
Exposure bias is a well-known issue in recommender systems where the exposure
is not fairly distributed among items in the recommendation results. This is
especially problematic when bias is amplified over time as a few items (e.g.,
popular ones) are repeatedly over-represented in recommendation lists and
users' interactions with those items will amplify bias towards those items over
time resulting in a feedback loop. This issue has been extensively studied in
the literature in static recommendation environment where a single round of
recommendation result is processed to improve the exposure fairness. However,
less work has been done on addressing exposure bias in a dynamic recommendation
setting where the system is operating over time, the recommendation model and
the input data are dynamically updated with ongoing user feedback on
recommended items at each round. In this paper, we study exposure bias in a
dynamic recommendation setting. Our goal is to show that existing bias
mitigation methods that are designed to operate in a static recommendation
setting are unable to satisfy fairness of exposure for items in long run. In
particular, we empirically study one of these methods and show that repeatedly
applying this method fails to fairly distribute exposure among items in long
run. To address this limitation, we show how this method can be adapted to
effectively operate in a dynamic recommendation setting and achieve exposure
fairness for items in long run. Experiments on a real-world dataset confirm
that our solution is superior in achieving long-term exposure fairness for the
items while maintaining the recommendation accuracy