18,235 research outputs found
A Distributed and Accountable Approach to Offline Recommender Systems Evaluation
Different software tools have been developed with the purpose of performing
offline evaluations of recommender systems. However, the results obtained with
these tools may be not directly comparable because of subtle differences in the
experimental protocols and metrics. Furthermore, it is difficult to analyze in
the same experimental conditions several algorithms without disclosing their
implementation details. For these reasons, we introduce RecLab, an open source
software for evaluating recommender systems in a distributed fashion. By
relying on consolidated web protocols, we created RESTful APIs for training and
querying recommenders remotely. In this way, it is possible to easily integrate
into the same toolkit algorithms realized with different technologies. In
details, the experimenter can perform an evaluation by simply visiting a web
interface provided by RecLab. The framework will then interact with all the
selected recommenders and it will compute and display a comprehensive set of
measures, each representing a different metric. The results of all experiments
are permanently stored and publicly available in order to support
accountability and comparative analyses.Comment: REVEAL 2018 Workshop on Offline Evaluation for Recommender System
- …