19,085 research outputs found
Overview of CLEF NEWSREEL 2014: News Recommendations Evaluation Labs
This paper summarises objectives, organisation, and results of the first
news recommendation evaluation lab (NEWSREEL 2014). NEWSREEL targeted
the evaluation of news recommendation algorithms in the form of a campaignstyle
evaluation lab. Participants had the chance to apply two types of evaluation
schemes. On the one hand, participants could apply their algorithms onto a data
set. We refer to this setting as off-line evaluation. On the other hand, participants
could deploy their algorithms on a server to interactively receive recommendation
requests. We refer to this setting as on-line evaluation. This setting ought to reveal
the actual performance of recommendation methods. The competition strived to
illustrate differences between evaluation with historical data and actual users. The
on-line evaluation does reflect all requirements which active recommender systems
face in practise. These requirements include real-time responses and large-scale
data volumes. We present the competition’s results and discuss commonalities
regarding participants’ approaches
Sequeval: A Framework to Assess and Benchmark Sequence-based Recommender Systems
In this paper, we present sequeval, a software tool capable of performing the
offline evaluation of a recommender system designed to suggest a sequence of
items. A sequence-based recommender is trained considering the sequences
already available in the system and its purpose is to generate a personalized
sequence starting from an initial seed. This tool automatically evaluates the
sequence-based recommender considering a comprehensive set of eight different
metrics adapted to the sequential scenario. sequeval has been developed
following the best practices of software extensibility. For this reason, it is
possible to easily integrate and evaluate novel recommendation techniques.
sequeval is publicly available as an open source tool and it aims to become a
focal point for the community to assess sequence-based recommender systems.Comment: REVEAL 2018 Workshop on Offline Evaluation for Recommender System
- …