An evaluation of novelty and diversity based on fuzzy logic

Abstract

Also published online by CEUR Workshop Proceedings (CEUR-WS.org, ISSN 1613-0073) Proceedings of the Workshop on Novelty and Diversity in Recommender Systems, DiveRS 2011Information retrieval systems are based on an estimation or prediction of the relevance of documents for certain topics associated to a query or, in the case of recommendation systems, for a certain user profile. Most systems use a graded relevance estimation (a.k.a. relevance status value), that is, a real value r(d,τ ) ∈ [0, 1] for the relevance of document d with respect to topic τ . In retrieval systems based on the Probability Ranking Principle [9], this value has a probabilistic interpretation, that is, r(d, τ ) is equivalent (in rank) to the probability that a user will consider the document relevant. We contend in this paper for an alternative interpretation, where the value r(d, τ ) is considered as the fuzzy truth value of the statement “d is relevant for τ”. We develop and evaluate two measures that determine the quality of a result set in terms of diversity and novelty based on this fuzzy interpretation