Location of Repository

Relevance Judgments between TREC and Non-TREC Assessors

By A. Al-Maskari, M. Sanderson and P. Clough

Abstract

This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment. Results show that 63% of documents judged relevant by our users matched official TREC judgments. Several factors contributed to differences in the agreements: the number of retrieved relevant documents; the number of relevant documents judged; system effectiveness per topic and the ranking of relevant documents

Publisher: ACM
Year: 2008
OAI identifier: oai:eprints.whiterose.ac.uk:4510

Suggested articles

Preview


To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.