Beyond relevance judgments : cognitive shifts and gratification

Abstract

Relevance judgments are an important element of evaluation for the field of information retrieval (IR). Recent studies also highlight the value of measuring users’ cognitive shifts and gratification judgments during interactive IR and Web searching. This paper argues for a more approach to interactive IR as a multi-level evaluation process, including users’ relevance judgments, cognitive shifts and gratification judgments.