Measuring Generality of Documents

Most traditional Information Retrieval (IR) systems, including web search engines, operationalize “relevant” as the word frequency in a document of a set of keywords. Because of this limitation, traditional IR systems frequently retrieve irrelevant documents in response to a user’s request. In this paper, we propose a new criterion, “generality, ” that provides an additional basis on which to rank retrieved documents. The generality is a level of abstraction to retrieve results based on desired generality appropriate for a user’s knowledge and interests. We compared our generality quantification algorithm with human judges ’ weighting of values to show that the developed algorithm is significantly correlated. 1

A NEW CRITERION FOR MEASURING GENERALITY OF DOCUMENTS

ABSTRACT: Most information retrieval systems, including Web search engines, use similarity ranking algorithms based on a vector space model to find relevant information in response to a user’s request. However, the retrieved information is frequently irrelevant, because most of the current information systems employ index terms or other techniques that are variants of term frequency. In this paper, we propose a new criterion, “generality, ” that provides an additional basis on which to rank retrieved documents. We compared our generality quantification algorithm with human judges’ weighting of values to show that the developed algorithm is significantly correlated

Generality: A New Criterion for Measuring Generality of Documents

Most information retrieval systems, including Web search engines, use similarity ranking algorithms based on a vector space model to find relevant information in response to a user’s request. However, the retrieved information is frequently irrelevant, because most of the current information systems employ index terms or other techniques that are variants of term frequency. In this paper, we propose a new criterion, “generality, ” that provides an additional basis on which to rank retrieved documents. We compared our generality quantification algorithm with human judges’ weighting of values to show that the developed algorithm is significantly correlated. 1