Partial coordination is a new method for cataloging documents for
subject access. It is especially designed to enhance the precision of document
searches in online environments. This paper reports a preliminary
evaluation of partial coordination which shows promising results compared
with full text retrieval. We also report the difficulties in empirically
evaluating the effectiveness of automatic full-text retrieval in contrast to
mixed methods such as partial coordination which combine human
cataloging with computerized retrieval. Based on our study we propose
research in this area will substantially benefit from a common framework for
failure analysis and a common data set. This will allow information retrieval
researchers adapting "library style" cataloging to large electronic document
collections, as well as those developing automated or mixed methods, to
directly compare their proposals for indexing and retrieval. This paper
concludes by suggesting guidelines for constructing such a testbed.Information Systems Working Papers Serie