This paper describes measures for evaluating the three determinants of how
well a probabilistic classifier performs on a given test set. These
determinants are the appropriateness, for the test set, of the results of (1)
feature selection, (2) formulation of the parametric form of the model, and (3)
parameter estimation. These are part of any model formulation procedure, even
if not broken out as separate steps, so the tradeoffs explored in this paper
are relevant to a wide variety of methods. The measures are demonstrated in a
large experiment, in which they are used to analyze the results of roughly 300
classifiers that perform word-sense disambiguation.Comment: 12 pages, uuencoded compressed postscript fil