Modeling rule precision

Abstract. This paper reports first results of an empirical study of the precision of classification rules on an independent test set. We generated a large number of rules using a general covering algorithm and recorded their coverage on training and test sets. These meta data are briefly presented and analyzed with respect to their variance among different domains and search heuristics. The main part of the paper describes experiments that aimed at modeling the precision of the learned rules on the test set in dependence of their coverage on the training set. To this end, we trained a neural network as an evaluation function for a rule learner, and learned optimal parameter settings for the m-heuristic and the generalized m-heuristic. These settings are optimal in the sense that they minimize the squared error of predicting the test set precision with training set coverage of positive and negative examples.