Opportunistic constructive induction: Using fragments of domain knowledge to guide construction

One subfield of machine learning is the induction of a representation of a concept from positive and negative examples of the concept. Given a set of training examples, the goal of the inductive system is to create a description capable of classifying the training examples, yet general enough to accurately predict the classification of unseen examples. Often the original attributes describing the instances are inadequate to capture important regularities in the concept. New descriptors, constructed through the application of operators to the original attributes, can provide the proper vocabulary to create concise concept representations at the right level of generalization to be highly predictive. Constructive induction is the process of generating and applying new descriptors during inductive learning.The large number of possible constructive operators and combinations of attributes defines an enormous search space for the inductive process. Knowledge about the concept or problem domain can be used to guide the construction of new descriptors. This thesis lays the foundation of opportunistic constructive induction in the context of decision-tree assembly, providing a framework for dynamically applying fragments of knowledge to produce potentially useful descriptors or hypotheses. A two-staged process of generating candidate descriptors (hypothesis generation) and focusing the induction on the most promising (hypothesis ordering) has been developed and partially implemented. This thesis concentrates on the development of a hypothesis ordering mechanism that incorporates the evaluation of multiple objectives to identify the most promising descriptors. Experiments in four test domains demonstrate the hypothesis ordering mechanism to be a robust, effective method of significantly reducing the potential computational burden created by prolific hypothesis generation. In addition, preliminary investigation of hypothesis generation indicates that small amounts of knowledge can provide substantial increases in the predictive accuracy of the induced decision-trees.U of I OnlyETDs are only available to UIUC Users without author permissio