A General Method for the Discovery and Use of Rules Induced by Feedforward Artificial Neural Networks

Abstract

GR2 is a hybrid connectionist/knowledge-based system consisting of a Multi-layer Perceptron and a rule-based system for knowledge representation and reasoning. Knowledge embedded in a trained Multi-layer Perceptron is extracted in the form of general (production) rules--a natural format for abstract knowledge representation. The rule extraction method integrates black-box and white-box techniques on the MLP, obtaining feature salient and statistical properties of the training pattern set. This is achieved via a heuristic based on the static connections strengths from input to output. The extracted general rules are quantified and selected in a rule validation process. Multiple inference modalities such as categorical reasoning, probablistic reasoning and exceptional reasoning can be performed in GR2. In addition, quantitative indications of a rule's validity within the domain and the importance of any antecedent within a rule can be calculated. Experiments are conducted in artificial (simple logic) domains and using data from emergency medicine. The predictive performance of the underlying neural networks is seen to be maintained whilst a valid set of rules is extracted. For the medical problem, favourable comparison is drawn with the C4.5 technique, an extension of the celebrated ID3 algorithm. The methodology can be applied to any feedforward neural network via straightforward extensions to the basic ideas and avoids the need for specialised architectures found in some other methods