4 research outputs found
Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks
This paper presents a generalized technology of extraction of explicit
knowledge from data. The main ideas are 1) maximal reduction of network
complexity (not only removal of neurons or synapses, but removal all the
unnecessary elements and signals and reduction of the complexity of elements),
2) using of adjustable and flexible pruning process (the pruning sequence
shouldn't be predetermined - the user should have a possibility to prune
network on his own way in order to achieve a desired network structure for the
purpose of extraction of rules of desired type and form), and 3) extraction of
rules not in predetermined but any desired form. Some considerations and notes
about network architecture and training process and applicability of currently
developed pruning techniques and rule extraction algorithms are discussed. This
technology, being developed by us for more than 10 years, allowed us to create
dozens of knowledge-based expert systems. In this paper we present a
generalized three-step technology of extraction of explicit knowledge from
empirical data.Comment: 9 pages, The talk was given at the IJCNN '99 (Washington DC, July
1999
Artificial Neural Network Pruning to Extract Knowledge
Artificial Neural Networks (NN) are widely used for solving complex problems
from medical diagnostics to face recognition. Despite notable successes, the
main disadvantages of NN are also well known: the risk of overfitting, lack of
explainability (inability to extract algorithms from trained NN), and high
consumption of computing resources. Determining the appropriate specific NN
structure for each problem can help overcome these difficulties: Too poor NN
cannot be successfully trained, but too rich NN gives unexplainable results and
may have a high chance of overfitting. Reducing precision of NN parameters
simplifies the implementation of these NN, saves computing resources, and makes
the NN skills more transparent. This paper lists the basic NN simplification
problems and controlled pruning procedures to solve these problems. All the
described pruning procedures can be implemented in one framework. The developed
procedures, in particular, find the optimal structure of NN for each task,
measure the influence of each input signal and NN parameter, and provide a
detailed verbal description of the algorithms and skills of NN. The described
methods are illustrated by a simple example: the generation of explicit
algorithms for predicting the results of the US presidential election.Comment: IJCNN 202