74,937 research outputs found
Extracting Symbolic Representations Learned by Neural Networks
Understanding what neural networks learn from training data is of great interest in data mining, data analysis, and critical applications, and in evaluating neural network models. Unfortunately, the product of neural network training is typically opaque matrices of floating point numbers that are not obviously understandable. This difficulty has inspired substantial past research on how to extract symbolic, human-readable representations from a trained neural network, but the results obtained so far are very limited (e.g., large rule sets produced). This problem occurs in part due to the distributed hidden layer representation created during learning. Most past symbolic knowledge extraction algorithms have focused on progressively more sophisticated ways to cluster this distributed representation. In contrast, in this dissertation, I take a different approach. I develop ways to alter the error backpropagation neural network training process itself so that it creates a representation of what has been learned in the hidden layer activation space that is more amenable to existing symbolic representation extraction methods.
In this context, this dissertation research makes four main contributions. First, modifications to the backpropagation learning procedure are derived mathematically, and it is shown that these modifications can be accomplished as local computations. Second, the effectiveness of the modified learning procedure for feedforward networks is established by showing that, on a set of benchmark tasks, it produces rule sets that are substantially simpler than those produced by standard backpropagation learning. Third, this approach is extended to simple recurrent networks, and experimental evaluation shows remarkable reduction in the sizes of the finite state machines extracted from the recurrent networks trained using this approach. Finally, this method is further modified to work on echo state networks, and computational experiments again show significant improvement in finite state machine extraction from these networks. These results clearly establish that principled modification of error backpropagation so that it constructs a better separated hidden layer representation is an effective way to improve contemporary symbolic extraction methods
Simplification of rules extracted from neural networks
Artificial neural networks (ANNs) have been proven to be successful general machine
learning techniques for, amongst others, pattern recognition and classification. Realworld
problems in agriculture (soybean, tea), medicine (cancer, cardiology, mammograms)
and finance (credit rating, stock market) are successfully solved using ANNs.
ANNs model biological neural systems. A biological neural system consists of neurons
interconnected through neural synapses. These neurons serve as information processing
units. Synapses carrt information to the neurons, which then processes or responds to
the data by sending a signal to the next level of neurons. Information is strengthened
or lessened according to the sign ..and magnitude of the weight associated with the
connection.
An ANN consists of cell-like entities called units (also called artificial neurons) and
weighted connections between these units referred to as links. ANNs can be viewed as
a directed graph with weighted connections. An unit belongs to one of three groups:
input, hidden or output. Input units receive the initial training patterns, which consist
of input attributes and the associated target attributes, from the environment. Hidden
units do not interact with the environment whereas output units presents the results to
the environment. Hidden and output units compute an output ai which is a function
f of the sum of its input weights w; multiplied by the output x; of the units j in the
preceding layer, together with a bias term fh that acts as a threshold for the unit. The
output ai for unit i with n input units is calculated as ai = f("f:,'J= 1 x;w; - 8i ). Training
of the ANN is done by adapting the weight values for each unit via a gradient search.
Given a set of input-target pairs, the ANN learns the functional relationship between
the input and the target.
A serious drawback of the neural network approach is the difficulty to determine why
a particular conclusion was reached. This is due to the inherit 'black box' nature of
the neural network approach. Neural networks rely on 'raw' training data to learn the
relationships between the initial inputs and target outputs. Knowledge is encoded in a
set of numeric weights and biases. Although this data driven aspect of neural network
allows easy adjustments when change of environment or events occur, it is difficult to
interpret numeric weights, making it difficult for humans to understand.
Concepts represent by symbolic learning algorithms are intuitive and therefore easily
understood by humans [Wnek 1994). One approach to understanding the representations
formed by neural networks is to extract such symbolic rules from networks.
Over the last few years, a number of rule extraction methods have been reported
(Craven 1993, Fu 1994). There are some general assumptions that these algorithms
adhere to. The first assumption that most rule extraction algorithms make, is that
non-input units are either maximally active (activation near 1) or inactive (activation
near 0). This Boolean valued activation is approximated by using the standard logistic
activation function /(z) = 1/( 1 + e-ā¢z ) and setting s 5.0. The use of the above
function parameters guarantees that non-input units always have non-negative activations
in the range [0,1). The second underlying premise of rule extraction is that each
hidden and output unit implements a symbolic rule. The concept associated with each
unit is the consequent of the rule, and certain subsets of the input units represent the
antecedent of the rule. Rule extraction algorithms search for those combinations of
input values to a particular hidden or output unit that results in it having an optimal
(near-one) activation. Here, rule extraction methods exploit a very basic principle of
biological neural networks. That is, if the sum of its weighted inputs exceeds a certain
threshold, then the biological neuron fires [Fu 1994). This condition is satisfied when
the sum of the weighted inputs exceeds the bias, where (E'Jiz,=::l w; > 9i)ā¢
It has been shown that most concepts described by humans usally can be expressed
as production rules in disjunctive normal form (DNF) notation. Rules expressed in
this notation are therefore highly comprehensible and intuitive. In addition, the number
of production rules may be reduced and the structure thereof simplified by using
propositional logic.
A method that extracts production rules in DNF is presented [Viktor 1995). The
basic idea of the method is the use of equivalence classes. Similarly weighted links
are grouped into a cluster, the assumption being that individual weights do not have
unique importance. Clustering considerably reduces the combinatorics of the method
as opposed to previously reported approaches.
Since the rules are in a logically manipulatable form, significant simplifications in the
structure thereof can be obtained, yielding a highly reduced and comprehensible set
of rules. Experimental results have shown that the accuracy of the extracted rules
compare favourably with the CN2 [Clark 1989] and C4.5 [Quinlan 1993] symbolic rule
extraction methods. The extracted rules are highly comprehensible and similar to those
extracted by traditional symfiolic methods
Abnormal Pattern Detection In Ppg Signals Using Time Series Analysis
The photoplethysmogram (PPG) signal is a data in continuous real-time series. It depicts the peripheral pulse wave that is produced due to heart activity, respiration, and other physiological effects. The time-series signal contains a lot of information which is difficult to be processed. The abnormal PPG signal is messy, non-periodic, and irregular. Several existing methods such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Deep Neural Network (DNN) and sensor had been used to detect abnormal pattern from PPG signal which can produce high performance and accuracy. However, these methods are higher in complexity or have uncertain repeatability. Therefore, this thesis proposed a method which is rule-based algorithm that is less complex, with quicker and more simple training, reducing the errors while still producing high accuracy. This projectās objectives are to implement rule-based algorithm method for abnormal pattern detection in PPG signals, and to investigate the accuracy and performance of rule-based algorithm in detecting the abnormal pattern. The signal processing, segmentation, feature extraction, training and testing for rule-based algorithm classifier, using wrist PPG during exercise dataset and pulse transmit time dataset, are done in this study to detect the abnormal pattern in PPG signals. The accuracy and coverage of rule for both training and testing process are recorded in order to determine the performance of the method used in this study. The abnormal PPG pattern detection using rule-based algorithm has produced accuracy of 87.30% in training process and 87.18% in testing process with coverage of rule for training and testing, 89.26% and 87.33%. The findings of this project can be further used for application of abnormal pattern in PPG signal such as healthcare and human activity recognition
Neural net diagnostics for VLSI test
This paper discusses the application of neural network pattern analysis algorithms to the IC fault diagnosis problem. A fault diagnostic is a decision rule combining what is known about an ideal circuit test response with information about how it is distorted by fabrication variations and measurement noise. The rule is used to detect fault existence in fabricated circuits using real test equipment. Traditional statistical techniques may be used to achieve this goal, but they can employ unrealistic a priori assumptions about measurement data. Our approach to this problem employs an adaptive pattern analysis technique based on feedforward neural networks. During training, a feedforward network automatically captures unknown sample distributions. This is important because distributions arising from the nonlinear effects of process variation can be more complex than is typically assumed. A feedforward network is also able to extract measurement features which contribute significantly to making a correct decision. Traditional feature extraction techniques employ matrix manipulations which can be particularly costly for large measurement vectors. In this paper we discuss a software system which we are developing that uses this approach. We also provide a simple example illustrating the use of the technique for fault detection in an operational amplifier
Data Mining by Soft Computing Methods for The Coronary Heart Disease Database
For improvement of data mining technology, the advantages and disadvantages on respective data mining methods
should be discussed by comparison under the same condition. For this purpose, the Coronary Heart Disease database (CHD DB) was developed in 2004, and the data mining competition was held in the International Conference on Knowledge-Based Intelligent Information and Engineering Systems (KES). In the competition, two methods based on soft computing were presented. In this paper, we report the overview of the CHD DB and the soft computing methods, and discuss the features of respective methods by comparison of the experimental results
- ā¦