INTERPRETATION TRAINED NEURAL NETWORKS BASED ON GENETIC ALGORITHMS

ABSTRAC

A New Data Mining Scheme Using Artificial Neural Networks

Classification is one of the data mining problems receiving enormous attention in the database community. Although artificial neural networks (ANNs) have been successfully applied in a wide range of machine learning applications, they are however often regarded as black boxes, i.e., their predictions cannot be explained. To enhance the explanation of ANNs, a novel algorithm to extract symbolic rules from ANNs has been proposed in this paper. ANN methods have not been effectively utilized for data mining tasks because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by human experts. With the proposed approach, concise symbolic rules with high accuracy, that are easily explainable, can be extracted from the trained ANNs. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and the accuracy. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of benchmark data mining classification problems

Rule-Extraction Methods From Feedforward Neural Networks: A Systematic Literature Review

Motivated by the interpretability question in ML models as a crucial element for the successful deployment of AI systems, this paper focuses on rule extraction as a means for neural networks interpretability. Through a systematic literature review, different approaches for extracting rules from feedforward neural networks, an important block in deep learning models, are identified and explored. The findings reveal a range of methods developed for over two decades, mostly suitable for shallow neural networks, with recent developments to meet deep learning models' challenges. Rules offer a transparent and intuitive means of explaining neural networks, making this study a comprehensive introduction for researchers interested in the field. While the study specifically addresses feedforward networks with supervised learning and crisp rules, future work can extend to other network types, machine learning methods, and fuzzy rule extraction

Extracting rules from neural networks by pruning and hidden-unit splitting

Neural Computation91205-22

Data mining techniques for protein sequence analysis

This thesis concerns two areas of bioinformatics related by their role in protein structure and function: protein structure prediction and post translational modification of proteins. The dihedral angles Ψ and Φ are predicted using support vector regression. For the prediction of Ψ dihedral angles the addition of structural information is examined and the normalisation of Ψ and Φ dihedral angles is examined. An application of the dihedral angles is investigated. The relationship between dihedral angles and three bond J couplings determined from NMR experiments is described by the Karplus equation. We investigate the determination of the correct solution of the Karplus equation using predicted Φ dihedral angles. Glycosylation is an important post translational modification of proteins involved in many different facets of biology. The work here investigates the prediction of N-linked and O-linked glycosylation sites using the random forest machine learning algorithm and pairwise patterns in the data. This methodology produces more accurate results when compared to state of the art prediction methods. The black box nature of random forest is addressed by using the trepan algorithm to generate a decision tree with comprehensible rules that represents the decision making process of random forest. The prediction of our program GPP does not distinguish between glycans at a given glycosylation site. We use farthest first clustering, with the idea of classifying each glycosylation site by the sugar linking the glycan to protein. This thesis demonstrates the prediction of protein backbone torsion angles and improves the current state of the art for the prediction of glycosylation sites. It also investigates potential applications and the interpretation of these methods

Data mining techniques for protein sequence analysis

This thesis concerns two areas of bioinformatics related by their role in protein structure and function: protein structure prediction and post translational modification of proteins. The dihedral angles Ψ and Φ are predicted using support vector regression. For the prediction of Ψ dihedral angles the addition of structural information is examined and the normalisation of Ψ and Φ dihedral angles is examined. An application of the dihedral angles is investigated. The relationship between dihedral angles and three bond J couplings determined from NMR experiments is described by the Karplus equation. We investigate the determination of the correct solution of the Karplus equation using predicted Φ dihedral angles. Glycosylation is an important post translational modification of proteins involved in many different facets of biology. The work here investigates the prediction of N-linked and O-linked glycosylation sites using the random forest machine learning algorithm and pairwise patterns in the data. This methodology produces more accurate results when compared to state of the art prediction methods. The black box nature of random forest is addressed by using the trepan algorithm to generate a decision tree with comprehensible rules that represents the decision making process of random forest. The prediction of our program GPP does not distinguish between glycans at a given glycosylation site. We use farthest first clustering, with the idea of classifying each glycosylation site by the sugar linking the glycan to protein. This thesis demonstrates the prediction of protein backbone torsion angles and improves the current state of the art for the prediction of glycosylation sites. It also investigates potential applications and the interpretation of these methods

A membership function selection method for fuzzy neural networks

Fuzzy neural networks provide for the extraction of fuzzy rules from artificial neural network architectures. In this paper we describe a general method, based on statistical analysis of the training data, for the selection of fuzzy membership functions to be used in connection with fuzzy neural networks. The technique is first described and then illustrated by means of two experimental examinations.Unpublished[Purvis et al, 1997] Purvis, M., Kasabov, N., Benwell, G., Zhou, Q. and Zhang, F. (1997) Neuro-fuzzy Methods for Environmental Modeling, To appear in Proceedings of the Second International Symposium on Environmental Software Systems, Chapman and Hall, London. [Kasabov, 1996] Kasabov, N., (1996) Foundation of Neural Networks, Fuzzy Systems and Knowledge Engineering. MITPress,Cambrige,MA. [Kasabov, 1993] Kasabov, N., (1993) Learning Fuzzy Rules and Membership Functions in Fuzzy Neural Networks, Proceeding of ANNES’93, Dunedin, New Zealand. [Hauptmann et al, 1995] Hauptmann, W., Heesche, K. (1995) A neural Net Topology for Bidirectional Fuzzy-Neuro Transformation. Proceedings of the International Joint Conference of the Fourth IEEE International Conference on Fuzzy Systems and the Second International Fuzzy Engineering Symposium, Y okohama, Japan, Vol. 3, 1071 – 1718. [Horikawa et al, 1992] Horikawa, A., Furuhashi, T. and Uchikawa, Y. (1992) On Fuzzy Modeling Using Fuzzy Neural Networks with the Back-Propagation Algorithm. IEEE Transactions On Neural Networks, Vol. 3, No. 5, 801 – 806. [Kasabov et al, 1997] Kasabov, N., Kim, J., Watts, M. and Gray, A. (1997) FuNN/2–A Fuzzy Neural Network Architecture for Adaptive Learning and Knowledge Acquisition. To appear in Information Science Application. [Mang et al, 1995] Mang, G., Lan, H. and Zhang, L. (1995) A Genetic-based method of Generating Fuzzy Rules and Membership Functions by Learning from Examples. Proceedings of ICONIP’95, China, Vol. 1, 335 – 338. [Setiono, 1997] Setiono, R. (1997) Extracting Rules from Neural Networks by Pruning and Hidden-unit Splitting. Neural Computation 9, No. 1, 321 – 341