Location of Repository

Intelligent data mining using artificial neural networks and genetic algorithms : techniques and applications

By Jianhua Yang

Abstract

Data Mining (DM) refers to the analysis of observational datasets to find\ud relationships and to summarize the data in ways that are both understandable\ud and useful. Many DM techniques exist. Compared with other DM techniques,\ud Intelligent Systems (ISs) based approaches, which include Artificial Neural\ud Networks (ANNs), fuzzy set theory, approximate reasoning, and derivative-free\ud optimization methods such as Genetic Algorithms (GAs), are tolerant of\ud imprecision, uncertainty, partial truth, and approximation. They provide\ud flexible information processing capability for handling real-life situations. This\ud thesis is concerned with the ideas behind design, implementation, testing and\ud application of a novel ISs based DM technique. The unique contribution of this\ud thesis is in the implementation of a hybrid IS DM technique (Genetic Neural\ud Mathematical Method, GNMM) for solving novel practical problems, the\ud detailed description of this technique, and the illustrations of several\ud applications solved by this novel technique.\ud GNMM consists of three steps: (1) GA-based input variable selection, (2) Multi-\ud Layer Perceptron (MLP) modelling, and (3) mathematical programming based\ud rule extraction. In the first step, GAs are used to evolve an optimal set of MLP\ud inputs. An adaptive method based on the average fitness of successive\ud generations is used to adjust the mutation rate, and hence the\ud exploration/exploitation balance. In addition, GNMM uses the elite group and\ud appearance percentage to minimize the randomness associated with GAs. In\ud the second step, MLP modelling serves as the core DM engine in performing\ud classification/prediction tasks. An Independent Component Analysis (ICA)\ud based weight initialization algorithm is used to determine optimal weights\ud before the commencement of training algorithms. The Levenberg-Marquardt\ud (LM) algorithm is used to achieve a second-order speedup compared to\ud conventional Back-Propagation (BP) training. In the third step, mathematical\ud programming based rule extraction is not only used to identify the premises of\ud multivariate polynomial rules, but also to explore features from the extracted\ud rules based on data samples associated with each rule. Therefore, the\ud methodology can provide regression rules and features not only in the\ud polyhedrons with data instances, but also in the polyhedrons without data\ud instances.\ud A total of six datasets from environmental and medical disciplines were used\ud as case study applications. These datasets involve the prediction of\ud longitudinal dispersion coefficient, classification of electrocorticography\ud (ECoG)/Electroencephalogram (EEG) data, eye bacteria Multisensor Data\ud Fusion (MDF), and diabetes classification (denoted by Data I through to Data VI). GNMM was applied to all these six datasets to explore its effectiveness,\ud but the emphasis is different for different datasets. For example, the emphasis\ud of Data I and II was to give a detailed illustration of how GNMM works; Data III\ud and IV aimed to show how to deal with difficult classification problems; the\ud aim of Data V was to illustrate the averaging effect of GNMM; and finally Data\ud VI was concerned with the GA parameter selection and benchmarking GNMM\ud with other IS DM techniques such as Adaptive Neuro-Fuzzy Inference System\ud (ANFIS), Evolving Fuzzy Neural Network (EFuNN), Fuzzy ARTMAP, and\ud Cartesian Genetic Programming (CGP). In addition, datasets obtained from\ud published works (i.e. Data II & III) or public domains (i.e. Data VI) where\ud previous results were present in the literature were also used to benchmark\ud GNMM’s effectiveness.\ud As a closely integrated system GNMM has the merit that it needs little human\ud interaction. With some predefined parameters, such as GA’s crossover\ud probability and the shape of ANNs’ activation functions, GNMM is able to\ud process raw data until some human-interpretable rules being extracted. This is\ud an important feature in terms of practice as quite often users of a DM system\ud have little or no need to fully understand the internal components of such a\ud system. Through case study applications, it has been shown that the GA-based\ud variable selection stage is capable of: filtering out irrelevant and noisy\ud variables, improving the accuracy of the model; making the ANN structure less\ud complex and easier to understand; and reducing the computational complexity\ud and memory requirements. Furthermore, rule extraction ensures that the MLP\ud training results are easily understandable and transferrable

Topics: QA
OAI identifier: oai:wrap.warwick.ac.uk:3831

Suggested articles

Preview

Citations

  1. (2002). Classification of bacteria responsible for ENT and eye infections using the cyranose system." doi
  2. (2003). Data mining: multimedia, soft computing, and bioinformatics. doi
  3. (2001). Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning." doi
  4. (2005). Methods Towards Invasive Human Brain Computer Interfaces." Advances in neural information processing systems.(17):
  5. (2005). Predicting longitudinal dispersion coefficient in natural streams by artificial neural network." doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.