A performance evaluation of pruning effects on hybrid neural network

In this paper, we explore the pruning effects on a hybrid mode sequential learning algorithmnamely FuzzyARTMAP-prunable Radial Basis Function (FAM-PRBF) that utilizes FuzzyARTMAP to learn a training dataset and Radial Basis Function Network (RBFN) to performregression and classification. The pruning algorithm is used to optimize the hidden layer ofthe RBFN. The experimental results show that FAM-PRBF has successfully reduced thecomplexity and computation time of the neural network.Keywords: pruning; radial basis function network; fuzzy ARTMAP

Hybrid Models Of Fuzzy Artmap And Qlearning For Pattern Classification

Pengelasan corak adalah salah satu isu utama dalam pelbagai tugas pencarian data. Dalam kajian ini, fokus penyelidikan tertumpu kepada reka bentuk dan pembinaan model hibrid yang menggabungkan rangkaian neural Teori Resonan Adaptif (ART) terselia dan model Pembelajaran Pengukuhan (RL) untuk pengelasan corak. Secara khususnya, rangkaian ARTMAP Kabur (FAM) dan Pembelajaran-Q dijadikan sebagai tulang belakang dalam merekabentuk dan membina model-model hibrid. Satu model QFAM baharu terlebih dahulu diperkenalkan bagi menambahbaik prestasi pengelasan rangkaian FAM. Strategi pruning dimasukkan bagi mengurangkan kekompleksan QFAM. Bagi mengatasi isu ketidak-telusan, Algoritma Genetik (GA) digunakan bagi mengekstrak hukum kabur if-then daripada QFAM. Model yang terhasil iaitu QFAM-GA, dapat memberi ramalan berserta dengan huraian dengan hanya menggunakan bilangan antisiden yang sedikit. Bagi menambahkan lagi kebolehtahanan model-model Q-FAM, penggunaan sistem agenpelbagai telah dicadangkan. Hasilnya, model gugusan QFAM berasaskan agen dengan ukuran percaya dan kaedah rundingan baharu telah dicadangkan. Pelbagai jenis masalah tanda-aras telah digunakan bagi penilaian model-model gugusan dan individu berasaskan QFAM. Hasilnya telah dianalisa dan dibandingkan dengan FAM serta model-model yang dilaporkan dalam kajian terdahulu. Sebagai tambahan, dua daripada masalah dunia-nyata digunakan bagi menunjukkan kebolehan praktikal model hibrid. Keputusan akhir menunjukkan keberkesanan modul berasaskan QFAM dalam menerajui tugas-tugas pengelasan corak. ________________________________________________________________________________________________________________________ Pattern classification is one of the primary issues in various data mining tasks. In this study, the main research focus is on the design and development of hybrid models, combining the supervised Adaptive Resonance Theory (ART) neural network and Reinforcement Learning (RL) models for pattern classification. Specifically, the Fuzzy ARTMAP (FAM) network and Q-learning are adopted as the backbone for designing and developing the hybrid models. A new QFAM model is first introduced to improve the classification performance of FAM network. A pruning strategy is incorporated to reduce the complexity of QFAM. To overcome the opaqueness issue, a Genetic Algorithm (GA) is used to extract fuzzy if-then rules from QFAM. The resulting model, i.e. QFAM-GA, is able to provide predictions with explanations using only a few antecedents. To further improve the robustness of QFAM-based models, the notion of multi agent systems is employed. As a result, an agent-based QFAM ensemble model with a new trust measurement and negotiation method is proposed. A variety of benchmark problems are used for evaluation of individual and ensemble QFAM-based models. The results are analyzed and compared with those from FAM as well as other models reported in the literature. In addition, two real-world problems are used to demonstrate the practicality of the hybrid models. The outcomes indicate the effectiveness of QFAM-based models in tackling pattern classification tasks

A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

A Systematic Review of Learning based Notion Change Acceptance Strategies for Incremental Mining

The data generated contemporarily from different communication environments is dynamic in content different from the earlier static data environments. The high speed streams have huge digital data transmitted with rapid context changes unlike static environments where the data is mostly stationery. The process of extracting, classifying, and exploring relevant information from enormous flowing and high speed varying streaming data has several inapplicable issues when static data based strategies are applied. The learning strategies of static data are based on observable and established notion changes for exploring the data whereas in high speed data streams there are no fixed rules or drift strategies existing beforehand and the classification mechanisms have to develop their own learning schemes in terms of the notion changes and Notion Change Acceptance by changing the existing notion, or substituting the existing notion, or creating new notions with evaluation in the classification process in terms of the previous, existing, and the newer incoming notions. The research in this field has devised numerous data stream mining strategies for determining, predicting, and establishing the notion changes in the process of exploring and accurately predicting the next notion change occurrences in Notion Change. In this context of feasible relevant better knowledge discovery in this paper we have given an illustration with nomenclature of various contemporarily affirmed models of benchmark in data stream mining for adapting the Notion Change

Incremental Cluster Validity Index-Guided Online Learning for Performance and Robustness to Presentation Order

In streaming data applications, the incoming samples are processed and discarded, and therefore, intelligent decision-making is crucial for the performance of lifelong learning systems. In addition, the order in which the samples arrive may heavily affect the performance of incremental learners. The recently introduced incremental cluster validity indices (iCVIs) provide valuable aid in addressing such class of problems. Their primary use case has been cluster quality monitoring; nonetheless, they have been recently integrated in a streaming clustering method. In this context, the work presented, here, introduces the first adaptive resonance theory (ART)-based model that uses iCVIs for unsupervised and semi-supervised online learning. Moreover, it shows how to use iCVIs to regulate ART vigilance via an iCVI-based match tracking mechanism. The model achieves improved accuracy and robustness to ordering effects by integrating an online iCVI module as module B of a topological ART predictive mapping (TopoARTMAP)—thereby being named iCVI-TopoARTMAP—and using iCVI-driven postprocessing heuristics at the end of each learning step. The online iCVI module provides assignments of input samples to clusters at each iteration in accordance to any of the several iCVIs. The iCVI-TopoARTMAP maintains useful properties shared by the ART predictive mapping (ARTMAP) models, such as stability, immunity to catastrophic forgetting, and the many-to-one mapping capability via the map field module. The performance and robustness to the presentation order of iCVI-TopoARTMAP were evaluated via experiments with synthetic and real-world datasets

Novel Art-Based Neural Network Models For Pattern Classification, Rule Extraction And Data Regression

This thesis is concerned with the development of novel neural network models for tackling pattern classification, rule extraction, and data regression problems. The research focuses on one of the advanced features of neural networks, i.e., the incremental learning ability. This ability relates to continuous learning of new knowledge without disturbing the existing knowledge base and without re-iterating through the training samples. The Adaptive Resonance Theory (ART) and Generalized Regression Neural Network (GRNN) models are employed as the backbone in this research

Anomaly intrusion detection: a distributed ARTMAP neural network approach

Computer network and information security are becoming more important as the use and availability of the Internet continues to increase. Intrusion detection systems are being integrated with computer network security to monitor for misuse or anomalous behavior within a computer network. In this thesis, a Distributed ARTMAP (dARTMAP) artificial neural network is used to analyze network packet data for anomalies. The dARTMAP artificial neural network can perform fast, stable on-line learning in real-time with novelty patterns and has the ability to encode rare events. These features make dARTMAP a strong candidate for intrusion detection. The dARTMAP\u27s on-line learning capability combined with noisy input data can cause the neural network to over-learn resulting in an abundance of category nodes and an increase in computations. The architecture of the Distributed ARTMAP was modified to incorporate category node pruning and category node hibernation to overcome issues associated with category proliferation and computational complexity. Pruning attempts to reduce the number of category nodes required to encode the information. Hibernation attempts to reduce the catastrophic forgetting introduced by pruning. Three pruning techniques and one hibernation technique were examined for their effectiveness in maximizing the accuracy of the neural network and minimizing the time required to train the network data. Since the purpose of the dARTMAP artificial neural network is to detect intrusions, the supervised neural network has to autonomously predict the level of intrusion and refine this prediction through the learning process. Triggers of the neural network\u27s dynamic learning process are used as indicators for intrusive events to predict training outputs for the neural network. These triggers correspond to the three fundamental properties of security: confidentiality, integrity, and availability

Data mining using intelligent systems : an optimized weighted fuzzy decision tree approach

Data mining can be said to have the aim to analyze the observational datasets to find relationships and to present the data in ways that are both understandable and useful. In this thesis, some existing intelligent systems techniques such as Self-Organizing Map, Fuzzy C-means and decision tree are used to analyze several datasets. The techniques are used to provide flexible information processing capability for handling real-life situations. This thesis is concerned with the design, implementation, testing and application of these techniques to those datasets. The thesis also introduces a hybrid intelligent systems technique: Optimized Weighted Fuzzy Decision Tree (OWFDT) with the aim of improving Fuzzy Decision Trees (FDT) and solving practical problems. This thesis first proposes an optimized weighted fuzzy decision tree, incorporating the introduction of Fuzzy C-Means to fuzzify the input instances but keeping the expected labels crisp. This leads to a different output layer activation function and weight connection in the neural network (NN) structure obtained by mapping the FDT to the NN. A momentum term was also introduced into the learning process to train the weight connections to avoid oscillation or divergence. A new reasoning mechanism has been also proposed to combine the constructed tree with those weights which had been optimized in the learning process. This thesis also makes a comparison between the OWFDT and two benchmark algorithms, Fuzzy ID3 and weighted FDT. SIx datasets ranging from material science to medical and civil engineering were introduced as case study applications. These datasets involve classification of composite material failure mechanism, classification of electrocorticography (ECoG)/Electroencephalogram (EEG) signals, eye bacteria prediction and wave overtopping prediction. Different intelligent systems techniques were used to cluster the patterns and predict the classes although OWFDT was used to design classifiers for all the datasets. In the material dataset, Self-Organizing Map and Fuzzy C-Means were used to cluster the acoustic event signals and classify those events to different failure mechanism, after the classification, OWFDT was introduced to design a classifier in an attempt to classify acoustic event signals. For the eye bacteria dataset, we use the bagging technique to improve the classification accuracy of Multilayer Perceptrons and Decision Trees. Bootstrap aggregating (bagging) to Decision Tree also helped to select those most important sensors (features) so that the dimension of the data could be reduced. Those features which were most important were used to grow the OWFDT and the curse of dimensionality problem could be solved using this approach. The last dataset, which is concerned with wave overtopping, was used to benchmark OWFDT with some other Intelligent Systems techniques, such as Adaptive Neuro-Fuzzy Inference System (ANFIS), Evolving Fuzzy Neural Network (EFuNN), Genetic Neural Mathematical Method (GNMM) and Fuzzy ARTMAP. Through analyzing these datasets using these Intelligent Systems Techniques, it has been shown that patterns and classes can be found or can be classified through combining those techniques together. OWFDT has also demonstrated its efficiency and effectiveness as compared with a conventional fuzzy Decision Tree and weighted fuzzy Decision Tree

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv