A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

Hybrid Models Of Fuzzy Artmap And Qlearning For Pattern Classification

Pengelasan corak adalah salah satu isu utama dalam pelbagai tugas pencarian data. Dalam kajian ini, fokus penyelidikan tertumpu kepada reka bentuk dan pembinaan model hibrid yang menggabungkan rangkaian neural Teori Resonan Adaptif (ART) terselia dan model Pembelajaran Pengukuhan (RL) untuk pengelasan corak. Secara khususnya, rangkaian ARTMAP Kabur (FAM) dan Pembelajaran-Q dijadikan sebagai tulang belakang dalam merekabentuk dan membina model-model hibrid. Satu model QFAM baharu terlebih dahulu diperkenalkan bagi menambahbaik prestasi pengelasan rangkaian FAM. Strategi pruning dimasukkan bagi mengurangkan kekompleksan QFAM. Bagi mengatasi isu ketidak-telusan, Algoritma Genetik (GA) digunakan bagi mengekstrak hukum kabur if-then daripada QFAM. Model yang terhasil iaitu QFAM-GA, dapat memberi ramalan berserta dengan huraian dengan hanya menggunakan bilangan antisiden yang sedikit. Bagi menambahkan lagi kebolehtahanan model-model Q-FAM, penggunaan sistem agenpelbagai telah dicadangkan. Hasilnya, model gugusan QFAM berasaskan agen dengan ukuran percaya dan kaedah rundingan baharu telah dicadangkan. Pelbagai jenis masalah tanda-aras telah digunakan bagi penilaian model-model gugusan dan individu berasaskan QFAM. Hasilnya telah dianalisa dan dibandingkan dengan FAM serta model-model yang dilaporkan dalam kajian terdahulu. Sebagai tambahan, dua daripada masalah dunia-nyata digunakan bagi menunjukkan kebolehan praktikal model hibrid. Keputusan akhir menunjukkan keberkesanan modul berasaskan QFAM dalam menerajui tugas-tugas pengelasan corak. ________________________________________________________________________________________________________________________ Pattern classification is one of the primary issues in various data mining tasks. In this study, the main research focus is on the design and development of hybrid models, combining the supervised Adaptive Resonance Theory (ART) neural network and Reinforcement Learning (RL) models for pattern classification. Specifically, the Fuzzy ARTMAP (FAM) network and Q-learning are adopted as the backbone for designing and developing the hybrid models. A new QFAM model is first introduced to improve the classification performance of FAM network. A pruning strategy is incorporated to reduce the complexity of QFAM. To overcome the opaqueness issue, a Genetic Algorithm (GA) is used to extract fuzzy if-then rules from QFAM. The resulting model, i.e. QFAM-GA, is able to provide predictions with explanations using only a few antecedents. To further improve the robustness of QFAM-based models, the notion of multi agent systems is employed. As a result, an agent-based QFAM ensemble model with a new trust measurement and negotiation method is proposed. A variety of benchmark problems are used for evaluation of individual and ensemble QFAM-based models. The results are analyzed and compared with those from FAM as well as other models reported in the literature. In addition, two real-world problems are used to demonstrate the practicality of the hybrid models. The outcomes indicate the effectiveness of QFAM-based models in tackling pattern classification tasks

A Neural Network Architecture for Autonomous Learning, Recognition, and Prediction in a Nonstationary World

In a constantly changing world, humans are adapted to alternate routinely between attending to familiar objects and testing hypotheses about novel ones. We can rapidly learn to recognize and narne novel objects without unselectively disrupting our memories of familiar ones. We can notice fine details that differentiate nearly identical objects and generalize across broad classes of dissimilar objects. This chapter describes a class of self-organizing neural network architectures--called ARTMAP-- that are capable of fast, yet stable, on-line recognition learning, hypothesis testing, and naming in response to an arbitrary stream of input patterns (Carpenter, Grossberg, Markuzon, Reynolds, and Rosen, 1992; Carpenter, Grossberg, and Reynolds, 1991). The intrinsic stability of ARTMAP allows the system to learn incrementally for an unlimited period of time. System stability properties can be traced to the structure of its learned memories, which encode clusters of attended features into its recognition categories, rather than slow averages of category inputs. The level of detail in the learned attentional focus is determined moment-by-moment, depending on predictive success: an error due to over-generalization automatically focuses attention on additional input details enough of which are learned in a new recognition category so that the predictive error will not be repeated. An ARTMAP system creates an evolving map between a variable number of learned categories that compress one feature space (e.g., visual features) to learned categories of another feature space (e.g., auditory features). Input vectors can be either binary or analog. Computational properties of the networks enable them to perform significantly better in benchmark studies than alternative machine learning, genetic algorithm, or neural network models. Some of the critical problems that challenge and constrain any such autonomous learning system will next be illustrated. Design principles that work together to solve these problems are then outlined. These principles are realized in the ARTMAP architecture, which is specified as an algorithm. Finally, ARTMAP dynamics are illustrated by means of a series of benchmark simulations.Advanced Research Projects Agency (N00014-92-J-4015); British Petroleum (89A-1204); National Science Foundation (IRI-90-J-4015); Office of Naval Research (N00014-91-J-4100); Air Force Office of Scientific Research (F49620-92-J-0225

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

An Ensemble Semi-Supervised Adaptive Resonance Theory Model with Explanation Capability for Pattern Classification

Most semi-supervised learning (SSL) models entail complex structures and iterative training processes as well as face difficulties in interpreting their predictions to users. To address these issues, this paper proposes a new interpretable SSL model using the supervised and unsupervised Adaptive Resonance Theory (ART) family of networks, which is denoted as SSL-ART. Firstly, SSL-ART adopts an unsupervised fuzzy ART network to create a number of prototype nodes using unlabeled samples. Then, it leverages a supervised fuzzy ARTMAP structure to map the established prototype nodes to the target classes using labeled samples. Specifically, a one-to-many (OtM) mapping scheme is devised to associate a prototype node with more than one class label. The main advantages of SSL-ART include the capability of: (i) performing online learning, (ii) reducing the number of redundant prototype nodes through the OtM mapping scheme and minimizing the effects of noisy samples, and (iii) providing an explanation facility for users to interpret the predicted outcomes. In addition, a weighted voting strategy is introduced to form an ensemble SSL-ART model, which is denoted as WESSL-ART. Every ensemble member, i.e., SSL-ART, assigns {\color{black}a different weight} to each class based on its performance pertaining to the corresponding class. The aim is to mitigate the effects of training data sequences on all SSL-ART members and improve the overall performance of WESSL-ART. The experimental results on eighteen benchmark data sets, three artificially generated data sets, and a real-world case study indicate the benefits of the proposed SSL-ART and WESSL-ART models for tackling pattern classification problems.Comment: 13 pages, 8 figure

Famtile: An Algorithm For Learning High-level Tactical Behavior From Observation

This research focuses on the learning of a class of behaviors defined as high-level behaviors. High-level behaviors are defined here as behaviors that can be executed using a sequence of identifiable behaviors. Represented by low-level contexts, these behaviors are known a priori to learning and can be modeled separately by a knowledge engineer. The learning task, which is achieved by observing an expert within simulation, then becomes the identification and representation of the low-level context sequence executed by the expert. To learn this sequence, this research proposes FAMTILE - the Fuzzy ARTMAP / Template-Based Interpretation Learning Engine. This algorithm attempts to achieve this learning task by constructing rules that govern the low-level context transitions made by the expert. By combining these rules with models for these low-level context behaviors, it is hypothesized that an intelligent model for the expert can be created that can adequately model his behavior. To evaluate FAMTILE, four testing scenarios were developed that attempt to achieve three distinct evaluation goals: assessing the learning capabilities of Fuzzy ARTMAP, evaluating the ability of FAMTILE to correctly predict expert actions and context choices given an observation, and creating a model of the expert\u27s behavior that can perform the high-level task at a comparable level of proficiency

Machine condition monitoring using artificial intelligence: The incremental learning and multi-agent system approach

Machine condition monitoring is gaining importance in industry due to the need to increase machine reliability and decrease the possible loss of production due to machine breakdown. Often the data available to build a condition monitoring system does not fully represent the system. It is also often common that the data becomes available in small batches over a period of time. Hence, it is important to build a system that is able to accommodate new data as it becomes available without compromising the performance of the previously learned data. In real-world applications, more than one condition monitoring technology is used to monitor the condition of a machine. This leads to large amounts of data, which require a highly skilled diagnostic specialist to analyze. In this thesis, artificial intelligence (AI) techniques are used to build a condition monitoring system that has incremental learning capabilities. Two incremental learning algorithms are implemented, the first method uses Fuzzy ARTMAP (FAM) algorithm and the second uses Learn++ algorithm. In addition, intelligent agents and multi-agent systems are used to build a condition monitoring system that is able to accommodate various analysis techniques. Experimentation was performed on two sets of condition monitoring data; the dissolved gas analysis (DGA) data obtained from high voltage bushings and the vibration data obtained from motor bearing. Results show that both Learn++ and FAM are able to accommodate new data without compromising the performance of classifiers on previously learned information. Results also show that intelligent agent and multi-agent system are able to achieve modularity and flexibility

Computational intelligence techniques for HVAC systems: a review

Buildings are responsible for 40% of global energy use and contribute towards 30% of the total CO2 emissions. The drive to reduce energy use and associated greenhouse gas emissions from buildings has acted as a catalyst in the development of advanced computational methods for energy efficient design, management and control of buildings and systems. Heating, ventilation and air conditioning (HVAC) systems are the major source of energy consumption in buildings and an ideal candidate for substantial reductions in energy demand. Significant advances have been made in the past decades on the application of computational intelligence (CI) techniques for HVAC design, control, management, optimization, and fault detection and diagnosis. This article presents a comprehensive and critical review on the theory and applications of CI techniques for prediction, optimization, control and diagnosis of HVAC systems.The analysis of trends reveals the minimization of energy consumption was the key optimization objective in the reviewed research, closely followed by the optimization of thermal comfort, indoor air quality and occupant preferences. Hardcoded Matlab program was the most widely used simulation tool, followed by TRNSYS, EnergyPlus, DOE–2, HVACSim+ and ESP–r. Metaheuristic algorithms were the preferred CI method for solving HVAC related problems and in particular genetic algorithms were applied in most of the studies. Despite the low number of studies focussing on MAS, as compared to the other CI techniques, interest in the technique is increasing due to their ability of dividing and conquering an HVAC optimization problem with enhanced overall performance. The paper also identifies prospective future advancements and research directions