A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

Evolving Ensemble Fuzzy Classifier

The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in this paper. pENsemble differs from existing architectures in the fact that it is built upon an evolving classifier from data streams, termed Parsimonious Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism, which estimates a localized generalization error of a base classifier. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. pENsemble adopts a dynamic ensemble structure to output a final classification decision where it features a novel drift detection scenario to grow the ensemble structure. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity.Comment: this paper has been published by IEEE Transactions on Fuzzy System

Comprehensive Review on Detection and Classification of Power Quality Disturbances in Utility Grid With Renewable Energy Penetration

The global concern with power quality is increasing due to the penetration of renewable energy (RE) sources to cater the energy demands and meet de-carbonization targets. Power quality (PQ) disturbances are found to be more predominant with RE penetration due to the variable outputs and interfacing converters. There is a need to recognize and mitigate PQ disturbances to supply clean power to the consumer. This article presents a critical review of techniques used for detection and classification PQ disturbances in the utility grid with renewable energy penetration. The broad perspective of this review paper is to provide various concepts utilized for extraction of the features to detect and classify the PQ disturbances even in the noisy environment. More than 220 research publications have been critically reviewed, classified and listed for quick reference of the engineers, scientists and academicians working in the power quality area

A Novel Feature-Selection Algorithm in IoT Networks for Intrusion Detection

The Internet of Things (IoT) and network-enabled smart devices are crucial to the digitally interconnected society of the present day. However, the increased reliance on IoT devices increases their susceptibility to malicious activities within network traffic, posing significant challenges to cybersecurity. As a result, both system administrators and end users are negatively affected by these malevolent behaviours. Intrusion-detection systems (IDSs) are commonly deployed as a cyber attack defence mechanism to mitigate such risks. IDS plays a crucial role in identifying and preventing cyber hazards within IoT networks. However, the development of an efficient and rapid IDS system for the detection of cyber attacks remains a challenging area of research. Moreover, IDS datasets contain multiple features, so the implementation of feature selection (FS) is required to design an effective and timely IDS. The FS procedure seeks to eliminate irrelevant and redundant features from large IDS datasets, thereby improving the intrusion-detection system’s overall performance. In this paper, we propose a hybrid wrapper-based feature-selection algorithm that is based on the concepts of the Cellular Automata (CA) engine and Tabu Search (TS)-based aspiration criteria. We used a Random Forest (RF) ensemble learning classifier to evaluate the fitness of the selected features. The proposed algorithm, CAT-S, was tested on the TON_IoT dataset. The simulation results demonstrate that the proposed algorithm, CAT-S, enhances classification accuracy while simultaneously reducing the number of features and the false positive rate.publishedVersio

Social Measurement and Causal Inference with Text

The digital age has dramatically increased access to large-scale collections of digitized text documents. These corpora include, for example, digital traces from social media, decades of archived news reports, and transcripts of spoken interactions in political, legal, and economic spheres. For social scientists, this new widespread data availability has potential for improved quantitative analysis of relationships between language use and human thought, actions, and societal structure. However, the large-scale nature of these collections means that traditional manual approaches to analyzing content are extremely costly and do not scale. Furthermore, incorporating unstructured text data into quantitative analysis is difficult due to texts’ high-dimensional nature and linguistic complexity. This thesis blends (a) the computational strengths of natural language processing (NLP) and machine learning to automate and scale-up quantitative text analysis with (b) two themes central to social scientific studies but often under-addressed in NLP: measurement—creating quantifiable summaries of empirical phenomena—and causal inference—estimating the effects of interventions. First, we address measuring class prevalence in document collections; we contribute a generative probabilistic modeling approach to prevalence estimation and show empirically that our model is more robust to shifts in class priors between training and inference. Second, we examine cross- document entity-event measurement; we contribute an empirical pipeline and a novel latent disjunction model to identify the names of civilians killed by police from our corpus of web-scraped news reports. Third, we gather and categorize applications that use text to reduce confounding from causal estimates and contribute a list of open problems as well as guidance about data processing and evaluation decisions in this area. Finally, we contribute a new causal research design to estimate the natural indirect and direct effects of social group signals (e.g. race or gender) on conversational outcomes with separate aspects of language as causal mediators; this chapter is motivated by a theoretical case study of U.S. Supreme Court oral arguments and the effect of an advocate’s gender on interruptions from justices. We conclude by discussing the relationship between measurement and causal inference with text and future work at this intersection

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

Development of advanced autonomous learning algorithms for nonlinear system identification and control

Identification of nonlinear dynamical systems, data stream analysis, etc. is usually handled by autonomous learning algorithms like evolving fuzzy and evolving neuro-fuzzy systems (ENFSs). They are characterized by the single-pass learning mode and open structure-property. Such features enable their effective handling of fast and rapidly changing natures of data streams. The underlying bottleneck of ENFSs lies in its design principle, which involves a high number of free parameters (rule premise and rule consequent) to be adapted in the training process. This figure can even double in the case of the type-2 fuzzy system. From this literature gap, a novel ENFS, namely Parsimonious Learning Machine (PALM) is proposed in this thesis. To reduce the number of network parameters significantly, PALM features utilization of a new type of fuzzy rule based on the concept of hyperplane clustering, where it has no rule premise parameters. PALM is proposed in both type-1 and type-2 fuzzy systems where all of them characterize a fully dynamic rule-based system. Thus, it is capable of automatically generating, merging, and tuning the hyperplane-based fuzzy rule in a single-pass manner. Moreover, an extension of PALM, namely recurrent PALM (rPALM), is proposed and adopts the concept of teacher-forcing mechanism in the deep learning literature. The efficacy of both PALM and rPALM have been evaluated through numerical study with data streams and to identify nonlinear unmanned aerial vehicle system. The proposed models showcase significant improvements in terms of computational complexity and the number of required parameters against several renowned ENFSs while attaining comparable and often better predictive accuracy. The ENFSs have also been utilized to develop three autonomous intelligent controllers (AICons) in this thesis. They are namely Generic (G) controller, Parsimonious controller (PAC), and Reduced Parsimonious Controller (RedPAC). All these controllers start operating from scratch with an empty set of fuzzy rules, and no offline training is required. To cope with the dynamic behavior of the plant, these controllers can add, merge or prune the rules on demand. Among three AICons, the G-controller is built by utilizing an advanced incremental learning machine, namely Generic Evolving Neuro-Fuzzy Inference System. The integration of generalized adaptive resonance theory provides a compact structure of the G-controller. Consequently, the faster evolution of structure is witnessed, which lowers its computational cost. Another AICon namely, PAC is rooted with PALM's architecture. Since PALM has a dependency on user-defined thresholds to adapt the structure, these thresholds are replaced with the concept of bias- variance trade-off in PAC. In RedPAC, the network parameters have further reduced in contrast with PALM-based PAC, where the number of consequent parameters has reduced to one parameter per rule. These AICons work with very minor expert domain knowledge and developed by incorporating the sliding mode control technique. In G-controller and RedPAC, the control law and adaptation laws for the consequent parameters are derived from the SMC algorithm to establish a stable closed-loop system, where the stability of these controllers are guaranteed by using the Lyapunov function and the uniform asymptotic convergence of tracking error to zero is witnessed through the implication of an auxiliary robustifying control term. While using PAC, the boundedness and convergence of the closed-loop control system's tracking error and the controller's consequent parameters are confirmed by utilizing the LaSalle-Yoshizawa theorem. Their efficacy is evaluated by observing various trajectory tracking performance of unmanned aerial vehicles. The accuracy of these controllers is comparable or better than the benchmark controllers where the proposed controllers incur significantly fewer parameters to attain similar or better tracking performance