A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

The Power of Data Visualization: A Prototype Energy Performance Map for a University Campus

[abstract not available]https://fount.aucegypt.edu/faculty_book_chapters/1432/thumbnail.jp

Dual Vigilance Fuzzy Adaptive Resonance Theory

Clusters retrieved by generic Adaptive Resonance Theory (ART) networks are limited to their internal categorical representation. This study extends the capabilities of ART by incorporating multiple vigilance thresholds in a single network: stricter (data compression) and looser (cluster similarity) vigilance values are used to obtain a many-to-one mapping of categories-to-clusters. It demonstrates this idea in the context of Fuzzy ART, presented as Dual Vigilance Fuzzy ART (DVFA), to improve the ability to capture clusters with arbitrary geometry. DVFA outperformed Fuzzy ART for the datasets in our experiments while yielding a statistically-comparable performance to another more complex, multi-prototype Fuzzy ART-based architecture

Dual Vigilance Hypersphere Adaptive Resonance Theory

The internal representation of the categories in Adaptive Resonance Theory (ART) neural networks can greatly affect the quality and compactness of the discovered clusters. Dual vigilance thresholds have been shown to yield a significant improvement in Fuzzy ART performance and allow it to retrieve arbitrarily shaped clusters while maintaining the important advantage of a simple architecture and implementation. In this study, we examine the use of Hypersphere ART within the same dual vigilance architecture, thereby presenting the Dual Vigilance Hypersphere ART (DVHA). We conduct an extensive comparison between 6 different ART-based approaches across a set of 30 benchmark datasets, using two different input ordering methods and 30 repeated runs. We find that DVHA ranks better than Dual Vigilance Fuzzy ART (DVFA) on average for many datasets, both in terms of performance and network compactness. Furthermore, although another multi-category-based architecture showed statistically superior results when the inputs are shuffled, we found no statistical difference in performance when the input was pre-processed using the visual assessment of cluster tendency (VAT), while generally being much simpler to implement and less computationally demanding. These findings make DVHA a viable alternative to its Fuzzy ART counterpart, and a simpler alternative to the other studied multi-category-based approaches in cases where resources are limited, such as embedded and hardware-based applications, provided that the input can be preprocessed using VAT

Biclustering ARTMAP Collaborative Filtering Recommender System

Collaborative filtering provides recommendations based on the behavior of each user combined with behavior of users with similar interests. Recommender systems are becoming widespread, helping people choose movies, books, and things to buy. In this study, we examine the use of Biclustering ARTMAP to build a collaborative filtering recommendation system. We introduce a novel modification to how the Biclustering ARTMAP algorithm computes the item-cluster similarity, and a way to adapt it for the prediction of user ratings. We apply the algorithm to the MovieLens 100k dataset, and find that it achieves promising performance compared to other collaborative filtering techniques

Distributed Dual Vigilance Fuzzy Adaptive Resonance Theory Learns Online, Retrieves Arbitrarily-Shaped Clusters, and Mitigates Order Dependence

This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with distributed higher-order activation and match functions and a dual vigilance mechanism. Together, these allow DDVFA to perform unsupervised modularization, create multi-prototype cluster representations, retrieve arbitrarily-shaped clusters, and reduce category proliferation. Another important contribution is the reduction of order-dependence, an issue that affects any agglomerative clustering method. This paper demonstrates two approaches for mitigating order-dependence: pre-processing using visual assessment of cluster tendency (VAT) or post-processing using a novel Merge ART module. The former is suitable for batch processing, whereas the latter also works for online learning. Experimental results in online mode carried out on 30 benchmark data sets show that DDVFA cascaded with Merge ART statistically outperformed the best other ART-based systems when samples were randomly presented. Conversely, they were found to be statistically equivalent in offline mode when samples were pre-processed using VAT. Remarkably, performance comparisons to non-ART-based clustering algorithms show that DDVFA (which learns incrementally) was also statistically equivalent to the non-incremental (offline) methods of density-based spatial clustering of applications with noise (DBSCAN), single linkage hierarchical agglomerative clustering (SL-HAC), and k-means, while retaining the appealing properties of ART. Links to the source code and data are provided. Considering the algorithm\u27s simplicity, online learning capability, and performance, it is an ideal choice for many agglomerative clustering applications

Matrix Factorization based Collaborative Filtering with Resilient Stochastic Gradient Descent

One of the leading approaches to collaborative filtering is to use matrix factorization to discover a set of latent factors that explain the pattern of preferences. In this paper, we apply a resilient stochastic gradient descent approach that uses only the sign of the gradient, similar to the R-Prop algorithm in neural network training, to matrix factorization for collaborative filtering. We evaluate the performance of our approach on the MovieLens 1M dataset, and find that test set accuracy markedly improves compared to standard gradient descent. As a follow-up experiment, we apply clustering to the learned item-factor matrix in factor space, and attempt to manually characterize each cluster of movies

A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to contemporary ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory, and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers