Incremental Cluster Validity Index-Guided Online Learning for Performance and Robustness to Presentation Order

In streaming data applications, the incoming samples are processed and discarded, and therefore, intelligent decision-making is crucial for the performance of lifelong learning systems. In addition, the order in which the samples arrive may heavily affect the performance of incremental learners. The recently introduced incremental cluster validity indices (iCVIs) provide valuable aid in addressing such class of problems. Their primary use case has been cluster quality monitoring; nonetheless, they have been recently integrated in a streaming clustering method. In this context, the work presented, here, introduces the first adaptive resonance theory (ART)-based model that uses iCVIs for unsupervised and semi-supervised online learning. Moreover, it shows how to use iCVIs to regulate ART vigilance via an iCVI-based match tracking mechanism. The model achieves improved accuracy and robustness to ordering effects by integrating an online iCVI module as module B of a topological ART predictive mapping (TopoARTMAP)—thereby being named iCVI-TopoARTMAP—and using iCVI-driven postprocessing heuristics at the end of each learning step. The online iCVI module provides assignments of input samples to clusters at each iteration in accordance to any of the several iCVIs. The iCVI-TopoARTMAP maintains useful properties shared by the ART predictive mapping (ARTMAP) models, such as stability, immunity to catastrophic forgetting, and the many-to-one mapping capability via the map field module. The performance and robustness to the presentation order of iCVI-TopoARTMAP were evaluated via experiments with synthetic and real-world datasets

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

Boletín Oficial de la Provincia de Oviedo: Número 46 - 1928 febrero 27

Statement of Problem: Biological systems are constantly evolving and multi-dimensional. They have subsystems that are coupled to each other with nonlinear interactions that are time dependent. Data measured from biological systems over time are nonstationary with changing mean and variance. In order to characterise, analyse and extract information from time dependent biological data, a model must be capable of evolving, be capable of categorising dynamic information and provide a mechanism for extraction of ongoing knowledge. In this thesis we examine existing artificial neural network (ANN) models and their capabilities in the application to real time, biological time series data. We investigate existing features extracted from biological time series data. We develop ANN techniques further incorporating extracted features in an ongoing basis and providing real time extraction of knowledge. Explanation of method and procedures: We study the human biological system by examining the time series constructed from the time differences between heart beats. Measures derived from this time series are known as heart rate variability (HRV). We extract time, frequency and fractal domain HRV features. The data was collected as part of this study from 31 post myocardial subjects and 31 age and sex matched healthy subjects. The heart beat interval time series for each subject was constructed from ECG records of twenty to thirty minute duration. Existing models are explored for data modelling including fuzzy c-means clustering, fuzzy neural networks and fuzzy adaptive resonance networks (fuzzy-ART). A new ANN model ARTdECOS is constructed, which incorporates aspects of fuzzy-ART and evolving connectionist systems (ECOS). ARTdECOS is implemented on a portable data capture device to show its viability in handling real time data and to reveal issues requiring further development. Summary of results and conclusions: Category nodes generated by fuzzy ART reach expansion limits, and multiple nodes are generated to represent a single classification state. A category amalgamation procedure in ARTdECOS allows consolidation of these multiple nodes into a single node. As a consequence, meaningful rule extraction is made possible. A graphical representation of feature boundary limits allows a quick and convenient way to extract knowledge from classification results in ARTdECOS. State switching dynamics are evident in HRV time series data through segmenting of data from individual subjects. Real time scaling of features is necessary to implement ARTdECOS in a real time environment. This is accomplished in ARTdECOS by rescaling weight vectors when input features are rescaled. ANN models are a useful tool in understanding and extracting knowledge from biological time series data. These tools may be applied to biofeedback applications in real time, ongoing environments. Fractal features provide a representation of the complexity of biological time series data, as part of multiple feature extraction across feature domains. Future research includes constructing ANN models that incorporate results generated over short time intervals into temporally global space. The global model would also incorporate anomaly information, for instance ectopic detection in HRV applications. Additional integrative ANN modelling is needed to provide a supervisory system to incorporate the addition of expert knowledge