A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

Auto-adaptative Robot-aided Therapy based in 3D Virtual Tasks controlled by a Supervised and Dynamic Neuro-Fuzzy System

This paper presents an application formed by a classification method based on the architecture of ART neural network (Adaptive Resonance Theory) and the Fuzzy Set Theory to classify physiological reactions in order to automatically and dynamically adapt a robot-assisted rehabilitation therapy to the patient needs, using a three-dimensional task in a virtual reality system. Firstly, the mathematical and structural model of the neuro-fuzzy classification method is described together with the signal and training data acquisition. Then, the virtual designed task with physics behavior and its development procedure are explained. Finally, the general architecture of the experimentation for the auto-adaptive therapy is presented using the classification method with the virtual reality exercise

Data-driven Soft Sensors in the Process Industry

In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of process industry fields, like the chemical industry, bioprocess industry, steel industry, etc. The focus of this work is put on the data-driven Soft Sensors because of their growing popularity, already demonstrated usefulness and huge, though yet not completely realised, potential. A comprehensive selection of case studies covering the three most important Soft Sensor application fields, a general introduction to the most popular Soft Sensor modelling techniques as well as a discussion of some open issues in the Soft Sensor development and maintenance and their possible solutions are the main contributions of this work

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

Sistemas de interfaz cerebro-ordenador basados en dispositivos EEG de bajo coste y modelos neurodifusos aplicados a la imaginación de movimiento

[SPA] La presente tesis doctoral detalla la evaluación de un dispositivo de electroencefalografía de bajo coste a partir de su inclusión en un sistema de interfaz cerebro-máquina completo (BCI del inglés Brain-Computer Interfaces) basado en técnicas neurodifusas. El paradigma elegido se centra en la imaginación de movimiento multiclase sin realimentación, donde la operación es completamente asíncrona. También se ha aportado un avance en el área de la selección de características, desarrollando e implementando una metodología capaz de minimizar las componentes del vector de características necesarias para operar sistemas BCI, facilitando así la integración de éstos en plataformas móviles. En primer lugar, se ha seleccionado el dispositivo Emotiv EPOC en torno a criterios de coste, número de sensores, acceso a la señal capturada, ergonomía y relevancia para la comunidad científica. Del mismo modo, se ha abordado el problema definido en el BCI Competition III dataset V dada la disponibilidad de las señales capturadas sobre el cuero cabelludo por un equipo profesional, la exhaustiva definición del experimento y la facilidad para reproducirlo. Adicionalmente, la existencia de estudios utilizando estos mismos datos ha ofrecido una guía sobre las mejores técnicas a aplicar. Entre éstas destaca el modelo neurodifuso S-dFasArt, que hasta ahora ha presentado los mejores resultados cumpliendo las restricciones del problema. Por tanto, se ha construido un sistema BCI propio utilizando Emotiv EPOC como dispositivo de obtención de la señal EEG y S-dFasArt como sistema de inteligencia artificial. La valoración se ha realizado desde el punto de vista de un sistema BCI completo por lo que, en lugar de examinar la forma de la señal detectada, se ha calculado el rendimiento del mismo para datos capturados con diferentes equipos. Para ello se han comparado los resultados alcanzados tanto a partir de la base de datos BCI Competition como de cuatro conjuntos propios en los que han colaborado 19 voluntarios, quienes han participado en uno o varios experimentos. Se han incluido tanto sesiones utilizando Emotiv EPOC para la obtención de datos, como pruebas donde se ha utilizado una versión híbrida del mismo, en la cual se mantiene la unidad de procesamiento pero varía la tecnología y la ubicación de los sensores. Así, se ha demostrado que el sistema BCI construido integrando Emotiv EPOC junto al clasificador S-dFasArt alcanza una precisión asimilable a la lograda sobre datos capturados por equipos de investigación manteniendo el problema y la posición de los sensores. Además, la ubicación de los mismos sobre la corteza motora ha hecho posible un incremento en torno al 7% en el nivel de acierto medio (desde el 62% al 66.53 %). Igualmente, se ha corroborado la influencia positiva de la realimentación, que ha permitido lograr precisiones de por encima del 70% con Emotiv EPOC. Finamente, se ha presentado una metodología de selección de características en la que el S-dFasArt se ha integrado con modelos basados en combinaciones entre el método estadístico y el criterio difuso con la selección por orden y GMDH. La metodología desarrollada ha seleccionado automáticamente las componentes más relevantes del vector de características, alcanzando el modelo reducido obtenido por las diferentes variantes mejores resultados que el completo para dos de cada tres sujetos. Igualmente, la disminución del tamaño del conjunto de datos es muy significativa, presentando un decremento medio desde 168 a 5 características para la mejor combinación. [ENG] This PhD thesis details the evaluation process of a low-cost electroencephalography device when included into a brain-computer interface system (BCI) based on neuro-fuzzy techniques. The chosen paradigm focuses on the multi-class motor imagery problem, with no feedback and asynchronous operation. Also, a contribution to the feature selection area is presented, developing and implementing a new methodology able to minimise the number of feature vector components required to operate BCI systems, thus facilitating their integration into mobile platforms. First, the Emotiv EPOC EEG device has been selected after performing an economic evaluation considering cost and aspects such as the number of sensors, the available capabilities to access the raw brain data, the ergonomics and the relevance for the scientist community. Likewise, the BCI Competition III Dataset V defined problem has been undertaken. This has been chosen based on the availability of the raw brain signals, the detailed description of the experiment and the ability to reproduce it. Also, the existence of a number of research papers has provided guidance about the best performing approaches tackling this problem. Among then, the S-dFasArt neuro-fuzzy model has shown the best performance following the experiment rules so far. Therefore, a new BCI system has been built using Emotiv EPOC as a data capture device and S-dFasArt as an artificial intelligence unit. This assessment has been performed from the perspective of a complete BCI system so, instead of examining the shape of the detected brainwave, the performance of the setup using different data gathering devices has been analysed. Given that, a comparison of the results obtained has been performed processing data from several databases, including the BCI Competition and other four purposely-built datasets containing brain signals recorded from 19 volunteers participating in one or more experiments. Datasets include scalp potentials recorded using Emotiv EPOC as well as sessions recorded by a hybrid version of it, which maintains the processing unit while integrating a different sensor technology and allowing the setup at different electrode locations. Thus, the BCI system built integrating Emotiv EPOC and the S-dFasArt classiffier has shown an accuracy level comparable to that achieved using research EEG devices for the same problem and sensor locations. Besides, placing the electrodes over the motor cortex has allowed a 7% increase of the average success rate (from 62% to 66.53 %). Additionally, the importance of providing users with live feedback of their performance has been corroborated, obtaining accuracy levels above 70% using Emotiv EPOC. Finally, a new methodology where S- S-dFasArt is integrated with a combination of eit-her the statistic method or the fuzzy criteria and the order selection or GMDH has been introduced. The developed methodology has automatically selected the most relevant components from the feature vector, allowing the reduced model calculated from the different variations to achieve better than original accuracy levels for two out of three subjects. Moreover, a very significant average reduction from 168 to 5 features has been achieved for the highest performing combination.Escuela Internacional de Doctorado de la Universidad Politécnica de CartagenaUniversidad Politécnica de CartagenaPrograma de Doctorado en Tecnologías Industriales por la Universidad Politécnica de Cartagen

On robust and adaptive soft sensors.

In process industries, there is a great demand for additional process information such as the product quality level or the exact process state estimation. At the same time, there is a large amount of process data like temperatures, pressures, etc. measured and stored every moment. This data is mainly measured for process control and monitoring purposes but its potential reaches far beyond these applications. The task of soft sensors is the maximal exploitation of this potential by extracting and transforming the latent information from the data into more useful process knowledge. Theoretically, achieving this goal should be straightforward since the process data as well as the tools for soft sensor development in the form of computational learning methods, are both readily available. However, contrary to this evidence, there are still several obstacles which prevent soft sensors from broader application in the process industry. The identification of the sources of these obstacles and proposing a concept for dealing with them is the general purpose of this work. The proposed solution addressing the issues of current soft sensors is a conceptual architecture for the development of robust and adaptive soft sensing algorithms. The architecture reflects the results of two review studies that were conducted during this project. The first one focuses on the process industry aspects of soft sensor development and application. The main conclusions of this study are that soft sensor development is currently being done in a non-systematic, ad-hoc way which results in a large amount of manual work needed for their development and maintenance. It is also found that a large part of the issues can be related to the process data upon which the soft sensors are built. The second review study dealt with the same topic but this time it was biased towards the machine learning viewpoint. The review focused on the identification of machine learning tools, which support the goals of this work. The machine learning concepts which are considered are: (i) general regression techniques for building of soft sensors; (ii) ensemble methods; (iii) local learning; (iv) meta-learning; and (v) concept drift detection and handling. The proposed architecture arranges the above techniques into a three-level hierarchy, where the actual prediction-making models operate at the bottom level. Their predictions are flexibly merged by applying ensemble methods at the next higher level. Finally from the top level, the underlying algorithm is managed by means of metalearning methods. The architecture has a modular structure that allows new pre-processing, predictive or adaptation methods to be plugged in. Another important property of the architecture is that each of the levels can be equipped with adaptation mechanisms, which aim at prolonging the lifetime of the resulting soft sensors. The relevance of the architecture is demonstrated by means of a complex soft sensing algorithm, which can be seen as its instance. This algorithm provides mechanisms for autonomous selection of data preprocessing and predictive methods and their parameters. It also includes five different adaptation mechanisms, some of which can be applied on a sample-by-sample basis without any requirement to store the on-line data. Other, more complex ones are started only on-demand if the performance of the soft sensor drops below a defined level. The actual soft sensors are built by applying the soft sensing algorithm to three industrial data sets. The different application scenarios aim at the analysis of the fulfilment of the defined goals. It is shown that the soft sensors are able to follow changes in dynamic environment and keep a stable performance level by exploiting the implemented adaptation mechanisms. It is also demonstrated that, although the algorithm is rather complex, it can be applied to develop simple and transparent soft sensors. In another experiment, the soft sensors are built without any manual model selection or parameter tuning, which demonstrates the ability of the algorithm to reduce the effort required for soft sensor development. However, if desirable, the algorithm is at the same time very flexible and provides a number of parameters that can be manually optimised. Evidence of the ability of the algorithm to deploy soft sensors with minimal training data and as such to provide the possibility to save the time consuming and costly training data collection is also given in this work

On robust and adaptive soft sensors

In process industries, there is a great demand for additional process information such as the product quality level or the exact process state estimation. At the same time, there is a large amount of process data like temperatures, pressures, etc. measured and stored every moment. This data is mainly measured for process control and monitoring purposes but its potential reaches far beyond these applications. The task of soft sensors is the maximal exploitation of this potential by extracting and transforming the latent information from the data into more useful process knowledge. Theoretically, achieving this goal should be straightforward since the process data as well as the tools for soft sensor development in the form of computational learning methods, are both readily available. However, contrary to this evidence, there are still several obstacles which prevent soft sensors from broader application in the process industry. The identification of the sources of these obstacles and proposing a concept for dealing with them is the general purpose of this work. The proposed solution addressing the issues of current soft sensors is a conceptual architecture for the development of robust and adaptive soft sensing algorithms. The architecture reflects the results of two review studies that were conducted during this project. The first one focuses on the process industry aspects of soft sensor development and application. The main conclusions of this study are that soft sensor development is currently being done in a non-systematic, ad-hoc way which results in a large amount of manual work needed for their development and maintenance. It is also found that a large part of the issues can be related to the process data upon which the soft sensors are built. The second review study dealt with the same topic but this time it was biased towards the machine learning viewpoint. The review focused on the identification of machine learning tools, which support the goals of this work. The machine learning concepts which are considered are: (i) general regression techniques for building of soft sensors; (ii) ensemble methods; (iii) local learning; (iv) meta-learning; and (v) concept drift detection and handling. The proposed architecture arranges the above techniques into a three-level hierarchy, where the actual prediction-making models operate at the bottom level. Their predictions are flexibly merged by applying ensemble methods at the next higher level. Finally from the top level, the underlying algorithm is managed by means of metalearning methods. The architecture has a modular structure that allows new pre-processing, predictive or adaptation methods to be plugged in. Another important property of the architecture is that each of the levels can be equipped with adaptation mechanisms, which aim at prolonging the lifetime of the resulting soft sensors. The relevance of the architecture is demonstrated by means of a complex soft sensing algorithm, which can be seen as its instance. This algorithm provides mechanisms for autonomous selection of data preprocessing and predictive methods and their parameters. It also includes five different adaptation mechanisms, some of which can be applied on a sample-by-sample basis without any requirement to store the on-line data. Other, more complex ones are started only on-demand if the performance of the soft sensor drops below a defined level. The actual soft sensors are built by applying the soft sensing algorithm to three industrial data sets. The different application scenarios aim at the analysis of the fulfilment of the defined goals. It is shown that the soft sensors are able to follow changes in dynamic environment and keep a stable performance level by exploiting the implemented adaptation mechanisms. It is also demonstrated that, although the algorithm is rather complex, it can be applied to develop simple and transparent soft sensors. In another experiment, the soft sensors are built without any manual model selection or parameter tuning, which demonstrates the ability of the algorithm to reduce the effort required for soft sensor development. However, if desirable, the algorithm is at the same time very flexible and provides a number of parameters that can be manually optimised. Evidence of the ability of the algorithm to deploy soft sensors with minimal training data and as such to provide the possibility to save the time consuming and costly training data collection is also given in this work.EThOS - Electronic Theses Online ServiceGBUnited Kingdo