Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective

Studies in Evolutionary Fuzzy Systems (EFSs) began in the 90s and have experienced a fast development since then, with applications to areas such as pattern recognition, curve‐fitting and regression, forecasting and control. An EFS results from the combination of a Fuzzy Inference System (FIS) with an Evolutionary Algorithm (EA). This relationship can be established for multiple purposes: fine‐tuning of FIS's parameters, selection of fuzzy rules, learning a rule base or membership functions from scratch, and so forth. Each facet of this relationship creates a strand in the literature, as membership function fine‐tuning, fuzzy rule‐based learning, and so forth and the purpose here is to outline some of what has been done in each aspect. Special focus is given to Genetic Programming‐based EFSs by providing a taxonomy of the main architectures available, as well as by pointing out the gaps that still prevail in the literature. The concluding remarks address some further topics of current research and trends, such as interpretability analysis, multiobjective optimization, and synthesis of a FIS through Evolving methods

Self-Organizing Fuzzy Belief Inference System for Classification

Evolving fuzzy systems (EFSs) are widely known as a powerful tool for streaming data prediction. In this paper, a novel zero-order EFS with a unique belief structure is proposed for data stream classification. Thanks to this new belief structure, the proposed model can handle the inter-class overlaps in a natural way and better capture the underlying multi-model structure of data streams in the form of prototypes. Utilizing data-driven soft thresholds, the proposed model self-organizes a set of prototype-based IF-THEN fuzzy belief rules from data streams for classification, and its learning outcomes are practically meaningful. With no requirement of prior knowledge in the problem domain, the proposed model is capable of self-determining the appropriate level of granularity for rule base construction, while enabling users to specify their preferences on the degree of fineness of its knowledge base. Numerical examples demonstrate the superior performance of the proposed model on a wide range of stationary and nonstationary classification benchmark problems

Self-Organizing Fuzzy Belief Inference System for Classification

Evolving fuzzy systems (EFSs) are widely known as a powerful tool for streaming data prediction. In this paper, a novel zero-order EFS with a unique belief structure is proposed for data stream classification. Thanks to this new belief structure, the proposed model can handle the inter-class overlaps in a natural way and better capture the underlying multi-model structure of data streams in the form of prototypes. Utilizing data-driven soft thresholds, the proposed model self-organizes a set of prototype-based IF-THEN fuzzy belief rules from data streams for classification, and its learning outcomes are practically meaningful. With no requirement of prior knowledge in the problem domain, the proposed model is capable of self-determining the appropriate level of granularity for rule base construction, while enabling users to specify their preferences on the degree of fineness of its knowledge base. Numerical examples demonstrate the superior performance of the proposed model on a wide range of stationary and nonstationary classification benchmark problems

Development of Self-Learning Type-2 Fuzzy Systems for System Identification and Control of Autonomous Systems

Modelling and control of dynamic systems are faced by multiple technical challenges, mainly due to the nature of uncertain complex, nonlinear, and time-varying systems. Traditional modelling techniques require a complete understanding of system dynamics and obtaining comprehensive mathematical models is not always achievable due to limited knowledge of the systems as well as the presence of multiple uncertainties in the environment. As universal approximators, fuzzy logic systems (FLSs), neural networks (NNs) and neuro-fuzzy systems have proved to be successful computational tools for representing the behaviour of complex dynamical systems. Moreover, FLSs, NNs and learning-based techniques have been gaining popularity for controlling complex, ill-defined, nonlinear, and time-varying systems in the face of uncertainties. However, fuzzy rules derived by experts can be too ad-hoc, and the performance is less than optimum. In other words, generating fuzzy rules and membership functions in fuzzy systems is a potential challenge especially for systems with many variables. Moreover, under the umbrella of FLSs, although type-1 fuzzy logic control systems (T1-FLCs) have been applied to control various complex nonlinear systems, they have limited capability to handle uncertainties. Aiming to accommodate uncertainties, type-2 fuzzy logic control systems (T2-FLCs) were established. This thesis aims to address the shortcomings of existing fuzzy techniques by utilisation of type-2 FLCs with novel adaptive capabilities. The first contribution of this thesis is a novel online system identification technique by means of a recursive interval type-2 Takagi-Sugeno fuzzy C-means clustering technique (IT2-TS-FC) to accommodate the footprint-of-uncertainties (FoUs). This development is meant to specifically address the shortcomings of type-1 fuzzy systems in capturing the footprint-of-uncertainties such as mechanical wear, rotor damage, battery drain and sensor and actuator faults. Unlike previous type-2 TS fuzzy models, the proposed method constructs two fuzzifiers (upper and lower) and two regression coefficients in the consequent part to handle uncertainties. The weighted least square method is employed to compute the regression coefficients. The proposed method is validated using two benchmarks, namely, real flight test data of a quadcopter drone and Mackey-Glass time series data. The algorithm has the capability to model uncertainties (e.g., noisy dataset). The second contribution of this thesis is the development of a novel self-adaptive interval type-2 fuzzy controller named the SAF2C for controlling multi-input multi-output (MIMO) nonlinear systems. The adaptation law is derived using sliding mode control (SMC) theory to reduce the computation time so that the learning process can be expedited by 80% compared to separate single-input single-output (SISO) controllers. The system employs the `Enhanced Iterative Algorithm with Stop Condition' (EIASC) type-reduction method, which is more computationally efficient than the `Karnik-Mendel' type-reduction algorithm. The stability of the SAF2C is proven using the Lyapunov technique. To ensure the applicability of the proposed control scheme, SAF2C is implemented to control several dynamical systems, including a simulated MIMO hexacopter unmanned aerial vehicle (UAV) in the face of external disturbance and parameter variations. The ability of SAF2C to filter the measurement noise is demonstrated, where significant improvement is obtained using the proposed controller in the face of measurement noise. Also, the proposed closed-loop control system is applied to control other benchmark dynamic systems (e.g., a simulated autonomous underwater vehicle and inverted pendulum on a cart system) demonstrating high accuracy and robustness to variations in system parameters and external disturbance. Another contribution of this thesis is a novel stand-alone enhanced self-adaptive interval type-2 fuzzy controller named the ESAF2C algorithm, whose type-2 fuzzy parameters are tuned online using the SMC theory. This way, we expect to design a computationally efficient adaptive Type-2 fuzzy system, suitable for real-time applications by introducing the EIASC type-reducer. The proposed technique is applied on a quadcopter UAV (QUAV), where extensive simulations and real-time flight tests for a hovering QUAV under wind disturbances are also conducted to validate the efficacy of the ESAF2C. Specifically, the control performance is investigated in the face of external wind gust disturbances, generated using an industrial fan. Stability analysis of the ESAF2C control system is investigated using the Lyapunov theory. Yet another contribution of this thesis is the development of a type-2 evolving fuzzy control system (T2-EFCS) to facilitate self-learning (either from scratch or from a certain predefined rule). T2-EFCS has two phases, namely, the structure learning and the parameters learning. The structure of T2-EFCS does not require previous information about the fuzzy structure, and it can start the construction of its rules from scratch with only one rule. The rules are then added and pruned in an online fashion to achieve the desired set-point. The proposed technique is applied to control an unmanned ground vehicle (UGV) in the presence of multiple external disturbances demonstrating the robustness of the proposed control systems. The proposed approach turns out to be computationally efficient as the system employs fewer fuzzy parameters while maintaining superior control performance

Aprendizaje de particiones difusas para razonamiento inductivo

Existe consenso entre los investigadores en que se pueden obtener sistemas más inteligentes por medio de la hibridación de metodologías de Soft Computing haciendo de este modo que las debilidades de unos sistemas se compensen con las bondades de otros. Los Sistemas Neurodifusos (SNDs) y los Sistemas Difusos Evolutivos (SDEs) constituyen la más notoria representatividad. Un Sistema Difuso Evolutivo es básicamente un sistema difuso robustecido por un proceso de aprendizaje basado en un Algoritmo Evolutivo (AE), en particular los Algoritmos Genéticos (AGs), los cuales están considerados actualmente como la técnica de búsqueda global más conocida y empleada. Este tipo de algoritmos presentan la capacidad de explorar y explotar espacios de búsqueda complejos, lo que les permite obtener soluciones muy próximas a la óptima. Además, la codificación genética que emplean les permite incorporar conocimiento a priori de una forma muy sencilla y aprovecharlo para guiar la búsqueda.En la presente tesis doctoral se proponen SDEs que tienen como objetivo principal el aprendizaje automático de particiones difusas para mejorar una técnica de modelado y simulación denominada Razonamiento Inductivo Difuso (FIR). Se persigue aprovechar las potencialidades de los AGs para aprender los parámetros de discretización de la metodología FIR, es decir, el número de clases por variable (granularidad) y las funciones de pertenencia (landmarks) que definen su semántica. Debido al hecho que es una metodología basada en lógica difusa, la eficiencia en el modelado y predicción de FIR está influenciada de forma directa por estos parámetros de discretización. Es así como, la determinación automática de parámetros adecuados de discretización en la metodología FIR surge como una alternativa de gran interés y utilidad al uso de valores heurísticos y/o por defecto. Más aún, automatizar la selección de los valores adecuados para estos parámetros permite el uso de la metodología FIR a usuarios no expertos en modelado de sistemas ni en lógica difusa garantizándoles el mejor rendimiento de esta metodología.Se presentan tres métodos evolutivos de aprendizaje automático de las particiones difusas: a) El aprendizaje de la granularidad con las funciones de pertenencia uniformes (AG1+EFP), b) El ajuste local de las funciones de pertenencia con un número fijo de clases para cada variable (AG1+AG2), y c) El aprendizaje en conjunto de la granularidad y de las funciones de pertenencia asociadas que definen su semántica (AG3). Dichos métodos han sido implementados en la herramienta de programación Matlab y sirven tanto para entornos Windows como para Linux.Los resultados obtenidos por los SDEs desarrollados han sido muy buenos en las cuatro aplicaciones estudiadas: sistema nervioso central humano, línea de media tensión en núcleos urbanos, estimación a corto plazo de concentraciones de ozono en Austria y estimación a largo plazo de concentraciones de ozono en México. Nuestros métodos evolutivos son los que presentan mayor eficiencia en el proceso de predicción si los comparamos con los obtenidos por otras metodologías en trabajos previos, por FIR usando valores por defecto y también, cosa no esperada, por FIR cuando los parámetros de fusificación han sido definidos por expertos en el área. En general, el AG3 y la combinación AG1+AG2, en ese orden, son los que han mostrado mejores resultados en todas las aplicaciones, seguidos por el AG1+EFP. Sin embargo el AG3 es el que presenta mayor costo computacional. Por lo tanto como conclusión general, debemos decir que los SDEs diseñados e implementados en esta tesis consiguen buenos resultados para la tarea que les ha sido encomendada en el entorno de la metodología FIR. Es pues el usuario quien debe decidir qué SDE resulta más conveniente para la aplicación que tiene entre manos, en función de las necesidades temporales y de precisión.It is commonly established that more intelligent systems can be obtained by the hybridization of Soft Computing methodologies, in order that the weaknesses of some systems be compensated with the strengths of others. Neural Fuzzy Systems (NFSs) and Evolutionary Fuzzy Systems (EFSs) are the most notorious representatives of these hybrid systems.An Evolutionary Fuzzy System is basically a fuzzy system augmented by a learning process based on an evolutionary algorithm (EA), particularly Genetic Algorithms (GAs), which are currently considered as the most well-known employed global search technique. This kind of algorithms have the ability to explore and to exploit complex search spaces, which allows the obtaining of solutions very close to the optimal ones within these spaces. Besides, the genetic codification employed allows to incorporate a priori knowledge in a very simple way and to use it to guide the search.In this PhD. thesis, we propose EFSs that improves a modeling and simulation technique the Fuzzy Inductive Reasoning (FIR). The main goal of the EFSs is to take advantage of the potentialities of GAs to learn the fuzzification parameters of FIR, i.e. the number of classes per variable (granularity) and the membership functions (landmarks) that define its semantics. Due to the fact that it is a methodology based on fuzzy logic, FIR modeling and prediction performance is directly influenced by these discretization parameters. Therefore, the automatic determination of precise fuzzification parameters in the FIR methodology is an interesting and useful alternative to the use of heuristics and/or default values. Moreover, it is expected that the automatic selection of adequate values for these parameters will open up the FIR methodology to new users, with no experience neither in systems modeling nor in fuzzy logic, guaranteeing the best performance of this methodology.Three evolutionary methods of automatic learning of fuzzy partitions are presented: a) The learning of the granularity with uniform membership functions (GA1+EFP), b) The local tuning of the membership functions with a fixed number of classes per variable (GA1+GA2), and c) The learning at the same time of the granularity and the membership functions associated that define its semantics (GA3). The evolutionary methods have been implemented in Matlab and they run in both Windows and Linux environments.The results obtained by the EFSs developed in the four applications studied, i.e. human central nervous system, maintenance costs of electrical medium line in Spanish towns, short-term estimation of ozone concentration in Austria and long-term estimation of ozone concentration in Mexico, were very good. The results obtained by our evolutionary methods have presented higher efficiency in the prediction process than those obtained by other methodologies in previous works, by FIR using default values and, even, by FIR when the fuzzification parameters have been defined by experts in the area. In general, the GA3 and the combination GA1+GA2, in that order, are the ones that have shown better results in all the applications, followed by the GA1+EFP. However, GA3 is the algorithm that presents the greatest computational cost. As general conclusion, we must say that the EFSs designed and implemented in this thesis yielded good results for the task which they were entrusted in FIR methodology. Therefore, the user should decide what EFS turns out to be more convenient for the modeling application at hand in function of time and precision needs.Postprint (published version

Soft computing applied to optimization, computer vision and medicine

Artificial intelligence has permeated almost every area of life in modern society, and its significance continues to grow. As a result, in recent years, Soft Computing has emerged as a powerful set of methodologies that propose innovative and robust solutions to a variety of complex problems. Soft Computing methods, because of their broad range of application, have the potential to significantly improve human living conditions. The motivation for the present research emerged from this background and possibility. This research aims to accomplish two main objectives: On the one hand, it endeavors to bridge the gap between Soft Computing techniques and their application to intricate problems. On the other hand, it explores the hypothetical benefits of Soft Computing methodologies as novel effective tools for such problems. This thesis synthesizes the results of extensive research on Soft Computing methods and their applications to optimization, Computer Vision, and medicine. This work is composed of several individual projects, which employ classical and new optimization algorithms. The manuscript presented here intends to provide an overview of the different aspects of Soft Computing methods in order to enable the reader to reach a global understanding of the field. Therefore, this document is assembled as a monograph that summarizes the outcomes of these projects across 12 chapters. The chapters are structured so that they can be read independently. The key focus of this work is the application and design of Soft Computing approaches for solving problems in the following: Block Matching, Pattern Detection, Thresholding, Corner Detection, Template Matching, Circle Detection, Color Segmentation, Leukocyte Detection, and Breast Thermogram Analysis. One of the outcomes presented in this thesis involves the development of two evolutionary approaches for global optimization. These were tested over complex benchmark datasets and showed promising results, thus opening the debate for future applications. Moreover, the applications for Computer Vision and medicine presented in this work have highlighted the utility of different Soft Computing methodologies in the solution of problems in such subjects. A milestone in this area is the translation of the Computer Vision and medical issues into optimization problems. Additionally, this work also strives to provide tools for combating public health issues by expanding the concepts to automated detection and diagnosis aid for pathologies such as Leukemia and breast cancer. The application of Soft Computing techniques in this field has attracted great interest worldwide due to the exponential growth of these diseases. Lastly, the use of Fuzzy Logic, Artificial Neural Networks, and Expert Systems in many everyday domestic appliances, such as washing machines, cookers, and refrigerators is now a reality. Many other industrial and commercial applications of Soft Computing have also been integrated into everyday use, and this is expected to increase within the next decade. Therefore, the research conducted here contributes an important piece for expanding these developments. The applications presented in this work are intended to serve as technological tools that can then be used in the development of new devices

Characterisation and Classification of Protein Sequences by Using Enhanced Amino Acid Indices and Signal Processing-Based Methods

Due to copyright reasons, the authors published papers have been removed from this copy of the thesis.Protein sequencing has produced overwhelming amount of protein sequences, especially in the last decade. Nevertheless, the majority of the proteins' functional and structural classes are still unknown, and experimental methods currently used to determine these properties are very expensive, laborious and time consuming. Therefore, automated computational methods are urgently required to accurately and reliably predict functional and structural classes of the proteins. Several bioinformatics methods have been developed to determine such properties of the proteins directly from their sequence information. Such methods that involve signal processing methods have recently become popular in the bioinformatics area and been investigated for the analysis of DNA and protein sequences and shown to be useful and generally help better characterise the sequences. However, there are various technical issues that need to be addressed in order to overcome problems associated with the signal processing methods for the analysis of the proteins sequences. Amino acid indices that are used to transform the protein sequences into signals have various applications and can represent diverse features of the protein sequences and amino acids. As the majority of indices have similar features, this project proposes a new set of computationally derived indices that better represent the original group of indices. A study is also carried out that resulted in finding a unique and universal set of best discriminating amino acid indices for the characterisation of allergenic proteins. This analysis extracts features directly from the protein sequences by using Discrete Fourier Transform (DFT) to build a classification model based on Support Vector Machines (SVM) for the allergenic proteins. The proposed predictive model yields a higher and more reliable accuracy than those of the existing methods. A new method is proposed for performing a multiple sequence alignment. For this method, DFT-based method is used to construct a new distance matrix in combination with multiple amino acid indices that were used to encode protein sequences into numerical sequences. Additionally, a new type of substitution matrix is proposed where the physicochemical similarities between any given amino acids is calculated. These similarities were calculated based on the 25 amino acids indices selected, where each one represents a unique biological protein feature. The proposed multiple sequence alignment method yields a better and more reliable alignment than the existing methods. In order to evaluate complex information that is generated as a result of DFT, Complex Informational Spectrum Analysis (CISA) is developed and presented. As the results show, when protein classes present similarities or differences according to the Common Frequency Peak (CFP) in specific amino acid indices, then it is probable that these classes are related to the protein feature that the specific amino acid represents. By using only the absolute spectrum in the analysis of protein sequences using the informational spectrum analysis is proven to be insufficient, as biologically related features can appear individually either in the real or the imaginary spectrum. This is successfully demonstrated over the analysis of influenza neuraminidase protein sequences. Upon identification of a new protein, it is important to single out amino acid responsible for the structural and functional classification of the protein, as well as the amino acids contributing to the protein's specific biological characterisation. In this work, a novel approach is presented to identify and quantify the relationship between individual amino acids and the protein. This is successfully demonstrated over the analysis of influenza neuraminidase protein sequences. Characterisation and identification problem of the Influenza A virus protein sequences is tackled through a Subgroup Discovery (SD) algorithm, which can provide ancillary knowledge to the experts. The main objective of the case study was to derive interpretable knowledge for the influenza A virus problem and to consequently better describe the relationships between subtypes of this virus. Finally, by using DFT-based sequence-driven features a Support Vector Machine (SVM)-based classification model was built and tested, that yields higher predictive accuracy than that of SD. The methods developed and presented in this study yield promising results and can be easily applied to proteomic fields