Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective

Studies in Evolutionary Fuzzy Systems (EFSs) began in the 90s and have experienced a fast development since then, with applications to areas such as pattern recognition, curve‐fitting and regression, forecasting and control. An EFS results from the combination of a Fuzzy Inference System (FIS) with an Evolutionary Algorithm (EA). This relationship can be established for multiple purposes: fine‐tuning of FIS's parameters, selection of fuzzy rules, learning a rule base or membership functions from scratch, and so forth. Each facet of this relationship creates a strand in the literature, as membership function fine‐tuning, fuzzy rule‐based learning, and so forth and the purpose here is to outline some of what has been done in each aspect. Special focus is given to Genetic Programming‐based EFSs by providing a taxonomy of the main architectures available, as well as by pointing out the gaps that still prevail in the literature. The concluding remarks address some further topics of current research and trends, such as interpretability analysis, multiobjective optimization, and synthesis of a FIS through Evolving methods

An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams

Existing FNNs are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This paper proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. DEVFNN is developed under the stacked generalization principle via the feature augmentation concept where a recently developed algorithm, namely gClass, drives the hidden layer. It is equipped by an automatic feature selection method which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent uncontrollable growth of dimensionality of input space due to the nature of feature augmentation approach in building a deep network structure. DEVFNN works in the sample-wise fashion and is compatible for data stream applications. The efficacy of DEVFNN has been thoroughly evaluated using seven datasets with non-stationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart where DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept drift detection method is an effective tool to control the depth of network structure while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance.Comment: This paper has been published in IEEE Transactions on Fuzzy System

Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A Survey

Major assumptions in computational intelligence and machine learning consist of the availability of a historical dataset for model development, and that the resulting model will, to some extent, handle similar instances during its online operation. However, in many real world applications, these assumptions may not hold as the amount of previously available data may be insufficient to represent the underlying system, and the environment and the system may change over time. As the amount of data increases, it is no longer feasible to process data efficiently using iterative algorithms, which typically require multiple passes over the same portions of data. Evolving modeling from data streams has emerged as a framework to address these issues properly by self-adaptation, single-pass learning steps and evolution as well as contraction of model components on demand and on the fly. This survey focuses on evolving fuzzy rule-based models and neuro-fuzzy networks for clustering, classification and regression and system identification in online, real-time environments where learning and model development should be performed incrementally. (C) 2019 Published by Elsevier Inc.Igor Škrjanc, Jose Antonio Iglesias and Araceli Sanchis would like to thank to the Chair of Excellence of Universidad Carlos III de Madrid, and the Bank of Santander Program for their support. Igor Škrjanc is grateful to Slovenian Research Agency with the research program P2-0219, Modeling, simulation and control. Daniel Leite acknowledges the Minas Gerais Foundation for Research and Development (FAPEMIG), process APQ-03384-18. Igor Škrjanc and Edwin Lughofer acknowledges the support by the ”LCM — K2 Center for Symbiotic Mechatronics” within the framework of the Austrian COMET-K2 program. Fernando Gomide is grateful to the Brazilian National Council for Scientific and Technological Development (CNPq) for grant 305906/2014-3

Survey on Neuro-Fuzzy systems and their applications in technical diagnostics and measurement

Both fuzzy logic, as the basis of many inference systems, and Neural Networks, as a powerful computational model for classification and estimation, have been used in many application fields since their birth. These two techniques are somewhat supplementary to each other in a way that what one is lacking of the other can provide. This led to the creation of Neuro-Fuzzy systems which utilize fuzzy logic to construct a complex model by extending the capabilities of Artificial Neural Networks. Generally speaking all type of systems that integrate these two techniques can be called Neuro-Fuzzy systems. Key feature of these systems is that they use input-output patterns to adjust the fuzzy sets and rules inside the model. The paper reviews the principles of a Neuro-Fuzzy system and the key methods presented in this field, furthermore provides survey on their applications for technical diagnostics and measurement. © 2015 Elsevier Ltd

Evolving neuro-fuzzy tools for system classification and prediction

"Classification and prediction algorithims have recently become very powerful tools to a wide array of real-world applications. Some real world applications include system condition monitoring, bioinformatics, robotics, predictive control, earthquake prediction, weather forecasting, stock market and traffic pattern prediction, just to name a few. Within this work, several novel approaches, as well as modifications to some existing approaches, are introduced in order to improve the performance of current classification and prediction paradigms. In the first section of this work, a novel weighted recurrent neuro-fuzzy inference system is introduced alongside two existing neural networks. It is found that the novel design outperforms both the existing neural networks in terms of equal-step and sequential-step inputs for time-series forecasting. The second contribution of this work is the development of a novel evolving clustering algorithim for classification and prediction. This particular algorithim starts without any priori knowledge of the distribution of the data set. The novel design is capable of revealing the true cluster configuration in a single pass of the data, estimating the location and variance of each cluster. After a rigorous performance evaluation, it is found that the novel design outperforms many existing clustering approaches including the well-known potential-based evolving Takagi-Sugeno (eTS) clustering scheme. The third and fourth contributions of this work are the development of a second novel clustering technique and a novel hybrid training technique. The clustering technique is a combination of the aforementioned scheme and the potential-based technique. The new training algorithm is a combination of the decoupled-extended Kalman filter (for the backward pass) and the recursive least-sequares estimate (for the forward pass). It is found that the novel clustering technique outperforms many available clustering techniques. Also, the novel training algorithm is proven to outperform most existing training techniques."--Abstrac

Evolving neural fuzzy classifier for machinery diagnostics / by Ofelia Antonia Jianu.

"The classical techniques for fault diagnosis require periodic shut down of machines for manual inspection. Although these techniques can be used for fault diagnosis in simple machines, they can rarely be used effectively for complex ones. Due to the rapid growing market competitiveness, more reliable and robust condition monitoring systems are critically needed in a wide array of industries to improve production quality and reduce cost. As a result, in recent years more efforts have been taken to develop intelligent techniques for online condition monitoring in machinery systems. Several neural fuzzy classification schemes have been proposed in literature for fault detection. However, the reasoning architecture of the classical neural fuzzy classifiers remains fixed, allowing only the system parameters to be updated in pattern classification operations. To improve the reliability of machinery fault diagnostics, an evolving fuzzy classifier is developed in this work for gear system condition monitoring

Incremental learning for interactive sketch recognition

International audienceIn this paper, we present the integration of a classifier, based on an incremental learning method, in an interactive sketch analyzer. The classifier recognizes the symbol with a degree of confidence. Sometimes the analyzer considers that the response is insufficient to make the right decision. The decision process then solicits the user to explicitly validate the right decision. The user associates the symbol to an existing class, to a newly created class or ignores this recognition. The classifier learns during the interpretation phase. We can thus have a method for auto-evolutionary interpretation of sketches. In fact, the user participation has a great impact to avoid error accumulation during the analysis. This paper demonstrates this integration in an interactive method based on a competitive breadth-first exploration of the analysis tree for interpreting the 2D architectural floor plans

Systèmes d'inférence floue auto-évolutifs : apprentissage incrémental pour la reconnaissance de gestes manuscrits

International audienceNous présentons dans ce papier une nouvelle méthode pour la conception de moteurs de reconnaissance de gestes manuscrits personnalisables et auto-évolutifs, c'est-à-dire capables de s'adapter au style d'écriture et aux habitudes de chacun, sans toutefois nécessiter de période d'apprentissage fastidieuse. Nous utilisons une approche d'apprentissage incrémental de classifieurs basés sur les systèmes d'inférence floue de type Takagi-Sugeno. Cette approche comprend d'une part, une adaptation des paramètres linéaires associés aux conclusions des règles en utilisant la méthode des moindres carrés récursifs, et d'autre part, un apprentissage incrémental des prémisses de ces règles afin de modifier les fonctions d'appartenance suivant l'évolution de la densité des données dans l'espace de classification

Multi-Objective Evolutionary Optimisation for Prototype-Based Fuzzy Classifiers

Evolving intelligent systems (EISs), particularly, the zero-order ones have demonstrated strong performance on many real-world problems concerning data stream classification, while offering high model transparency and interpretability thanks to their prototype-based nature. Zero-order EISs typically learn prototypes by clustering streaming data online in a “one pass” manner for greater computation efficiency. However, such identified prototypes often lack optimality, resulting in less precise classification boundaries, thereby hindering the potential classification performance of the systems. To address this issue, a commonly adopted strategy is to minimise the training error of the models on historical training data or alternatively, to iteratively minimise the intra-cluster variance of the clusters obtained via online data partitioning. This recognises the fact that the ultimate classification performance of zero-order EISs is driven by the positions of prototypes in the data space. Yet, simply minimising the training error may potentially lead to overfitting, whilst minimising the intra-cluster variance does not necessarily ensure the optimised prototype-based models to attain improved classification outcomes. To achieve better classification performance whilst avoiding overfitting for zero-order EISs, this paper presents a novel multi-objective optimisation approach, enabling EISs to obtain optimal prototypes via involving these two disparate but complementary strategies simultaneously. Five decision-making schemes are introduced for selecting a suitable solution to deploy from the final non-dominated set of the resulting optimised models. Systematic experimental studies are carried out to demonstrate the effectiveness of the proposed optimisation approach in improving the classification performance of zero-order EISs