Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective

Studies in Evolutionary Fuzzy Systems (EFSs) began in the 90s and have experienced a fast development since then, with applications to areas such as pattern recognition, curve‐fitting and regression, forecasting and control. An EFS results from the combination of a Fuzzy Inference System (FIS) with an Evolutionary Algorithm (EA). This relationship can be established for multiple purposes: fine‐tuning of FIS's parameters, selection of fuzzy rules, learning a rule base or membership functions from scratch, and so forth. Each facet of this relationship creates a strand in the literature, as membership function fine‐tuning, fuzzy rule‐based learning, and so forth and the purpose here is to outline some of what has been done in each aspect. Special focus is given to Genetic Programming‐based EFSs by providing a taxonomy of the main architectures available, as well as by pointing out the gaps that still prevail in the literature. The concluding remarks address some further topics of current research and trends, such as interpretability analysis, multiobjective optimization, and synthesis of a FIS through Evolving methods

Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification

Classification techniques are becoming essential in the financial world for reducing risks and possible disasters. Managers are interested in not only high accuracy, but in interpretability and transparency as well. It is widely accepted now that the comprehension of how inputs and outputs are related to each other is crucial for taking operative and strategic decisions. Furthermore, inputs are often affected by contextual factors and characterized by a high level of uncertainty. In addition, financial data are usually highly skewed toward the majority class. With the aim of achieving high accuracies, preserving the interpretability, and managing uncertain and unbalanced data, this paper presents a novel method to deal with financial data classification by adopting type-2 fuzzy rule-based classifiers (FRBCs) generated from data by a multiobjective evolutionary algorithm (MOEA). The classifiers employ an approach, denoted as scaled dominance, for defining rule weights in such a way to help minority classes to be correctly classified. In particular, we have extended PAES-RCS, an MOEA-based approach to learn concurrently the rule and data bases of FRBCs, for managing both interval type-2 fuzzy sets and unbalanced datasets. To the best of our knowledge, this is the first work that generates type-2 FRBCs by concurrently maximizing accuracy and minimizing the number of rules and the rule length with the objective of producing interpretable models of real-world skewed and incomplete financial datasets. The rule bases are generated by exploiting a rule and condition selection (RCS) approach, which selects a reduced number of rules from a heuristically generated rule base and a reduced number of conditions for each selected rule during the evolutionary process. The weight associated with each rule is scaled by the scaled dominance approach on the fuzzy frequency of the output class, in order to give a higher weight to the minority class. As regards the data base learning, the membership function parameters of the interval type-2 fuzzy sets used in the rules are learned concurrently to the application of RCS. Unbalanced datasets are managed by using, in addition to complexity, selectivity and specificity as objectives of the MOEA rather than only the classification rate. We tested our approach, named IT2-PAES-RCS, on 11 financial datasets and compared our results with the ones obtained by the original PAES-RCS with three objectives and with and without scaled dominance, the FRBCs, fuzzy association rule-based classification model for high-dimensional dataset (FARC-HD) and fuzzy unordered rules induction algorithm (FURIA), the classical C4.5 decision tree algorithm, and its cost-sensitive version. Using nonparametric statistical tests, we will show that IT2-PAES-RCS generates FRBCs with, on average, accuracy statistically comparable with and complexity lower than the ones generated by the two versions of the original PAES-RCS. Further, the FRBCs generated by FARC-HD and FURIA and the decision trees computed by C4.5 and its cost-sensitive version, despite the highest complexity, result to be less accurate than the FRBCs generated by IT2-PAES-RCS. Finally, we will highlight how these FRBCs are easily interpretable by showing and discussing one of them

HYEI: A New Hybrid Evolutionary Imperialist Competitive Algorithm for Fuzzy Knowledge Discovery

In recent years, imperialist competitive algorithm (ICA), genetic algorithm (GA), and hybrid fuzzy classification systems have been successfully and effectively employed for classification tasks of data mining. Due to overcoming the gaps related to ineffectiveness of current algorithms for analysing high-dimension independent datasets, a new hybrid approach, named HYEI, is presented to discover generic rule-based systems in this paper. This proposed approach consists of three stages and combines an evolutionary-based fuzzy system with two ICA procedures to generate high-quality fuzzy-classification rules. Initially, the best feature subset is selected by using the embedded ICA feature selection, and then these features are used to generate basic fuzzy-classification rules. Finally, all rules are optimized by using an ICA algorithm to reduce their length or to eliminate some of them. The performance of HYEI has been evaluated by using several benchmark datasets from the UCI machine learning repository. The classification accuracy attained by the proposed algorithm has the highest classification accuracy in 6 out of the 7 dataset problems and is comparative to the classification accuracy of the 5 other test problems, as compared to the best results previously published

Improving Transparency in Approximate Fuzzy Modeling Using Multi-objective Immune-Inspired Optimisation

In this paper, an immune inspired multi-objective fuzzy modeling (IMOFM) mechanism is proposed specifically for high-dimensional regression problems. For such problems, prediction accuracy is often the paramount requirement. With such a requirement in mind, however, one should also put considerable efforts in eliciting models which are as transparent as possible, a ‘tricky’ exercise in itself. The proposed mechanism adopts a multi-stage modeling procedure and a variable length coding scheme to account for the enlarged search space due to simultaneous optimisation of the rule-base structure and its associated parameters. We claim here that IMOFM can account for both Singleton and Mamdani Fuzzy Rule-Based Systems (FRBS) due to the carefully chosen output membership functions, the inference scheme and the defuzzification method. The proposed modeling approach has been compared to other representatives using a benchmark problem, and was further applied to a high-dimensional problem, taken from the steel industry, which concerns the prediction of mechanical properties of hot rolled steels. Results confirm that IMOFM is capable of eliciting not only accurate but also transparent FRBSs from quantitative data

Hierarchical Fuzzy Systems: Interpretability and Complexity

Hierarchical fuzzy systems (HFSs) have been regarded as a useful solution for overcoming the major issues in fuzzy logic systems (FLSs), i.e., rule explosion due to the increase in the number of input variables. In HFS, the standard FLS are reformed into a low-dimensional FLS subsystem network. Moreover, the rules in HFS usually have antecedents with fewer variables than the rules in standard FLS with equivalent functions, because the number of input variables in each subsystem is less. Consequently, HFSs manage to decrease rule explosion, which minimises complexity and improves model interpretability. Nevertheless, the issues related to the question of “Does the complexity reduction of HFSs that have multiple subsystems, layers and different topologies really improve their interpretability?” are not clear and persist. In this paper, a comparison focusing on interpretability and complexity is made between two HFS’ topologies: parallel and serial. A detailed measurement of the interpretability and complexity with different configurations for both topologies is provided. This comparative study aims to examine the correlation between interpretability and complexity in HFS

A Genetic-Programming-Based Approach for the Learning of Compact Fuzzy Rule-Based Classification Systems

In the design of an interpretable fuzzy rule-based classification system (FRBCS) the precision as much as the simplicity of the extracted knowledge must be considered as objectives. In any inductive learning algorithm, when we deal with problems with a large number of features, the exponential growth of the fuzzy rule search space makes the learning process more difficult. Moreover it leads to an FRBCS with a rule base with a high cardinality. In this paper, we propose a genetic-programming-based method for the learning of an FRBCS, where disjunctive normal form (DNF) rules compete and cooperate among themselves in order to obtain an understandable and compact set of fuzzy rules, which presents a good classification performance with high dimensionality problems. This proposal uses a token competition mechanism to maintain the diversity of the population. The good results obtained with several classification problems support our proposal.Spanish Ministry of Science and Technology TIN-2005-08386-C05-03 and TIN-2005-08386-C05-0

Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling

This paper aims at providing an in-depth overview of designing interpretable fuzzy inference models from data within a unified framework. The objective of complex system modelling is to develop reliable and understandable models for human being to get insights into complex real-world systems whose first-principle models are unknown. Because system behaviour can be described naturally as a series of linguistic rules, data-driven fuzzy modelling becomes an attractive and widely used paradigm for this purpose. However, fuzzy models constructed from data by adaptive learning algorithms usually suffer from the loss of model interpretability. Model accuracy and interpretability are two conflicting objectives, so interpretation preservation during adaptation in data-driven fuzzy system modelling is a challenging task, which has received much attention in fuzzy system modelling community. In order to clearly discriminate the different roles of fuzzy sets, input variables, and other components in achieving an interpretable fuzzy model, a taxonomy of fuzzy model interpretability is first proposed in terms of low-level interpretability and high-level interpretability in this paper. The low-level interpretability of fuzzy models refers to fuzzy model interpretability achieved by optimizing the membership functions in terms of semantic criteria on fuzzy set level, while the high-level interpretability refers to fuzzy model interpretability obtained by dealing with the coverage, completeness, and consistency of the rules in terms of the criteria on fuzzy rule level. Some criteria for low-level interpretability and high-level interpretability are identified, respectively. Different data-driven fuzzy modelling techniques in the literature focusing on the interpretability issues are reviewed and discussed from the perspective of low-level interpretability and high-level interpretability. Furthermore, some open problems about interpretable fuzzy models are identified and some potential new research directions on fuzzy model interpretability are also suggested. Crown Copyright © 2008