Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms

AbstractThis paper presents a hybrid method for identification of Pareto-optimal fuzzy classifiers (FCs). In contrast to many existing methods, the initial population for multiobjective evolutionary algorithms (MOEAs) is neither created randomly nor a priori knowledge is required. Instead, it is created by the proposed two-step initialization method. First, a decision tree (DT) created by C4.5 algorithm is transformed into an FC. Therefore, relevant variables are selected and initial partition of input space is performed. Then, the rest of the population is created by randomly replacing some parameters of the initial FC, such that, the initial population is widely spread. That improves the convergence of MOEAs into the correct Pareto front. The initial population is optimized by NSGA-II algorithm and a set of Pareto-optimal FCs representing the trade-off between accuracy and interpretability is obtained. The method does not require any a priori knowledge of the number of fuzzy sets, distribution of fuzzy sets or the number of relevant variables. They are all determined by it. Performance of the obtained FCs is validated by six benchmark data sets from the literature. The obtained results are compared to a recently published paper [H. Ishibuchi, Y. Nojima, Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning, International Journal of Approximate Reasoning 44 (1) (2007) 4–31] and the benefits of our method are clearly shown

Coevolutionary fuzzy modeling

This thesis presents Fuzzy CoCo, a novel approach for system design, conducive to explaining human decisions. Based on fuzzy logic and coevolutionary computation, Fuzzy CoCo is a methodology for constructing systems able to accurately predict the outcome of a human decision-making process, while providing an understandable explanation of the underlying reasoning. Fuzzy logic provides a formal framework for constructing systems exhibiting both good numeric performance (precision) and linguistic representation (interpretability). From a numeric point of view, fuzzy systems exhibit nonlinear behavior and can handle imprecise and incomplete information. Linguistically, they represent knowledge in the form of rules, a natural way for explaining decision processes. Fuzzy modeling —meaning the construction of fuzzy systems— is an arduous task, demanding the identification of many parameters. This thesis analyses the fuzzy-modeling problem and different approaches to coping with it, focusing on evolutionary fuzzy modeling —the design of fuzzy inference systems using evolutionary algorithms— which constitutes the methodological base of my approach. In order to promote this analysis the parameters of a fuzzy system are classified into four categories: logic, structural, connective, and operational. The central contribution of this work is the use of an advanced evolutionary technique —cooperative coevolution— for dealing with the simultaneous design of connective and operational parameters. Cooperative coevolutionary fuzzy modeling succeeds in overcoming several limitations exhibited by other standard evolutionary approaches: stagnation, convergence to local optima, and computational costliness. Designing interpretable systems is a prime goal of my approach, which I study thoroughly herein. Based on a set of semantic and syntactic criteria, regarding the definition of linguistic concepts and their causal connections, I propose a number of strategies for producing more interpretable fuzzy systems. These strategies are implemented in Fuzzy CoCo, resulting in a modeling methodology providing high numeric precision, while incurring as little a loss of interpretability as possible. After testing Fuzzy CoCo on a benchmark problem —Fisher's Iris data— I successfully apply the algorithm to model the decision processes involved in two breast-cancer diagnostic problems: the WBCD problem and the Catalonia mammography interpretation problem. For the WBCD problem, Fuzzy CoCo produces systems both of high performance and high interpretability, comparable (if not better) than the best systems demonstrated to date. For the Catalonia problem, an evolved high-performance system was embedded within a web-based tool —called COBRA— for aiding radiologists in mammography interpretation. Several aspects of Fuzzy CoCo are thoroughly analyzed to provide a deeper understanding of the method. These analyses show the consistency of the results. They also help derive a stepwise guide to applying Fuzzy CoCo, and a set of qualitative relationships between some of its parameters that facilitate setting up the algorithm. Finally, this work proposes and explores preliminarily two extensions to the method: Island Fuzzy CoCo and Incremental Fuzzy CoCo, which together with the original CoCo constitute a family of coevolutionary fuzzy modeling techniques. The aim of these extensions is to guide the choice of an adequate number of rules for a given problem. While Island Fuzzy CoCo performs an extended search over different problem sizes, Incremental Fuzzy CoCo bases its search power on a mechanism of incremental evolution

A Survey on Evolutionary Computation for Computer Vision and Image Analysis: Past, Present, and Future Trends

Computer vision (CV) is a big and important field in artificial intelligence covering a wide range of applications. Image analysis is a major task in CV aiming to extract, analyse and understand the visual content of images. However, imagerelated tasks are very challenging due to many factors, e.g., high variations across images, high dimensionality, domain expertise requirement, and image distortions. Evolutionary computation (EC) approaches have been widely used for image analysis with significant achievement. However, there is no comprehensive survey of existing EC approaches to image analysis. To fill this gap, this paper provides a comprehensive survey covering all essential EC approaches to important image analysis tasks including edge detection, image segmentation, image feature analysis, image classification, object detection, and others. This survey aims to provide a better understanding of evolutionary computer vision (ECV) by discussing the contributions of different approaches and exploring how and why EC is used for CV and image analysis. The applications, challenges, issues, and trends associated to this research field are also discussed and summarised to provide further guidelines and opportunities for future research

Data and Feature Reduction in Fuzzy Modeling through Particle Swarm Optimization

The study is concerned with data and feature reduction in fuzzy modeling. As these reduction activities are advantageous to fuzzy models in terms of both the effectiveness of their construction and the interpretation of the resulting models, their realization deserves particular attention. The formation of a subset of meaningful features and a subset of essential instances is discussed in the context of fuzzy-rule-based models. In contrast to the existing studies, which are focused predominantly on feature selection (namely, a reduction of the input space), a position advocated here is that a reduction has to involve both data and features to become efficient to the design of fuzzy model. The reduction problem is combinatorial in its nature and, as such, calls for the use of advanced optimization techniques. In this study, we use a technique of particle swarm optimization (PSO) as an optimization vehicle of forming a subset of features and data (instances) to design a fuzzy model. Given the dimensionality of the problem (as the search space involves both features and instances), we discuss a cooperative version of the PSO along with a clustering mechanism of forming a partition of the overall search space. Finally, a series of numeric experiments using several machine learning data sets is presented

Curvature-based sparse rule base generation for fuzzy rule interpolation

Fuzzy logic has been successfully widely utilised in many real-world applications. The most common application of fuzzy logic is the rule-based fuzzy inference system, which is composed of mainly two parts including an inference engine and a fuzzy rule base. Conventional fuzzy inference systems always require a rule base that fully covers the entire problem domain (i.e., a dense rule base). Fuzzy rule interpolation (FRI) makes inference possible with sparse rule bases which may not cover some parts of the problem domain (i.e., a sparse rule base). In addition to extending the applicability of fuzzy inference systems, fuzzy interpolation can also be used to reduce system complexity for over-complex fuzzy inference systems. There are typically two methods to generate fuzzy rule bases, i.e., the knowledge driven and data-driven approaches. Almost all of these approaches only target dense rule bases for conventional fuzzy inference systems. The knowledge-driven methods may be negatively affected by the limited availability of expert knowledge and expert knowledge may be subjective, whilst redundancy often exists in fuzzy rule-based models that are acquired from numerical data. Note that various rule base reduction approaches have been proposed, but they are all based on certain similarity measures and are likely to cause performance deterioration along with the size reduction. This project, for the first time, innovatively applies curvature values to distinguish important features and instances in a dataset, to support the construction of a neat and concise sparse rule base for fuzzy rule interpolation. In addition to working in a three-dimensional problem space, the work also extends the natural three-dimensional curvature calculation to problems with high dimensions, which greatly broadens the applicability of the proposed approach. As a result, the proposed approach alleviates the ‘curse of dimensionality’ and helps to reduce the computational cost for fuzzy inference systems. The proposed approach has been validated and evaluated by three real-world applications. The experimental results demonstrate that the proposed approach is able to generate sparse rule bases with less rules but resulting in better performance, which confirms the power of the proposed system. In addition to fuzzy rule interpolation, the proposed curvature-based approach can also be readily used as a general feature selection tool to work with other machine learning approaches, such as classifiers

Heuristic design of fuzzy inference systems: a review of three decades of research

This paper provides an in-depth review of the optimal design of type-1 and type-2 fuzzy inference systems (FIS) using five well known computational frameworks: genetic-fuzzy systems (GFS), neuro-fuzzy systems (NFS), hierarchical fuzzy systems (HFS), evolving fuzzy systems (EFS), and multi-objective fuzzy systems (MFS), which is in view that some of them are linked to each other. The heuristic design of GFS uses evolutionary algorithms for optimizing both Mamdani-type and Takagi–Sugeno–Kang-type fuzzy systems. Whereas, the NFS combines the FIS with neural network learning systems to improve the approximation ability. An HFS combines two or more low-dimensional fuzzy logic units in a hierarchical design to overcome the curse of dimensionality. An EFS solves the data streaming issues by evolving the system incrementally, and an MFS solves the multi-objective trade-offs like the simultaneous maximization of both interpretability and accuracy. This paper offers a synthesis of these dimensions and explores their potentials, challenges, and opportunities in FIS research. This review also examines the complex relations among these dimensions and the possibilities of combining one or more computational frameworks adding another dimension: deep fuzzy systems

Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling

This paper aims at providing an in-depth overview of designing interpretable fuzzy inference models from data within a unified framework. The objective of complex system modelling is to develop reliable and understandable models for human being to get insights into complex real-world systems whose first-principle models are unknown. Because system behaviour can be described naturally as a series of linguistic rules, data-driven fuzzy modelling becomes an attractive and widely used paradigm for this purpose. However, fuzzy models constructed from data by adaptive learning algorithms usually suffer from the loss of model interpretability. Model accuracy and interpretability are two conflicting objectives, so interpretation preservation during adaptation in data-driven fuzzy system modelling is a challenging task, which has received much attention in fuzzy system modelling community. In order to clearly discriminate the different roles of fuzzy sets, input variables, and other components in achieving an interpretable fuzzy model, a taxonomy of fuzzy model interpretability is first proposed in terms of low-level interpretability and high-level interpretability in this paper. The low-level interpretability of fuzzy models refers to fuzzy model interpretability achieved by optimizing the membership functions in terms of semantic criteria on fuzzy set level, while the high-level interpretability refers to fuzzy model interpretability obtained by dealing with the coverage, completeness, and consistency of the rules in terms of the criteria on fuzzy rule level. Some criteria for low-level interpretability and high-level interpretability are identified, respectively. Different data-driven fuzzy modelling techniques in the literature focusing on the interpretability issues are reviewed and discussed from the perspective of low-level interpretability and high-level interpretability. Furthermore, some open problems about interpretable fuzzy models are identified and some potential new research directions on fuzzy model interpretability are also suggested. Crown Copyright © 2008

Advances in Evolutionary Algorithms

With the recent trends towards massive data sets and significant computational power, combined with evolutionary algorithmic advances evolutionary computation is becoming much more relevant to practice. Aim of the book is to present recent improvements, innovative ideas and concepts in a part of a huge EA field

Automated Telescience: Active Machine Learning Of Remote Dynamical Systems

Automated science is an emerging field of research and technology that aims to extend the role of computers in science from a tool that stores and analyzes data to one that generates hypotheses and designs experiments. Despite the tremendous discoveries and advancements brought forth by the scientific method, it is a process that is fundamentally driven by human insight and ingenuity. Automated science aims to develop algorithms, protocols and design philosophies that are capable of automating the scientific process. This work presents advances the field of automated science and the specific contributions of this work fall into three categories: coevolutionary search methods and applications, inferring the underlying structure of dynamical systems, and remote controlled automated science. First, a collection of coevolutionary search methods and applications are presented. These approaches include: a method to reduce the computational overhead of evolutionary algorithms via trainer selection strategies in a rank predictor framework, an approach for optimal experiment design for nonparametric models using Shannon information, and an application of coevolutionary algorithms to infer kinematic poses from RGBD images. Second, three algorithms are presented that infer the underlying structure of dynamical systems: a method to infer discrete-continuous hybrid dynamical systems from unlabeled data, an approach to discovering ordinary differential equations of arbitrary order, and a principle to uncover the existence and dynamics of hidden state variables that correspond to physical quantities from nonlinear differential equations. All of these algorithms are able to uncover structure in an unsupervised manner without any prior domain knowledge. Third, a remote controlled, distributed system is demonstrated to autonomously generate scientific models by perturbing and observing a system in an intelligent fashion. By automating the components of physical experimentation, scientific modeling and experimental design, models of luminescent chemical reactions and multi-compartmental pharmacokinetic systems were discovered without any human intervention, which illustrates how a set of distributed machines can contribute scientific knowledge while scaling beyond geographic constraints