Fuzzy Integral Driven Ensemble Classification using A Priori Fuzzy Measures

Aggregation operators are mathematical functions that enable the fusion of information from multiple sources. Fuzzy Integrals (FIs) are widely used aggregation operators, which combine information in respect to a Fuzzy Measure (FM) which captures the worth of both the individual sources and all their possible combinations. However, FIs suffer from the potential drawback of not fusing information according to the intuitively interpretable FM, leading to non-intuitive results. The latter is particularly relevant when a FM has been defined using external information (e.g. experts). In order to address this and provide an alternative to the FI, the Recursive Average (RAV) aggregation operator was recently proposed which enables intuitive data fusion in respect to a given FM. With an alternative fusion operator in place, in this paper, we define the concept of ‘A Priori’ FMs which are generated based on external information (e.g. classification accuracy) and thus provide an alternative to the traditional approaches of learning or manually specifying FMs. We proceed to develop one specific instance of such an a priori FM to support the decision level fusion step in ensemble classification. We evaluate the resulting approach by contrasting the performance of the ensemble classifiers for different FMs, including the recently introduced Uriz and the Sugeno lambda-measure; as well as by employing both the Choquet FI and the RAV as possible fusion operators. Results are presented for 20 datasets from machine learning repositories and contextualised to the wider literature by comparing them to state-of-the-art ensemble classifiers such as Adaboost, Bagging, Random Forest and Majority Voting

Interpretability indices for hierarchical fuzzy systems

Hierarchical fuzzy systems (HFSs) have been shown to have the potential to improve interpretability of fuzzy logic systems (FLSs). In recent years, a variety of indices have been proposed to measure the interpretability of FLSs such as the Nauck index and Fuzzy index. However, interpretability indices associated with HFSs have not so far been discussed. The structure of HFSs, with multiple layers, subsystems, and varied topologies, is the main challenge in constructing interpretability indices for HFSs. Thus, the comparison of interpretability between FLSs and HFSs-even at the index level-is still subject to open discussion. This paper begins to address these challenges by introducing extensions to the FLS Nauck and Fuzzy interpretability indices for HFSs. Using the proposed indices, we explore the concept of interpretability in relation to the different structures in FLSs and HFSs. Initial experiments on benchmark datasets show that based on the proposed indices, HFSs with equivalent function to FLSs produce higher indices, i.e. are more interpretable than their corresponding FLSs

On transitioning from type-1 to interval type-2 fuzzy logic systems

Capturing the uncertainty arising from system noise has been a core feature of fuzzy logic systems (FLSs) for many years. This paper builds on previous work and explores the methodological transition of type-l (Tl) to interval type-2 fuzzy sets (IT2 FSs) for given "levels" of uncertainty. Specifically, we propose to transition from Tl to IT2 FLSs through varying the size of the Footprint Of Uncertainty (FOU) of their respective FSs while maintaining the original FS shape (e.g., triangular) and keeping the size of the FOU over the FS as constant as possible. The latter is important as it enables the systematic relating of FOU size to levels of uncertainty and vice versa, while the former enables an intuitive comparison between the Tl and T2 FSs. The effectiveness of the proposed method is demonstrated through a series of experiments using the well-known Mackey-Glass (MG) time series prediction problem. The results are compared with the results of the IT2 FS creation method introduced in [1] which follows a similar methodology as the proposed approach but does not maintain the membership function (MF) shape

Contrasting singleton type-1 and interval type-2 non-singleton type-1 fuzzy logic systems

Most applications of both type-1 and type-2 fuzzy logic systems are employing singleton fuzzification due to its simplicity and reduction in its computational speed. However, using singleton fuzzification assumes that the input data (i.e., measurements) are precise with no uncertainty associated with them. This paper explores the potential of combining the uncertainty modelling capacity of interval type-2 fuzzy sets with the simplicity of type-1 fuzzy logic systems (FLSs) by using interval type-2 fuzzy sets solely as part of the non-singleton input fuzzifier. This paper builds on previous work and uses the methodological design of the footprint of uncertainty (FOU) of interval type-2 fuzzy sets for given levels of uncertainty. We provide a detailed investigation into the ability of both types of fuzzy sets (type-1 and interval type-2) to capture and model different levels of uncertainty/noise through varying the size of the FOU of the underlying input fuzzy sets from type-1 fuzzy sets to very “wide” interval type-2 fuzzy sets as part of type-1 non-singleton FLSs using interval type-2 input fuzzy sets. By applying the study in the context of chaotic time-series prediction, we show how, as uncertainty/noise increases, interval type-2 input fuzzy sets with FOUs of increasing size become more and more viable

Improved uncertainty capture for nonsingleton fuzzy systems

In non-singleton fuzzy logic systems (NSFLSs), input uncertainties are modelled with input fuzzy sets in order to capture input uncertainty (e.g., sensor noise). The performance of NSFLSs in handling such uncertainties depends on both: the appropriate modelling in the input fuzzy sets of the uncertainties present in the system’s inputs, and on how the input fuzzy sets (and their inherent model of uncertainty) interact with the antecedent and thus affect the inference within the remainder of the NSFLS. This paper proposes a novel development on the latter. Specifically, an alteration to the standard composition method of type-1 fuzzy relations is proposed, and applied to build a new type of NSFLS. The proposed approach is based on employing the centroid of the intersection of input and antecedent sets as origin of the firing degree, rather than the traditional maximum of their intersection, thus making the NSFLS more sensitive to changes in the input’s uncertainty characteristics. The traditional and novel approach to NSFLSs are experimentally compared for two well-known problems of Mackey-Glass and Lorenz chaotic time series predictions, where the NSFLSs’ inputs have been perturbed with different levels of Gaussian noise. Experiments are repeated for system training under noisy and noise-free conditions. Analyses of the results show that the new method outperforms the traditional approach. Moreover, it is shown that while formally more complex, in practice, the new method has no significant computational overhead compared to the standard approach

An illustration of new methods in machine condition monitoring, Part I: Stochastic resonance

There have been many recent developments in the application of data-based methods to machine condition monitoring. A powerful methodology based on machine learning has emerged, where diagnostics are based on a two-step procedure: extraction of damage sensitive features, followed by unsupervised learning (novelty detection) or supervised learning (classification). The objective of the current pair of papers is simply to illustrate one state-of the-art procedure for each step, using synthetic data representative of reality in terms of size and complexity. The first paper in the pair will deal with feature extraction. Although some papers have appeared in the recent past considering stochastic resonance as a means of amplifying damage information in signals, they have largely relied on ad hoc specifications of the resonator used. In contrast, the current paper will adopt a principled optimisation-based approach to the resonator design. The paper will also show that a discrete dynamical system can provide all the benefits of a continuous system, but also provide a considerable speed-up in terms of simulation time in order to facilitate the optimisation approach

Comparison of Fuzzy Integral-Fuzzy Measure based Ensemble Algorithms with the State-of-the-art Ensemble Algorithms

The Fuzzy Integral (FI) is a non-linear aggregation operator which enables the fusion of information from multiple sources in respect to a Fuzzy Measure (FM) which captures the worth of both the individual sources and all their possible combinations. Based on the expected potential of non-linear aggregation offered by the FI, its application to decision-level fusion in ensemble classifiers, i.e. to fuse multiple classifiers outputs towards one superior decision level output, has recently been explored. A key example of such a FI-FM ensemble classification method is the Decision-level Fuzzy Integral Multiple Kernel Learning (DeFIMKL) algorithm, which aggregates the outputs of kernel based classifiers through the use of the Choquet FI with respect to a FM learned through a regularised quadratic programming approach. While the approach has been validated against a number of classifiers based on multiple kernel learning, it has thus far not been compared to the state-of-the-art in ensemble classification. Thus, this paper puts forward a detailed comparison of FI-FM based ensemble methods, specifically the DeFIMKL algorithm, with state-of-the art ensemble methods including Adaboost, Bagging, Random Forest and Majority Voting over 20 public datasets from the UCI machine learning repository. The results on the selected datasets suggest that the FI based ensemble classifier performs both well and efficiently, indicating that it is a viable alternative when selecting ensemble classifiers and indicating that the non-linear fusion of decision level outputs offered by the FI provides expected potential and warrants further study

An Illustration of New Methods in Machine Condition Monitoring, Part II: Adaptive outlier detection

There have been many recent developments in the application of data-based methods to machine condition monitoring. A powerful methodology based on machine learning has emerged, where diagnostics are based on a two-step procedure: extraction of damagesensitive features, followed by unsupervised learning (novelty detection) or supervised learning (classification). The objective of the current pair of papers is simply to illustrate one state-ofthe-art procedure for each step, using synthetic data representative of reality in terms of size and complexity. The second paper in the pair will deal with novelty detection. Although there has been considerable progress in the use of outlier analysis for novelty detection, most of the papers produced so far have suffered from the fact that simple algorithms break down if multiple outliers are present or if damage is already present in a training set. The objective of the current paper is to illustrate the use of phase-space thresholding; an algorithm which has the ability to detect multiple outliers inclusively in a data set

Towards a framework for capturing interpretability of hierarchical fuzzy systems - a participatory design approach

Hierarchical fuzzy systems (HFSs) have been shown to have the potential to improve the interpretability of fuzzy logic systems (FLSs). However, challenges remain, such as: "How can we measure their interpretability?", "How can we make an informed assessment of how HFSs should be designed to enhance interpretability?". The challenges of measuring the interpretability of HFSs include issues such as their topological structure, the number of layers, the meaning of intermediate variables, and so on. In this paper, an initial framework to measure the interpretability of HFSs is proposed, combined with a participatory user design process to create a specific instance of the framework for an application context. This approach enables the subjective views of a range of practitioners, experts in the design and creation of FLSs, to be taken into account in shaping the design of a generic framework for measuring interpretability in HFSs. This design process and framework are demonstrated through two classification application examples, showing the ability of the resulting index to appropriately capture interpretability as perceived by system design experts