Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.Comment: IEEE Transactions on Fuzzy System

An Overview of Classifier Fusion Methods

A number of classifier fusion methods have been recently developed opening an alternative approach leading to a potential improvement in the classification performance. As there is little theory of information fusion itself, currently we are faced with different methods designed for different problems and producing different results. This paper gives an overview of classifier fusion methods and attempts to identify new trends that may dominate this area of research in future. A taxonomy of fusion methods trying to bring some order into the existing “pudding of diversities” is also provided

An Overview of Classifier Fusion Methods

A number of classifier fusion methods have been recently developed opening an alternative approach leading to a potential improvement in the classification performance. As there is little theory of information fusion itself, currently we are faced with different methods designed for different problems and producing different results. This paper gives an overview of classifier fusion methods and attempts to identify new trends that may dominate this area of research in future. A taxonomy of fusion methods trying to bring some order into the existing “pudding of diversities” is also provided

Efficient Data Driven Multi Source Fusion

Data/information fusion is an integral component of many existing and emerging applications; e.g., remote sensing, smart cars, Internet of Things (IoT), and Big Data, to name a few. While fusion aims to achieve better results than what any one individual input can provide, often the challenge is to determine the underlying mathematics for aggregation suitable for an application. In this dissertation, I focus on the following three aspects of aggregation: (i) efficient data-driven learning and optimization, (ii) extensions and new aggregation methods, and (iii) feature and decision level fusion for machine learning with applications to signal and image processing. The Choquet integral (ChI), a powerful nonlinear aggregation operator, is a parametric way (with respect to the fuzzy measure (FM)) to generate a wealth of aggregation operators. The FM has 2N variables and N(2N − 1) constraints for N inputs. As a result, learning the ChI parameters from data quickly becomes impractical for most applications. Herein, I propose a scalable learning procedure (which is linear with respect to training sample size) for the ChI that identifies and optimizes only data-supported variables. As such, the computational complexity of the learning algorithm is proportional to the complexity of the solver used. This method also includes an imputation framework to obtain scalar values for data-unsupported (aka missing) variables and a compression algorithm (lossy or losselss) of the learned variables. I also propose a genetic algorithm (GA) to optimize the ChI for non-convex, multi-modal, and/or analytical objective functions. This algorithm introduces two operators that automatically preserve the constraints; therefore there is no need to explicitly enforce the constraints as is required by traditional GA algorithms. In addition, this algorithm provides an efficient representation of the search space with the minimal set of vertices. Furthermore, I study different strategies for extending the fuzzy integral for missing data and I propose a GOAL programming framework to aggregate inputs from heterogeneous sources for the ChI learning. Last, my work in remote sensing involves visual clustering based band group selection and Lp-norm multiple kernel learning based feature level fusion in hyperspectral image processing to enhance pixel level classification

Fusion of an Ensemble of Augmented Image Detectors for Robust Object Detection

A significant challenge in object detection is accurate identification of an object's position in image space, whereas one algorithm with one set of parameters is usually not enough, and the fusion of multiple algorithms and/or parameters can lead to more robust results. Herein, a new computational intelligence fusion approach based on the dynamic analysis of agreement among object detection outputs is proposed. Furthermore, we propose an online versus just in training image augmentation strategy. Experiments comparing the results both with and without fusion are presented. We demonstrate that the augmented and fused combination results are the best, with respect to higher accuracy rates and reduction of outlier influences. The approach is demonstrated in the context of cone, pedestrian and box detection for Advanced Driver Assistance Systems (ADAS) applications.Comment: 21 pages, 12 figures, journal paper, MDPI Sensors, 201

Neuro-inspired edge feature fusion using Choquet integrals

It is known that the human visual system performs a hierarchical information process in which early vision cues (or primitives) are fused in the visual cortex to compose complex shapes and descriptors. While different aspects of the process have been extensively studied, such as lens adaptation or feature detection, some other aspects, such as feature fusion, have been mostly left aside. In this work, we elaborate on the fusion of early vision primitives using generalizations of the Choquet integral, and novel aggregation operators that have been extensively studied in recent years. We propose to use generalizations of the Choquet integral to sensibly fuse elementary edge cues, in an attempt to model the behaviour of neurons in the early visual cortex. Our proposal leads to a fully-framed edge detection algorithm whose performance is put to the test in state-of-the-art edge detection datasets.The authors gratefully acknowledge the financial support of the Spanish Ministry of Science and Technology (project PID2019-108392GB-I00 (AEI/10.13039/501100011033), the Research Services of Universidad Pública de Navarra, CNPq (307781/2016-0, 301618/2019-4), FAPERGS (19/2551-0001660) and PNPD/CAPES (464880/2019-00)

Use of the Choquet Integral for Combination of Classifiers in P300 Based Brain-Computer Interface

One of the key issues in the development of braincomputer interfaces (BCIs) is the improvement of their current information transfer rate. In order to achieve that objective at least two aspects of BCI design should be considered: classification accuracy and protocol specification. In this paper we show how combination of classifiers using fuzzy measures and the Choquet integral can be applied to the context of EEG-based BCI and study whether its use, together with an appropriate application protocol, can lead to an increase in the information transfer rate

A multi-attribute decision making procedure using fuzzy numbers and hybrid aggregators

The classical Analytical Hierarchy Process (AHP) has two limitations. Firstly, it disregards the aspect of uncertainty that usually embedded in the data or information expressed by human. Secondly, it ignores the aspect of interdependencies among attributes during aggregation. The application of fuzzy numbers aids in confronting the former issue whereas, the usage of Choquet Integral operator helps in dealing with the later issue. However, the application of fuzzy numbers into multi-attribute decision making (MADM) demands some additional steps and inputs from decision maker(s). Similarly, identification of monotone measure weights prior to employing Choquet Integral requires huge number of computational steps and amount of inputs from decision makers, especially with the increasing number of attributes. Therefore, this research proposed a MADM procedure which able to reduce the number of computational steps and amount of information required from the decision makers when dealing with these two aspects simultaneously. To attain primary goal of this research, five phases were executed. First, the concept of fuzzy set theory and its application in AHP were investigated. Second, an analysis on the aggregation operators was conducted. Third, the investigation was narrowed on Choquet Integral and its associate monotone measure. Subsequently, the proposed procedure was developed with the convergence of five major components namely Factor Analysis, Fuzzy-Linguistic Estimator, Choquet Integral, Mikhailov‘s Fuzzy AHP, and Simple Weighted Average. Finally, the feasibility of the proposed procedure was verified by solving a real MADM problem where the image of three stores located in Sabak Bernam, Selangor, Malaysia was analysed from the homemakers‘ perspective. This research has a potential in motivating more decision makers to simultaneously include uncertainties in human‘s data and interdependencies among attributes when solving any MADM problems