An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams

Existing FNNs are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This paper proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. DEVFNN is developed under the stacked generalization principle via the feature augmentation concept where a recently developed algorithm, namely gClass, drives the hidden layer. It is equipped by an automatic feature selection method which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent uncontrollable growth of dimensionality of input space due to the nature of feature augmentation approach in building a deep network structure. DEVFNN works in the sample-wise fashion and is compatible for data stream applications. The efficacy of DEVFNN has been thoroughly evaluated using seven datasets with non-stationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart where DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept drift detection method is an effective tool to control the depth of network structure while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance.Comment: This paper has been published in IEEE Transactions on Fuzzy System

Towards Anthropomorphic Machine Learning

In this paper, we introduce and discuss the concept of anthropomorphic machine learning as an emerging direction for the future development in the area of artificial intelligence (AI) and data science. We start with outlining research challenges and opportunities, which the contemporary landscape offers. We focus on machine learning, statistical learning, deep learning and computational intelligence as theoretical and methodological areas of greater promise for breakthrough results and underpinning the future revolutionary changes in technology development as well as in our everyday life and societies. Our critical analysis brings us to the open problems and we formulate the paradigm shift in the understanding of machine learning. In a nutshell, our vision for the next generational machine learning methods and algorithms is anthropomorphic, which resembles the way people/humans learn from data. This concept brings machine learning from the statistics to the area of computational intelligence and AI

A Semi-Supervised Deep Rule-Based Approach for Complex Satellite Sensor Image Analysis

Large-scale (large-area), fine spatial resolution satellite sensor images are valuable data sources for Earth observation while not yet fully exploited by research communities for practical applications. Often, such images exhibit highly complex geometrical structures and spatial patterns, and distinctive characteristics of multiple land-use categories may appear at the same region. Autonomous information extraction from these images is essential in the field of pattern recognition within remote sensing, but this task is extremely challenging due to the spectral and spatial complexity captured in satellite sensor imagery. In this research, a semi-supervised deep rule-based approach for satellite sensor image analysis (SeRBIA) is proposed, where large-scale satellite sensor images are analysed autonomously and classified into detailed land-use categories. Using an ensemble feature descriptor derived from pre-trained AlexNet and VGG-VD-16 models, SeRBIA is capable of learning continuously from both labelled and unlabelled images through self-adaptation without human involvement or intervention. Extensive numerical experiments were conducted on both benchmark datasets and real-world satellite sensor images to comprehensively test the validity and effectiveness of the proposed method. The novel information mining technique developed here can be applied to analyse large-scale satellite sensor images with high accuracy and interpretability, across a wide range of real-world applications

Highly interpretable hierarchical deep rule-based classifier

Pioneering the traditional fuzzy rule-based (FRB) systems, deep rule-based (DRB) classifiers are able to offer both human-level performance and transparent system structure on image classification problems by integrating zero-order fuzzy rule base with a multi-layer image-processing architecture that is typical for deep learning. Nonetheless, it is frequently observed that the inner structure of DRB can become over sophisticated and not interpretable for humans when applied to large-scale, complex problems. To tackle the issue, one feasible solution is to construct a tree structural classification model by aggregating the possibly huge number of prototypes identified from data into a much smaller number of more descriptive and highly abstract ones. Therefore, in this paper, we present a novel hierarchical deep rule-based (H-DRB) approach that is capable of summarizing the less descriptive raw prototypes into highly generalized ones and self-arranging them into a hierarchical prototype-based structure according to their descriptive abilities. By doing so, H-DRB can offer high-level performance and, most importantly, full transparency and human-interpretability on various problems including large-scale ones. The proposed concept and generical principles are verified through numerical experiments based on a wide variety of popular benchmark image sets. Numerical results demonstrate that the promise of H-DRB

Semi-supervised deep rule-based approach for image classification

In this paper, a semi-supervised learning approach based on a deep rule-based (DRB) classifier is introduced. With its unique prototype-based nature, the semi-supervised DRB (SSDRB) classifier is able to generate human interpretable IF...THEN...rules through the semi-supervised learning process in a self-organising and highly transparent manner. It supports online learning on a sample-by-sample basis or on a chunk-by-chunk basis. It is also able to perform classification on out-of-sample images. Moreover, the SSDRB classifier can learn new classes from unlabelled images in an active way becoming dynamically self-evolving. Numerical examples based on large-scale benchmark image sets demonstrate the strong performance of the proposed SSDRB classifier as well as its distinctive features compared with the “state-of-the-art” approaches