An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams

Existing FNNs are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This paper proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. DEVFNN is developed under the stacked generalization principle via the feature augmentation concept where a recently developed algorithm, namely gClass, drives the hidden layer. It is equipped by an automatic feature selection method which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent uncontrollable growth of dimensionality of input space due to the nature of feature augmentation approach in building a deep network structure. DEVFNN works in the sample-wise fashion and is compatible for data stream applications. The efficacy of DEVFNN has been thoroughly evaluated using seven datasets with non-stationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart where DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept drift detection method is an effective tool to control the depth of network structure while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance.Comment: This paper has been published in IEEE Transactions on Fuzzy System

An EM-based identification algorithm for a class of hybrid systems with application to power electronics

In this paper we present an identification algorithm for a class of continuous-time hybrid systems. In such systems, both continuous-time and discrete-time dynamics are involved. We apply the expectation-maximisation algorithm to obtain the maximum likelihood estimate of the parameters of a discrete-time model expressed in incremental form. The main advantage of this approach is that the continuous-time parameters can directly be recovered. The technique is particularly well suited to fast-sampling rates. As an application, we focus on a standard identification problem in power electronics. In this field, our proposed algorithm is of importance since accurate modelling of power converters is required in high- performance applications and for fault diagnosis. As an illustrative example, and to verify the performance of our proposed algorithm, we apply our results to a flying capacitor multicell converter. © 2014 © 2014 Taylor & Francis

Simultaneous State and Unknown Input Set-Valued Observers for Some Classes of Nonlinear Dynamical Systems

In this paper, we propose fixed-order set-valued (in the form of l2-norm hyperballs) observers for some classes of nonlinear bounded-error dynamical systems with unknown input signals that simultaneously find bounded hyperballs of states and unknown inputs that include the true states and inputs. Necessary and sufficient conditions in the form of Linear Matrix Inequalities (LMIs) for the stability (in the sense of quadratic stability) of the proposed observers are derived for ($\mathcal{M},\gamma$)- Quadratically Constrained (($\mathcal{M},\gamma$)-QC) systems, which includes several classes of nonlinear systems: (I) Lipschitz continuous, (II) ($\mathcal{A},\gamma$)-QC* and (III) Linear Parameter-Varying (LPV) systems. This new quadratic constraint property is at least as general as the incremental quadratic constraint property for nonlinear systems and is proven in the paper to embody a broad range of nonlinearities. In addition, we design the optimal $\mathcal{H}_{\infty}$ observer among those that satisfy the quadratic stability conditions and show that the design results in Uniformly Bounded-Input Bounded-State (UBIBS) estimate radii/error dynamics and uniformly bounded sequences of the estimate radii. Furthermore, we provide closed-form upper bound sequences for the estimate radii and sufficient condition for their convergence to steady state. Finally, the effectiveness of the proposed set-valued observers is demonstrated through illustrative examples, where we compare the performance of our observers with some existing observers.Comment: Under review in Automatic

Evolving Ensemble Fuzzy Classifier

The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in this paper. pENsemble differs from existing architectures in the fact that it is built upon an evolving classifier from data streams, termed Parsimonious Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism, which estimates a localized generalization error of a base classifier. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. pENsemble adopts a dynamic ensemble structure to output a final classification decision where it features a novel drift detection scenario to grow the ensemble structure. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity.Comment: this paper has been published by IEEE Transactions on Fuzzy System