91 research outputs found

    A randomized neural network for data streams

    Get PDF
    © 2017 IEEE. Randomized neural network (RNN) is a highly feasible solution in the era of big data because it offers a simple and fast working principle in processing dynamic and evolving data streams. This paper proposes a novel RNN, namely recurrent type-2 random vector functional link network (RT2McRVFLN), which provides a highly scalable solution for data streams in a strictly online and integrated framework. It is built upon the psychologically inspired concept of metacognitive learning, which covers three basic components of human learning: what-to-learn, how-to-learn, and when-to-learn. The what-to-learn selects important samples on the fly with the use of online active learning scenario, which renders our algorithm an online semi-supervised algorithm. The how-to-learn process combines an open structure of evolving concept and a randomized learning algorithm of random vector functional link network (RVFLN). The efficacy of the RT2McRVFLN has been numerically validated through two real-world case studies and comparisons with its counterparts, which arrive at a conclusive finding that our algorithm delivers a tradeoff between accuracy and simplicity

    Evolving Ensemble Fuzzy Classifier

    Full text link
    The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in this paper. pENsemble differs from existing architectures in the fact that it is built upon an evolving classifier from data streams, termed Parsimonious Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism, which estimates a localized generalization error of a base classifier. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. pENsemble adopts a dynamic ensemble structure to output a final classification decision where it features a novel drift detection scenario to grow the ensemble structure. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity.Comment: this paper has been published by IEEE Transactions on Fuzzy System

    A Novel Progressive Multi-label Classifier for Classincremental Data

    Full text link
    In this paper, a progressive learning algorithm for multi-label classification to learn new labels while retaining the knowledge of previous labels is designed. New output neurons corresponding to new labels are added and the neural network connections and parameters are automatically restructured as if the label has been introduced from the beginning. This work is the first of the kind in multi-label classifier for class-incremental learning. It is useful for real-world applications such as robotics where streaming data are available and the number of labels is often unknown. Based on the Extreme Learning Machine framework, a novel universal classifier with plug and play capabilities for progressive multi-label classification is developed. Experimental results on various benchmark synthetic and real datasets validate the efficiency and effectiveness of our proposed algorithm.Comment: 5 pages, 3 figures, 4 table

    Scaffolding type-2 classifier for incremental learning under concept drifts

    Full text link
    © 2016 Elsevier B.V. The proposal of a meta-cognitive learning machine that embodies the three pillars of human learning: what-to-learn, how-to-learn, and when-to-learn, has enriched the landscape of evolving systems. The majority of meta-cognitive learning machines in the literature have not, however, characterized a plug-and-play working principle, and thus require supplementary learning modules to be pre-or post-processed. In addition, they still rely on the type-1 neuron, which has problems of uncertainty. This paper proposes the Scaffolding Type-2 Classifier (ST2Class). ST2Class is a novel meta-cognitive scaffolding classifier that operates completely in local and incremental learning modes. It is built upon a multivariable interval type-2 Fuzzy Neural Network (FNN) which is driven by multivariate Gaussian function in the hidden layer and the non-linear wavelet polynomial in the output layer. The what-to-learn module is created by virtue of a novel active learning scenario termed the uncertainty measure; the how-to-learn module is based on the renowned Schema and Scaffolding theories; and the when-to-learn module uses a standard sample reserved strategy. The viability of ST2Class is numerically benchmarked against state-of-the-art classifiers in 12 data streams, and is statistically validated by thorough statistical tests, in which it achieves high accuracy while retaining low complexity

    A Novel Meta-Cognitive Extreme Learning Machine to Learning from Data Streams

    Full text link
    © 2015 IEEE. Extreme Learning Machine (ELM) is an answer to an increasing demand for a low-cost learning algorithm to handle big data applications. Nevertheless, existing ELMs leave four uncharted problems: complexity, uncertainty, concept drifts, curse of dimensionality. To correct these issues, a novel incremental meta-cognitive ELM, namely Evolving Type-2 Extreme Learning Machine (eT2ELM), is proposed. Et2Elm is built upon the three pillars of meta-cognitive learning, namely what-To-learn, how-To-learn, when-To-learn, where the notion of ELM is implemented in the how-To-learn component. On the other hand, eT2ELM is driven by a generalized interval type-2 Fuzzy Neural Network (FNN) as the cognitive constituent, where the interval type-2 multivariate Gaussian function is used in the hidden layer, whereas the nonlinear Chebyshev function is embedded in the output layer. The efficacy of eT2ELM is proven with four data streams possessing various concept drifts, comparisons with prominent classifiers, and statistical tests, where eT2ELM demonstrates the most encouraging learning performances in terms of accuracy and complexity

    Development of advanced autonomous learning algorithms for nonlinear system identification and control

    Full text link
    Identification of nonlinear dynamical systems, data stream analysis, etc. is usually handled by autonomous learning algorithms like evolving fuzzy and evolving neuro-fuzzy systems (ENFSs). They are characterized by the single-pass learning mode and open structure-property. Such features enable their effective handling of fast and rapidly changing natures of data streams. The underlying bottleneck of ENFSs lies in its design principle, which involves a high number of free parameters (rule premise and rule consequent) to be adapted in the training process. This figure can even double in the case of the type-2 fuzzy system. From this literature gap, a novel ENFS, namely Parsimonious Learning Machine (PALM) is proposed in this thesis. To reduce the number of network parameters significantly, PALM features utilization of a new type of fuzzy rule based on the concept of hyperplane clustering, where it has no rule premise parameters. PALM is proposed in both type-1 and type-2 fuzzy systems where all of them characterize a fully dynamic rule-based system. Thus, it is capable of automatically generating, merging, and tuning the hyperplane-based fuzzy rule in a single-pass manner. Moreover, an extension of PALM, namely recurrent PALM (rPALM), is proposed and adopts the concept of teacher-forcing mechanism in the deep learning literature. The efficacy of both PALM and rPALM have been evaluated through numerical study with data streams and to identify nonlinear unmanned aerial vehicle system. The proposed models showcase significant improvements in terms of computational complexity and the number of required parameters against several renowned ENFSs while attaining comparable and often better predictive accuracy. The ENFSs have also been utilized to develop three autonomous intelligent controllers (AICons) in this thesis. They are namely Generic (G) controller, Parsimonious controller (PAC), and Reduced Parsimonious Controller (RedPAC). All these controllers start operating from scratch with an empty set of fuzzy rules, and no offline training is required. To cope with the dynamic behavior of the plant, these controllers can add, merge or prune the rules on demand. Among three AICons, the G-controller is built by utilizing an advanced incremental learning machine, namely Generic Evolving Neuro-Fuzzy Inference System. The integration of generalized adaptive resonance theory provides a compact structure of the G-controller. Consequently, the faster evolution of structure is witnessed, which lowers its computational cost. Another AICon namely, PAC is rooted with PALM's architecture. Since PALM has a dependency on user-defined thresholds to adapt the structure, these thresholds are replaced with the concept of bias- variance trade-off in PAC. In RedPAC, the network parameters have further reduced in contrast with PALM-based PAC, where the number of consequent parameters has reduced to one parameter per rule. These AICons work with very minor expert domain knowledge and developed by incorporating the sliding mode control technique. In G-controller and RedPAC, the control law and adaptation laws for the consequent parameters are derived from the SMC algorithm to establish a stable closed-loop system, where the stability of these controllers are guaranteed by using the Lyapunov function and the uniform asymptotic convergence of tracking error to zero is witnessed through the implication of an auxiliary robustifying control term. While using PAC, the boundedness and convergence of the closed-loop control system's tracking error and the controller's consequent parameters are confirmed by utilizing the LaSalle-Yoshizawa theorem. Their efficacy is evaluated by observing various trajectory tracking performance of unmanned aerial vehicles. The accuracy of these controllers is comparable or better than the benchmark controllers where the proposed controllers incur significantly fewer parameters to attain similar or better tracking performance

    Incremental Market Behavior Classification in Presence of Recurring Concepts

    Get PDF
    In recent years, the problem of concept drift has gained importance in the financial domain. The succession of manias, panics and crashes have stressed the non-stationary nature and the likelihood of drastic structural or concept changes in the markets. Traditional systems are unable or slow to adapt to these changes. Ensemble-based systems are widely known for their good results predicting both cyclic and non-stationary data such as stock prices. In this work, we propose RCARF (Recurring Concepts Adaptive Random Forests), an ensemble tree-based online classifier that handles recurring concepts explicitly. The algorithm extends the capabilities of a version of Random Forest for evolving data streams, adding on top a mechanism to store and handle a shared collection of inactive trees, called concept history, which holds memories of the way market operators reacted in similar circumstances. This works in conjunction with a decision strategy that reacts to drift by replacing active trees with the best available alternative: either a previously stored tree from the concept history or a newly trained background tree. Both mechanisms are designed to provide fast reaction times and are thus applicable to high-frequency data. The experimental validation of the algorithm is based on the prediction of price movement directions one second ahead in the SPDR (Standard & Poor's Depositary Receipts) S&P 500 Exchange-Traded Fund. RCARF is benchmarked against other popular methods from the incremental online machine learning literature and is able to achieve competitive results.This research was funded by the Spanish Ministry of Economy and Competitiveness under grant number ENE2014-56126-C2-2-R

    Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A Survey

    Get PDF
    Major assumptions in computational intelligence and machine learning consist of the availability of a historical dataset for model development, and that the resulting model will, to some extent, handle similar instances during its online operation. However, in many real world applications, these assumptions may not hold as the amount of previously available data may be insufficient to represent the underlying system, and the environment and the system may change over time. As the amount of data increases, it is no longer feasible to process data efficiently using iterative algorithms, which typically require multiple passes over the same portions of data. Evolving modeling from data streams has emerged as a framework to address these issues properly by self-adaptation, single-pass learning steps and evolution as well as contraction of model components on demand and on the fly. This survey focuses on evolving fuzzy rule-based models and neuro-fuzzy networks for clustering, classification and regression and system identification in online, real-time environments where learning and model development should be performed incrementally. (C) 2019 Published by Elsevier Inc.Igor Škrjanc, Jose Antonio Iglesias and Araceli Sanchis would like to thank to the Chair of Excellence of Universidad Carlos III de Madrid, and the Bank of Santander Program for their support. Igor Škrjanc is grateful to Slovenian Research Agency with the research program P2-0219, Modeling, simulation and control. Daniel Leite acknowledges the Minas Gerais Foundation for Research and Development (FAPEMIG), process APQ-03384-18. Igor Škrjanc and Edwin Lughofer acknowledges the support by the ”LCM — K2 Center for Symbiotic Mechatronics” within the framework of the Austrian COMET-K2 program. Fernando Gomide is grateful to the Brazilian National Council for Scientific and Technological Development (CNPq) for grant 305906/2014-3
    • …
    corecore