Data-driven adaptive model-based predictive control with application in wastewater systems

This study is concerned with the development of a new data-driven adaptive model-based predictive controller (MBPC) with input constraints. The proposed methods employ subspace identification technique and a singular value decomposition (SVD)-based optimisation strategy to formulate the control algorithm and incorporate the input constraints. Both direct adaptive model-based predictive controller (DAMBPC) and indirect adaptive model-based predictive controller (IAMBPC) are considered. In DAMBPC, the direct identification of controller parameters is desired to reduce the design effort and computational load while the IAMBPC involves a two-stage process of model identification and controller design. The former method only requires a single QR decomposition for obtaining the controller parameters and uses a receding horizon approach to process input/output data for the identification. A suboptimal SVD-based optimisation technique is proposed to incorporate the input constraints. The proposed techniques are implemented and tested on a fourth order non-linear model of a wastewater system. Simulation results are presented to compare the direct and indirect adaptive methods and to demonstrate the performance of the proposed algorithms

Online Tensor Methods for Learning Latent Variable Models

We introduce an online tensor decomposition based approach for two latent variable modeling problems namely, (1) community detection, in which we learn the latent communities that the social actors in social networks belong to, and (2) topic modeling, in which we infer hidden topics of text articles. We consider decomposition of moment tensors using stochastic gradient descent. We conduct optimization of multilinear operations in SGD and avoid directly forming the tensors, to save computational and storage costs. We present optimized algorithm in two platforms. Our GPU-based implementation exploits the parallelism of SIMD architectures to allow for maximum speed-up by a careful optimization of storage and data transfer, whereas our CPU-based implementation uses efficient sparse matrix computations and is suitable for large sparse datasets. For the community detection problem, we demonstrate accuracy and computational efficiency on Facebook, Yelp and DBLP datasets, and for the topic modeling problem, we also demonstrate good performance on the New York Times dataset. We compare our results to the state-of-the-art algorithms such as the variational method, and report a gain of accuracy and a gain of several orders of magnitude in the execution time.Comment: JMLR 201

Feature Space Modeling for Accurate and Efficient Learning From Non-Stationary Data

A non-stationary dataset is one whose statistical properties such as the mean, variance, correlation, probability distribution, etc. change over a specific interval of time. On the contrary, a stationary dataset is one whose statistical properties remain constant over time. Apart from the volatile statistical properties, non-stationary data poses other challenges such as time and memory management due to the limitation of computational resources mostly caused by the recent advancements in data collection technologies which generate a variety of data at an alarming pace and volume. Additionally, when the collected data is complex, managing data complexity, emerging from its dimensionality and heterogeneity, can pose another challenge for effective computational learning. The problem is to enable accurate and efficient learning from non-stationary data in a continuous fashion over time while facing and managing the critical challenges of time, memory, concept change, and complexity simultaneously. Feature space modeling is one of the most effective solutions to address this problem. For non-stationary data, selecting relevant features is even more critical than stationary data due to the reduction of feature dimension which can ensure the best use a computational resource to produce higher accuracy and efficiency by data mining algorithms. In this dissertation, we investigated a variety of feature space modeling techniques to improve the overall performance of data mining algorithms. In particular, we built Relief based feature sub selection method in combination with data complexity iv analysis to improve the classification performance using ovarian cancer image data collected in a non-stationary batch mode. We also collected time series health sensor data in a streaming environment and deployed feature space transformation using Singular Value Decomposition (SVD). This led to reduced dimensionality of feature space resulting in better accuracy and efficiency produced by Density Ration Estimation Method in identifying potential change points in data over time. We have also built an unsupervised feature space modeling using matrix factorization and Lasso Regression which was successfully deployed in conjugate with Relative Density Ratio Estimation to address the botnet attacks in a non-stationary environment. Relief based feature model improved 16% accuracy of Fuzzy Forest classifier. For change detection framework, we observed 9% improvement in accuracy for PCA feature transformation. Due to the unsupervised feature selection model, for 2% and 5% malicious traffic ratio, the proposed botnet detection framework exhibited average 20% better accuracy than One Class Support Vector Machine (OSVM) and average 25% better accuracy than Autoencoder. All these results successfully demonstrate the effectives of these feature space models. The fundamental theme that repeats itself in this dissertation is about modeling efficient feature space to improve both accuracy and efficiency of selected data mining models. Every contribution in this dissertation has been subsequently and successfully employed to capitalize on those advantages to solve real-world problems. Our work bridges the concepts from multiple disciplines ineffective and surprising ways, leading to new insights, new frameworks, and ultimately to a cross-production of diverse fields like mathematics, statistics, and data mining

Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms

AbstractThe identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. An important characteristic that distinguishes fuzzy systems from other techniques in this area is their transparency and interpretability. Especially in the construction of a fuzzy system from a set of given training examples, little attention has been paid to the analysis of the trade-off between complexity and accuracy maintaining the interpretability of the final fuzzy system. In this paper a multi-objective evolutionary approach is proposed to determine a Pareto-optimum set of fuzzy systems with different compromises between their accuracy and complexity. In particular, two fundamental and competing objectives concerning fuzzy system modeling are addressed: fuzzy rule parameter optimization and the identification of system structure (i.e. the number of membership functions and fuzzy rules), taking always in mind the transparency of the obtained system. Another key aspect of the algorithm presented in this work is the use of some new expert evolutionary operators, specifically designed for the problem of fuzzy function approximation, that try to avoid the generation of worse solutions in order to accelerate the convergence of the algorithm

Application of Adaptive Νeuro-Fuzzy Inference System in Interest Rates Effects on Stock Returns

In the current study we examine the effects of interest rate changes on common stock returns of Greek banking sector. We examine the Generalized Autoregressive Heteroskedasticity (GARCH) process and an Adaptive Neuro-Fuzzy Inference System (ANFIS). The conclusions of our findings are that the changes of interest rates, based on GARCH model, are insignificant on common stock returns during the period we examine. On the other hand, with ANFIS we can get the rules and in each case we can have positive or negative effects depending on the conditions and the firing rules of inputs, which information is not possible to be retrieved with the traditional econometric modelling. Furthermore we examine the forecasting performance of both models and we conclude that ANFIS outperforms GARCH model in both in-sample and out-of-sample periods

Extracting takagi-sugeno fuzzy rules with interpretable submodels via regularization of linguistic modifiers

In this paper, a method for constructing Takagi-Sugeno (TS) fuzzy system from data is proposed with the objective of preserving TS submodel comprehensibility, in which linguistic modifiers are suggested to characterize the fuzzy sets. A good property held by the proposed linguistic modifiers is that they can broaden the cores of fuzzy sets while contracting the overlaps of adjoining membership functions (MFs) during identification of fuzzy systems from data. As a result, the TS submodels identified tend to dominate the system behaviors by automatically matching the global model (GM) in corresponding subareas, which leads to good TS model interpretability while producing distinguishable input space partitioning. However, the GM accuracy and model interpretability are two conflicting modeling objectives, improving interpretability of fuzzy models generally degrades the GM performance of fuzzy models, and vice versa. Hence, one challenging problem is how to construct a TS fuzzy model with not only good global performance but also good submodel interpretability. In order to achieve a good tradeoff between GM performance and submodel interpretability, a regularization learning algorithm is presented in which the GM objective function is combined with a local model objective function defined in terms of an extended index of fuzziness of identified MFs. Moreover, a parsimonious rule base is obtained by adopting a QR decomposition method to select the important fuzzy rules and reduce the redundant ones. Experimental studies have shown that the TS models identified by the suggested method possess good submodel interpretability and satisfactory GM performance with parsimonious rule bases. © 2006 IEEE

On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection

Type-2 fuzzy systems are increasing in popularity, and there are many examples of successful applications. While many techniques have been proposed for creating parsimonious type-1 fuzzy systems, there is a lack of such techniques for type-2 systems. The essential problem is to reduce the number of rules, while maintaining the system's approximation performance. In this paper, four novel indexes for ranking the relative contribution of type-2 fuzzy rules are proposed, which are termed R-values, c-values, ω1 -values, and ω2 -values. The R-values of type-2 fuzzy rules are obtained by applying a QR decomposition pivoting algorithm to the firing strength matrices of the trained fuzzy model. The c-values rank rules based on the effects of rule consequents, while the ω1 -values and ω2 -values consider both the rule-base structure (via firing strength matrices) and the output contribution of fuzzy rule consequents. Two procedures for utilizing these indexes in fuzzy rule selection (termed "forward selection" and "backward elimination") are described. Experiments are presented which demonstrate that by using the proposed methodology, the most influential type-2 fuzzy rules can be effectively retained in order to construct parsimonious type-2 fuzzy models

On constructing parsimonious type-2 fuzzy logic systems via influential rule selection

Type-2 fuzzy systems are increasing in popularity, and there are many examples of successful applications. While many techniques have been proposed for creating parsimonious type-1 fuzzy systems, there is a lack of such techniques for type-2 systems. The essential problem is to reduce the number of rules, while maintaining the system's approximation performance. In this paper, four novel indexes for ranking the relative contribution of type-2 fuzzy rules are proposed, which are termed values, c-values, ω1-values, and ω2 -values. The R-values of type-2 fuzzy rules are obtained by applying a QR decomposition pivoting algorithm to the firing strength matrices of the trained fuzzy model. The c-values rank rules based on the effects of rule consequents, while the ω1-values and ω2-values consider both the rule-base structure (via firing strength matrices) and the output contribution of fuzzy rule consequents. Two procedures for utilizing these indexes in fuzzy rule selection (termed "forward selection"and "backward elimination") are described. Experiments are presented which demonstrate that by using the proposed methodology, the most influential type-2 fuzzy rules can be effectively retained in order to construct parsimonious type-2 fuzzy models. © 2009 IEEE