Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A Survey

Major assumptions in computational intelligence and machine learning consist of the availability of a historical dataset for model development, and that the resulting model will, to some extent, handle similar instances during its online operation. However, in many real world applications, these assumptions may not hold as the amount of previously available data may be insufficient to represent the underlying system, and the environment and the system may change over time. As the amount of data increases, it is no longer feasible to process data efficiently using iterative algorithms, which typically require multiple passes over the same portions of data. Evolving modeling from data streams has emerged as a framework to address these issues properly by self-adaptation, single-pass learning steps and evolution as well as contraction of model components on demand and on the fly. This survey focuses on evolving fuzzy rule-based models and neuro-fuzzy networks for clustering, classification and regression and system identification in online, real-time environments where learning and model development should be performed incrementally. (C) 2019 Published by Elsevier Inc.Igor Škrjanc, Jose Antonio Iglesias and Araceli Sanchis would like to thank to the Chair of Excellence of Universidad Carlos III de Madrid, and the Bank of Santander Program for their support. Igor Škrjanc is grateful to Slovenian Research Agency with the research program P2-0219, Modeling, simulation and control. Daniel Leite acknowledges the Minas Gerais Foundation for Research and Development (FAPEMIG), process APQ-03384-18. Igor Škrjanc and Edwin Lughofer acknowledges the support by the ”LCM — K2 Center for Symbiotic Mechatronics” within the framework of the Austrian COMET-K2 program. Fernando Gomide is grateful to the Brazilian National Council for Scientific and Technological Development (CNPq) for grant 305906/2014-3

Dynaamisten mallien puoliautomaattinen parametrisointi käyttäen laitosdataa

The aim of this thesis was to develop a new methodology for estimating parameters of NAPCON ProsDS dynamic simulator models to better represent data containing several operating points. Before this thesis, no known methodology had existed for combining operating point identification with parameter estimation of NAPCON ProsDS simulator models. The methodology was designed by assessing and selecting suitable methods for operating space partitioning, parameter estimation and parameter scheduling. Previously implemented clustering algorithms were utilized for the operating space partition. Parameter estimation was implemented as a new tool in the NAPCON ProsDS dynamic simulator and iterative parameter estimation methods were applied. Finally, lookup tables were applied for tuning the model parameters according to the state. The methodology was tested by tuning a heat exchanger model to several operating points based on plant process data. The results indicated that the developed methodology was able to tune the simulator model to better represent several operating states. However, more testing with different models is required to verify general applicability of the methodology.Tämän diplomityön tarkoitus oli kehittää uusi parametrien estimointimenetelmä NAPCON ProsDS -simulaattorin dynaamisille malleille, jotta ne vastaisivat paremmin dataa useista prosessitiloista. Ennen tätä diplomityötä NAPCON ProsDS -simulaattorin malleille ei ollut olemassa olevaa viritysmenetelmää, joka yhdistäisi operointitilojen tunnistuksen parametrien estimointiin. Menetelmän kehitystä varten tutkittiin ja valittiin sopivat menetelmät operointiavaruuden jakamiselle, parametrien estimoinnille ja parametrien virittämiseen prosessitilan mukaisesti. Aikaisemmin ohjelmoituja klusterointialgoritmeja hyödynnettiin operointiavaruuden jakamisessa. Parametrien estimointi toteutettiin uutena työkaluna NAPCON ProsDS -simulaattoriin ja estimoinnissa käytettiin iteratiivisia optimointimenetelmiä. Lopulta hakutaulukoita sovellettiin mallin parametrien hienosäätöön prosessitilojen mukaisesti. Menetelmää testattiin virittämällä lämmönvaihtimen malli kahteen eri prosessitilaan käyttäen laitokselta kerättyä prosessidataa. Tulokset osoittavat että kehitetty menetelmä pystyi virittämään simulaattorin mallin vastaamaan paremmin dataa useista prosessitiloista. Kuitenkin tarvitaan lisää testausta erityyppisten mallien kanssa, jotta voidaan varmistaa menetelmän yleinen soveltuvuus

Evolving Clustering Algorithms And Their Application For Condition Monitoring, Diagnostics, & Prognostics

Applications of Condition-Based Maintenance (CBM) technology requires effective yet generic data driven methods capable of carrying out diagnostics and prognostics tasks without detailed domain knowledge and human intervention. Improved system availability, operational safety, and enhanced logistics and supply chain performance could be achieved, with the widespread deployment of CBM, at a lower cost level. This dissertation focuses on the development of a Mutual Information based Recursive Gustafson-Kessel-Like (MIRGKL) clustering algorithm which operates recursively to identify underlying model structure and parameters from stream type data. Inspired by the Evolving Gustafson-Kessel-like Clustering (eGKL) algorithm, we applied the notion of mutual information to the well-known Mahalanobis distance as the governing similarity measure throughout. This is also a special case of the Kullback-Leibler (KL) Divergence where between-cluster shape information (governed by the determinant and trace of the covariance matrix) is omitted and is only applicable in the case of normally distributed data. In the cluster assignment and consolidation process, we proposed the use of the Chi-square statistic with the provision of having different probability thresholds. Due to the symmetry and boundedness property brought in by the mutual information formulation, we have shown with real-world data that the algorithm’s performance becomes less sensitive to the same range of probability thresholds which makes system tuning a simpler task in practice. As a result, improvement demonstrated by the proposed algorithm has implications in improving generic data driven methods for diagnostics, prognostics, generic function approximations and knowledge extractions for stream type of data. The work in this dissertation demonstrates MIRGKL’s effectiveness in clustering and knowledge representation and shows promising results in diagnostics and prognostics applications

Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation

We propose an automatic methodology framework for short- and long-term prediction of time series by means of fuzzy inference systems. In this methodology, fuzzy techniques and statistical techniques for nonparametric residual variance estimation are combined in order to build autoregressive predictive models implemented as fuzzy inference systems. Nonparametric residual variance estimation plays a key role in driving the identification and learning procedures. Concrete criteria and procedures within the proposed methodology framework are applied to a number of time series prediction problems. The learn from examples method introduced by Wang and Mendel (W&M) is used for identification. The Levenberg–Marquardt (L–M) optimization method is then applied for tuning. The W&M method produces compact and potentially accurate inference systems when applied after a proper variable selection stage. The L–M method yields the best compromise between accuracy and interpretability of results, among a set of alternatives. Delta test based residual variance estimations are used in order to select the best subset of inputs to the fuzzy inference systems as well as the number of linguistic labels for the inputs. Experiments on a diverse set of time series prediction benchmarks are compared against least-squares support vector machines (LS-SVM), optimally pruned extreme learning machine (OP-ELM), and k-NN based autoregressors. The advantages of the proposed methodology are shown in terms of linguistic interpretability, generalization capability and computational cost. Furthermore, fuzzy models are shown to be consistently more accurate for prediction in the case of time series coming from real-world applications.Ministerio de Ciencia e Innovación TEC2008-04920Junta de Andalucía P08-TIC-03674, IAC07-I-0205:33080, IAC08-II-3347:5626

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Speckle Detection in Ultrasonic Images Using Unsupervised Clustering Techniques

Research for the improvement of the quality of clinical ultrasound images has been a topic of interest for researchers and physicians. One of the challenges is the presence of speckle artifacts. This dissertation reviews the speckle phenomena in such images, and develops algorithms to better identify this artifact in sonographic images. Speckle artifact is categorized into two groups: partially developed speckles and fully developed speckles (FDS). This concept has been used, along with the classification techniques, to segment the ultrasound images into patches and classify the patches in the image as FDS or non-FDS. The proposed algorithms and the results of the experiments have been validated using simulation, phantom and real data that were created for the purposes of this study or taken from other research groups. Current speckle detection methods do not optimize statistical features and they are not based on machine learning techniques. For the first time this work introduces a novel method for searching and extracting the best features for optimizing speckle detection rate using statistical machine learning and ensemble classification. Potential applications include strain imaging by tracking speckle displacement, elastography, speckle tracking and suppression applications, and needle-tracking applications.Ph.D., Biomedical Engineering -- Drexel University, 201

Modelling of an electro-hydraulic actutor using extended adaptive distance gap statistic approach

The existence of high degree of non-linearity in Electro-Hydraulic Actuator (EHA) system has imposed a challenging task in developing its model so that effective control algorithm can be proposed. In general, there are two modelling approaches available for EHA system, which are the dynamic equation modelling method and the system identification modelling method. Both approaches have disadvantages, where the dynamic equation modelling is hard to apply and some parameters are difficult to obtain, while the system identification method is less accurate when the system’s nature is complicated with wide variety of parameters, nonlinearity and uncertainties. This thesis presents a new modelling procedure of an EHA system by using fuzzy approach. Two sets of input variables are obtained, where the first set of variables are selected based on mathematical modelling of the EHA system. The reduction of input dimension is done by the Principal Component Analysis (PCA) method for the second set of input variables. A new gap statistic with a new within-cluster dispersion calculation is proposed by introducing an adaptive distance norm in distance calculation. The new gap statistic applies Gustafson Kessel (GK) clustering algorithm to obtain the optimal number of cluster of each input. GK clustering algorithm also provides the location and characteristic of every cluster detected. The information of input variables, number of clusters, cluster’s locations and characteristics, and fuzzy rules are used to generate initial Fuzzy Inference System (FIS) with Takagi-Sugeno type. The initial FIS is trained using Adaptive Network Fuzzy Inference System (ANFIS) hybrid training algorithm with an identification data set. The ANFIS EHA model and ANFIS PCA model obtained using proposed modelling procedure, have shown the ability to accurately estimate EHA system’s performance at 99.58% and 99.11% best fitting accuracy compared to conventional linear Autoregressive with External Input (ARX) model at 94.97%. The models validation result on different data sets also suggests high accuracy in ANFIS EHA and ANFIS PCA model compared to ARX model

Joint inversion of magnetotelluric and seismic data for crustal characterization

Tese de doutoramento, Ciências Geofísicas e da Geoinformação (Geofisíca), Universidade de Lisboa, Faculdade de Ciências, 2014The research that has been performed in this thesis contributes to integration two different geophysical methods (magnetotellurics and seismic refraction). Pattern recognition has not been used for the cooperative inversion of these two methods before. This study demonstrates potential of soft clustering methods for integration of two data sets. Gathering information from both inversions, a multiparameter model was made. This multi-parameter model contains regions, each of which is characterized by a consistent relationship between the model parameters. Both synthetic and field data results were used to discuss an assumption that mutual exchange of information can lead to better results of inversions in both cases. The method developed in this thesis was successfully applied at experimental data collected in the Iberian Pyrite Belt.A investigação desenvolvida nesta tese é uma contribuição para a integração de dois métodos geofísicos (magneto-telúrico e sísmica de refracção). A técnica de reconhecimento de padrões não tinha sido ainda utilizada na inversão cooperativa de dados destes dois métodos. Este estudo mostra as potencialidades dos métodos de análise de “clusters” na integração dos dois conjuntos de dados.Um modelo “multi-parâmetros” foi desenvolvido para extrair informação de ambos os métodos geofísicos.Este modelo contém regiões caracterizadas por relações consistentes entre os parâmetros. Resultados obtidos com dados sintéticos e de campo foram usados para mostrar que o uso da informação contida nos dois conjuntos de dados conduz a melhores resultados. O método desenvolvido nesta tese foi aplicado, com resultados satisfatórios, a um conjunto de dados reais obtidos na Faixa Piritosa.Fundação para a Ciência e Tecnologia (FCT, SFRH/BD/66455/2009