An interactively recurrent functional neural fuzzy network with fuzzy differential evolution and its applications

In this paper, an interactively recurrent functional neural fuzzy network (IRFNFN) with fuzzy differential evolution (FDE) learning method was proposed for solving the control and the prediction problems. The traditional differential evolution (DE) method easily gets trapped in a local optimum during the learning process, but the proposed fuzzy differential evolution algorithm can overcome this shortcoming. Through the information sharing of nodes in the interactive layer, the proposed IRFNFN can effectively reduce the number of required rule nodes and improve the overall performance of the network. Finally, the IRFNFN model and associated FDE learning algorithm were applied to the control system of the water bath temperature and the forecast of the sunspot number. The experimental results demonstrate the effectiveness of the proposed method

A Locally Recurrent Fuzzy Neural Network With Support Vector Regression for Dynamic-System Modeling

This paper proposes a new recurrent model, known as the locally recurrent fuzzy neural network with support vector regression (LRFNN-SVR), that handles problems with temporal properties. Structurally, an LRFNN-SVR is a five-layered recurrent network. The recurrent structure in an LRFNN-SVR comes from locally feeding the firing strength of each fuzzy rule back to itself. The consequent layer in an LRFNN-SVR is a Takagi-Sugeno-Kang (T-S-K)-type consequent, which is a linear function of current states, regardless of system input and output delays. For the structure learning, a one-pass clustering algorithm clusters the input-training data and determines the number of network nodes in hidden layers. For the parameter learning, an iterative linear SVR algorithm is proposed to tune free parameters in the rule consequent part and feedback loops. The motivation for using SVR for parameter learning is to improve the LRFNN-SVR generalization ability. This paper demonstrates LRFNN-SVR capabilities by conducting simulations in dynamic system prediction and identification problems with noiseless and noisy data. In addition, this paper compares simulation results from the LRFNN-SVR with other recurrent fuzzy models

Modelos polinomiais narx obtidos através de metaheurísticas com codificação binária

Orientador: Prof. Dr. Gideon Villar LeandroDissertação (mestrado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Elétrica. Defesa : Curitiba, 24/06/2020Inclui referências: p. 126-137Resumo: Dentro do contexto da identificação de sistemas, a etapa de seleção de estrutura representa uma tarefa complexa por se tratar de um problema de otimização combinatória do tipo binária. Para solucionar este problema, diversas técnicas vêm sendo aplicadas, dentre elas a seleção por meio de metaheurísticas. Entretanto, devido à grande diversidade de metaheurísticas existentes na literatura, a escolha daquela mais adequada para cumprir esta tarefa pode ser algo desafiador para o projetista. Neste trabalho, é realizada uma análise do comportamento de metaheurísticas aplicadas ao problema de seleção de estrutura, além de ser apresentado um novo método de codificação binária, chamado de Modulação em Ângulo Modificada (MAM), que tende a melhorar o desempenho das metaheurísticas neste tipo de problema. Normalmente as metaheurísticas atuam na seleção de estrutura originalmente manipulando soluções binárias ou através de associação com alguma técnica de codificação binária. Foram avaliados os desempenhos das metaheurísticas Algoritmo Genético, Evolução Diferencial e Algoritmo do Morcego aplicadas ao problema de seleção de estrutura de modelos não lineares autorregressivos com entrada externa (NARX, do inglês Nonlinear AutoRegressive with eXogenous input). O Algoritmo Genético já consiste em uma metaheurística projetada para manipular soluções binárias. A Evolução Diferencial e o Algoritmo do Morcego, por sua vez, tiveram suas versões binárias implementadas através das codificações Função de Transferência (TF), Prioridade de Maior Valor (GVP) e Modulação em Ângulo (AM). Além disso, a forma de associação entre as metaheurísticas e a codificação AM foi modificada, dando origem à codificação MAM. Dois estudos de caso foram conduzidos utilizando dados de um conversor buck e de um aquecedor elétrico. Os resultados das simulações mostram que as versões binárias da Evolução Diferencial obtidas com as codificações TF, GVP e MAM foram as que apresentaram melhores desempenho, superando o Algoritmo Genético e o Algoritmo do Morcego. Além disso, levando em conta a convergência das soluções e a capacidade de localizar bons modelos, em todos os cenários analisados o desempenho da Evolução Diferencial codificada com MAM melhorou substancialmente em relação à sua versão original codificada com AM. Os melhores modelos encontrados neste trabalho apresentaram bom desempenho ao serem aplicados métodos de validação (simulação livre e análise de resíduos) e ao serem comparados com modelos da literatura. Palavras-chave: Identificação de sistemas. Seleção de estrutura. Modelo NARX. Metaheurísticas. Codificação binária.Abstract: During a system identification procedure, structure selection represents a complex task because it is a binary combinatorial optimization problem. To solve this problem, several techniques have been applied, the metaheuristics is one of them. However, there is a great diversity of metaheuristics, thus choosing the most suitable one to perform the task is difficult for the designer. In this study, we performed a performance analysis of metaheuristics applied to a structure selection problem. In addition, we present a new binarization technique, called Modified Angle Modulation (MAM), which tends to improve the performance of metaheuristics. Usually, metaheuristics perform the structure selection taking binary solutions directly or through the association with a binarization technique. We evaluated the performances of three metaheuristic techniques, being the Genetic Algorithm, Differential Evolution and Bat Algorithm, all working at a structure selection problem for nonlinear autoregressive models with exogenous input (NARX). The Genetic Algorithm is originally a binary metaheuristics. Binary versions of the Differential Evolution and the Bat Algorithm were developed through the Transfer Function (TF), Great Value Priority (GVP) and Angle Modulation (AM) binarizations. In addition, the form of association between metaheuristics and AM binarization has been modified, originating the MAM binarization. We conducted two case studies using data from a buck converter and an electric heater. Binary versions of the Differential Evolution developed through TF, GVP and MAM binarizations performed better than the Genetic Algorithm and Bat Algorithm. Furthermore, considering the convergence of solutions and the ability to locate good models, the performance of the binary version of the Differential Evolution developed with MAM substantially improved in relation to its original version developed with AM. Finally, the best estimated models performed well not only during validation tests (free run simulation and statistical validation), but also when compared with other models available in the literature. Keywords: System identification. Structure selection. NARX models. Metaheuristics. Binarization

Fuzzy Classification and Regression Model Design Using Fuzzy Clustering and Support Vector Machine

本論文的目的是採用支持向量機(Support Vector Machine)設計模糊分類(fuzzy classification)、前饋式(feedforward)及遞迴式回歸模型(recurrent regression models)。支持向量機(SVM)對模糊模型的訓練可具有降低雜訊的影響(noise effects)及達到良好的推廣能力(generalization ability)。這些設計的模糊模型可應用在不同的分類和回歸問題，其包含通道等化器(channel equalization)、 函數近似(function approximation)、系統鑑別(system identification)及 序列估測(sequence prediction). 文中設計四種新的模糊模型，第一是模糊分類模糊模型即是模糊C平均-支持向量機(Fuzzy C-means based Support Vector Machine)，此模型的輸出是輸入資料對各群歸屬值的權重和，利用線性核心支持向量機學習權重參數，使此模糊模型具有良好的分類推廣能力，模擬結果顯示模糊C平均-支持向量機在通道等化的問題上呈現出不錯的雜訊抑制效果。第二和第三均是前饋式模糊回歸模型。第二是TS模糊系統-支持向量回歸(Takagi-Sugeno Fuzzy System based Support Vector Regression)，採用一次通行的模糊分群演算法進行訓練資料的分群，一個新的TS-核心是依據TS-模式的模糊規則而來，其架構是群(cluster)的輸出和輸入變數線性組合的乘積，所以此模型的輸出是此TS-核心的線性權重和，使用線性支持向量迴歸學習權重參數。第三是利用自我分裂產生規則數和疊代支持向量回歸建構TS型式模糊系統(Takagi-Sugeno (TS)-type Fuzzy System Constructed by self-splitting Rule Generation and iterative Linear Support Vector Regression)，此模型可自動產生規則數是引入自我分裂的技術到K-平均分群演算法中。每條規則的前件部(Antecedent)及後件部(Consequent)被表示成由輸入資料轉換後向量的線性組合係數，可使用線性支持向量回歸作參數學習。第四是一個遞迴式回歸模型，即區域遞迴式模糊類神經網路-支持向量回歸(Locally Recurrent Fuzzy Neural Network with Support Vector Regression)。此模型的目的是處理具有時間特性的問題。遞迴式結構是區域性地將各規則的激發量迴傳給各規則本身，採用一次通行的模糊分群演算法進行訓練資料的分群並決定網路隱藏層的節點(node)數量。使用疊代式線性支持向量回歸對回授路徑及後件部作參數學習。經由不同的模擬範例與其他的分類、回歸模型作比較，證明本論文所提的所有模型有雜訊抑制能力及良好的推廣能力。This dissertation presents the design of fuzzy classification, feedforward, and recurrent regression models using support vector machine (SVM). The use of SVM for fuzzy model training helps reduce noise effects and achieve high generalization ability. The designed fuzzy models are applied to different classification and regression problems, including channel equalization, function approximation, system identification, and sequence prediction. Four novel fuzzy models are proposed in this dissertation. The first one is a fuzzy classification model and is called Fuzzy C-means based Support Vector Machine (FCM-SVM). In FCM-SVM, input training data is clustered by fuzzy c-means. The output of FCM-SVM is a weighted sum of the degrees where each input data belongs to the clusters. To achieve high generalization ability, FCM-SVM weights are learned through linear SVM. Simulation results on channel equalization problems show that the FCM-SVM performance is good in reducing noise influence. The second and third are feedforward fuzzy regression models. The second one is the Takagi-Sugeno (TS) Fuzzy System-based Support Vector Regression (TSFS-SVR). In TSFS-SVR, a one-pass clustering algorithm clusters the input training data. A new TS-kernel, which corresponds to a TS-type fuzzy rule, is then constructed by the product of a cluster output and a linear combination of input variables. The TSFS-SVR output is a linear weighted sum of the TS kernels. TSFS-SVR weights are learned through linear SVR. The third one is the TS-type Fuzzy System constructed by self-splitting Rule Generation and iterative Linear Support Vector Regression (FS-RGLSVR). The rules in the FS-RGLSSVR are automatically generated by introducing the self-splitting technique to the K-means clustering algorithm. Each of the consequent and antecedent part parameters is expressed as a linear combination coefficient in a transformed input space so that the linear SVR is applicable. The fourth one is a recurrent fuzzy regression model and is called Locally Recurrent Fuzzy Neural Network with Support Vector Regression (LRFNN-SVR). The LRFNN-SVR is proposed for handling problems with temporal properties. The recurrent structure in a LRFNN-SVR comes from locally feeding the firing strength of each fuzzy rule back to itself. A one-pass clustering algorithm clusters the input training data and determines the number of network nodes in hidden layers. An iterative linear support vector regression (SVR) algorithm is proposed to tune free parameters in the rule consequent part and feedback loops. Comparisons with other classification and regression models in different simulation examples demonstrate the noise robustness and generalization abilities of the proposed fuzzy models.Chinese Abstract…………………………………………i English Abstract………………………………………ii Acknowledgments ………………………………………iv Contents…………………………………………………v List of Figures………………………………………vii List of Tables…………………………………………x Chapter1 Introduction………………………1 1.1 Literature Review………………………………………………1 1.1.1. SVM and Fuzzy Classification Models…………………1 1.1.2. SVR and Fuzzy regression Models…………………2 1.1.3. Recurrent Fuzzy Regression Models………………5 1.2 Motivation and Objectives……………………………………7 1.3 Contribution of the Dissertation……………………………7 1.4 Organization of the Dissertation……………………………9 Chapter 2 Fuzzy C-Means Based Support Vector Machine for Channel Equalization………………………………………………11 2.1 Data transmission system……………………………………11 2.2 Basic SVM concepts……………………………………………14 2.3 FCM-SVM Structure and Learning……………………………16 2.3.1. FCM-SVM Structure……………………………………16 2.3.2. SVM Learning…………………………………………18 2.4 Simulation Results……………………………………………21 2.5 Conclusion………………………………………………………31 Chapter 3 TS-Fuzzy System Based Support Vector Regression 32 3.1 Basic SVR concepts……………………………………………32 3.2 TSFS-SVR Structure and Learning…………………………35 3.2.1. TSFS-SVR Model………………………………………35 3.2.2. TSFS-SVR Learning…………………………………38 3.3 Simulation Results…………………………………………41 3.4 Discussion……………………………………………………61 3.5 Conclusion……………………………………………………64 Chapter 4 A TS-type Fuzzy System Constructed by Self-Splitting Rule Generation And Iterative Linear Support Vector Regression………………………………………………66 4.1 FS-RGLSVR STRUCTURE………………………………………66 4.2 FS-RGLSVR STRUCTURE LEARNING…………………………69 4.3 FS-RGLSVR PARAMETER LEARNING………………………71 4.4 Simulation Results………………………………………77 4.5 Conclusion…………………………………………………89 Chapter 5 A Locally Recurrent Fuzzy Neural Network with Support Vector Regression for Dynamic System Modeling…91 5.1 LRFNN-SVR STRUCTURE…………………………………………91 5.2 LRFNN-SVR STRUCTURE LEARNING………………………94 5.3 LRFNN-SVR PARAMETER LEARNING……………………………97 5.4 Simulation Results…………………………………………101 5.5 Discussion……………………………………………………115 5.6 Conclusion……………………………………………………120 Chapter 6 Conclusions and Future Work……………………122 6.1 Conclusions…………………………………………………122 6.2 Future Work…………………………………………………124 Bibliography………………………………………………………125 Publication list………………………………………………13