Research on an online self-organizing radial basis function neural network

A new growing and pruning algorithm is proposed for radial basis function (RBF) neural network structure design in this paper, which is named as self-organizing RBF (SORBF). The structure of the RBF neural network is introduced in this paper first, and then the growing and pruning algorithm is used to design the structure of the RBF neural network automatically. The growing and pruning approach is based on the radius of the receptive field of the RBF nodes. Meanwhile, the parameters adjusting algorithms are proposed for the whole RBF neural network. The performance of the proposed method is evaluated through functions approximation and dynamic system identification. Then, the method is used to capture the biochemical oxygen demand (BOD) concentration in a wastewater treatment system. Experimental results show that the proposed method is efficient for network structure optimization, and it achieves better performance than some of the existing algorithms

A performance evaluation of pruning effects on hybrid neural network

In this paper, we explore the pruning effects on a hybrid mode sequential learning algorithmnamely FuzzyARTMAP-prunable Radial Basis Function (FAM-PRBF) that utilizes FuzzyARTMAP to learn a training dataset and Radial Basis Function Network (RBFN) to performregression and classification. The pruning algorithm is used to optimize the hidden layer ofthe RBFN. The experimental results show that FAM-PRBF has successfully reduced thecomplexity and computation time of the neural network.Keywords: pruning; radial basis function network; fuzzy ARTMAP

A new data-driven neural fuzzy system with collaborative fuzzy clustering mechanism

© 2015 Elsevier B.V. In this paper, a novel fuzzy rule transfer mechanism for self-constructing neural fuzzy inference networks is being proposed. The features of the proposed method, termed data-driven neural fuzzy system with collaborative fuzzy clustering mechanism (DDNFS-CFCM) are; (1) Fuzzy rules are generated facilely by fuzzy c-means (FCM) and then adapted by the preprocessed collaborative fuzzy clustering (PCFC) technique, and (2) Structure and parameter learning are performed simultaneously without selecting the initial parameters. The DDNFS-CFCM can be applied to deal with big data problems by the virtue of the PCFC technique, which is capable of dealing with immense datasets while preserving the privacy and security of datasets. Initially, the entire dataset is organized into two individual datasets for the PCFC procedure, where each of the dataset is clustered separately. The knowledge of prototype variables (cluster centers) and the matrix of just one halve of the dataset through collaborative technique are deployed. The DDNFS-CFCM is able to achieve consistency in the presence of collective knowledge of the PCFC and boost the system modeling process by parameter learning ability of the self-constructing neural fuzzy inference networks (SONFIN). The proposed method outperforms other existing methods for time series prediction problems

Adaptive structure radial basis function network model for processes with operating region migration

An adaptive structure radial basis function (RBF) network model is proposed in this paper to model nonlinear processes with operating region migration. The recursive orthogonal least squares algorithm is adopted to select new centers on-line, as well as to train the network weights. Based on the R matrix in the orthogonal decomposition, an initial center bank is formed and updated in each sample period. A new learning strategy is proposed to gain information from the new data for network structure adaptation. A center grouping algorithm is also developed to divide the centers into active and non-active groups, so that a structure with a smaller size is maintained in the final network model. The proposed RBF model is evaluated and compared to the two fixed-structure RBF networks by modeling a nonlinear time-varying numerical example. The results demonstrate that the proposed adaptive structure model is capable of adapting its structure to fit the operating region of the process effectively with a more compact structure and it significantly outperforms the two fixed structure RBF models

RMSE-ELM: Recursive Model based Selective Ensemble of Extreme Learning Machines for Robustness Improvement

Extreme learning machine (ELM) as an emerging branch of shallow networks has shown its excellent generalization and fast learning speed. However, for blended data, the robustness of ELM is weak because its weights and biases of hidden nodes are set randomly. Moreover, the noisy data exert a negative effect. To solve this problem, a new framework called RMSE-ELM is proposed in this paper. It is a two-layer recursive model. In the first layer, the framework trains lots of ELMs in different groups concurrently, then employs selective ensemble to pick out an optimal set of ELMs in each group, which can be merged into a large group of ELMs called candidate pool. In the second layer, selective ensemble is recursively used on candidate pool to acquire the final ensemble. In the experiments, we apply UCI blended datasets to confirm the robustness of our new approach in two key aspects (mean square error and standard deviation). The space complexity of our method is increased to some degree, but the results have shown that RMSE-ELM significantly improves robustness with slightly computational time compared with representative methods (ELM, OP-ELM, GASEN-ELM, GASEN-BP and E-GASEN). It becomes a potential framework to solve robustness issue of ELM for high-dimensional blended data in the future.Comment: Accepted for publication in Mathematical Problems in Engineering, 09/22/201

Gradient-based training and pruning of radial basis function networks with an application in materials physics

Many applications, especially in physics and other sciences, call for easily interpretable and robust machine learning techniques. We propose a fully gradient-based technique for training radial basis function networks with an efficient and scalable open-source implementation. We derive novel closed form optimization criteria for pruning the models for continuous as well as binary data which arise in a challenging real-world material physics problem. The pruned models are optimized to provide compact and interpretable versions of larger models based on informed assumptions about the data distribution. Visualizations of the pruned models provide insight into the atomic configurations that determine atom-level migration processes in solid matter; these results may inform future research on designing more suitable descriptors for use with machine learning algorithms. (c) 2020 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).Peer reviewe

RBF-sítě s dynamickou architekturou

V tejto diplomovej práci som zrekapituloval viacero metód vhodných pre klastrovanie vstupných dát. Predstavil som dva dobré známe klastrovacie algoritmy, a to konkrétne K-means algoritmus a Fuzzy C-means (FCM) algoritmus. Uviedol som niekoľko metód vhodných pre odhad optimálneho počtu klastrov. Ďalej som predstavil Kohonenové mapy a dva modely Kohonenových máp s adaptívnou topológiou, konkrétne Kohonenové mapy s rastúcou mriežkou a model rastúcich neurónových plynov. Ako posledný som predstavil pomerne nový model radiálne bázických neurónových sieti. Pre tento typ neurónových sieti som uviedol viacero učiacich algoritmov. V závere práce som aplikoval jednotlivé klastrovacie metódy na reálne dáta popisujúce vzájomný obchod štátov sveta.In this master thesis I recapitulated several methods for clustering input data. Two well known clustering algorithms, concretely K-means algorithm and Fuzzy C-means (FCM) algorithm, were described in the submitted work. I presented several methods, which could help estimate the optimal number of clusters. Further, I described Kohonen maps and two models of Kohonen's maps with dynamically changing structure, namely Kohonen map with growing grid and the model of growing neural gas. At last I described quite new model of radial basis function neural networks. I presented several learning algorithms for this model of neural networks. In the end of this work I made some clustering experiments with real data. This data describes the international trade among states of the whole world.Department of Theoretical Computer Science and Mathematical LogicKatedra teoretické informatiky a matematické logikyFaculty of Mathematics and PhysicsMatematicko-fyzikální fakult

Gradient-Based Training and Pruning of Radial Basis Function Networks with an Application in Materials Physics

Many applications, especially in physics and other sciences, call for easily interpretable and robust machine learning techniques. We propose a fully gradient-based technique for training radial basis function networks with an efficient and scalable open-source implementation. We derive novel closed-form optimization criteria for pruning the models for continuous as well as binary data which arise in a challenging real-world material physics problem. The pruned models are optimized to provide compact and interpretable versions of larger models based on informed assumptions about the data distribution. Visualizations of the pruned models provide insight into the atomic configurations that determine atom-level migration processes in solid matter; these results may inform future research on designing more suitable descriptors for use with machine learning algorithms