An heuristic pattern correction scheme for GRNNs and its application to speech recognition

Published versio

Heuristic pattern correction scheme using adaptively trained generalized regression neural networks

In many pattern classification problems, an intelligent neural system is required which can learn the newly encountered but misclassified patterns incrementally, while keeping a good classification performance over the past patterns stored in the network. In the paper, an heuristic pattern correction scheme is proposed using adaptively trained generalized regression neural networks (GRNNs). The scheme is based upon both network growing and dual-stage shrinking mechanisms. In the network growing phase, a subset of the misclassified patterns in each incoming data set is iteratively added into the network until all the patterns in the incoming data set are classified correctly. Then, the redundancy in the growing phase is removed in the dual-stage network shrinking. Both long- and short-term memory models are considered in the network shrinking, which are motivated from biological study of the brain. The learning capability of the proposed scheme is investigated through extensive simulation studie

Numerical Non-Linear Modelling Algorithm Using Radial Kernels on Local Mesh Support

Estimation problems are frequent in several fields such as engineering, economics, and physics, etc. Linear and non-linear regression are powerful techniques based on optimizing an error defined over a dataset. Although they have a strong theoretical background, the need of supposing an analytical expression sometimes makes them impractical. Consequently, a group of other approaches and methodologies are available, from neural networks to random forest, etc. This work presents a new methodology to increase the number of available numerical techniques and corresponds to a natural evolution of the previous algorithms for regression based on finite elements developed by the authors improving the computational behavior and allowing the study of problems with a greater number of points. It possesses an interesting characteristic: Its direct and clear geometrical meaning. The modelling problem is presented from the point of view of the statistical analysis of the data noise considered as a random field. The goodness of fit of the generated models has been tested and compared with some other methodologies validating the results with some experimental campaigns obtained from bibliography in the engineering field, showing good approximation. In addition, a small variation on the data estimation algorithm allows studying overfitting in a model, that it is a problematic fact when numerical methods are used to model experimental values.This research has been partially funded by the Spanish Ministry of Science, Innovation and Universities, grant number RTI2018-101148-B-I00

Simple estimate of the width in Gaussian kernel with adaptive scaling technique

This paper presents a simple method to estimate the width of Gaussian kernel based on an adaptive scaling technique. The Gaussian kernel is widely employed in radial basis function (RBF) network, support vector machine (SVM), least squares support vector machine (LS-SVM), Kriging models, and so on. It is widely known that the width of the Gaussian kernel in these machine learning techniques plays an important role. Determination of the optimal width is a time-consuming task. Therefore, it is preferable to determine the width with a simple manner. In this paper, we first examine a simple estimate of the width proposed by Nakayama et al. Through the examination, four sufficient conditions for the simple estimate of the width are described. Then, a new simple estimate for the width is proposed. In order to obtain the proposed estimate of the width, all dimensions are equally scaled. A simple technique called the adaptive scaling technique is also developed. It is expected that the proposed simple method to estimate the width is applicable to wide range of machine learning techniques employing the Gaussian kernel. Through examples, the validity of the proposed simple method to estimate the width is examined. © 2011 Elsevier B.V. All rights reserved

Designing radial basis function networks for regression

U diplomskom radu opisan je problem regresije, te pristup njegovom rješavanju korištenjem radijalnih mreža. Objašnjen je koncept i arhitektura radijalnih mreža i objašnjene su sličnosti i razlike s višeslojnim perceptronom te tijek izgradnje koji se sastoji od određivanja parametara koristeći nadzirane i nenadzirane metode učenja. Navedeni su različiti načini određivanja svakog od parametara. Razvijeno je programsko rješenje koje omogućuje izvršavanje regresijske analize na danom skupu podataka tako da se učita skup podataka, konfigurira radijalna mreža, te koje pruža na uvid rezultate. Izvršena je eksperimentalna analiza korištenjem razvijenog programskog rješenja za regresijsku analizu pet stvarnih skupa podataka čiji su rezultati prikazani i komentirani. Dobiveni rezultati pokazuju važnost korištenja algoritma za grupiranje i skaliranja širina radijalnih funkcija pri određivanju parametara radijalne mreže.Graduation thesis provides a description of regression analysis as well as an approach to regression analysis using radial basis function networks. The general concept and architecture of radial basis function networks are explained, and also the process of modelling such a network which is done by determining its parameters using supervised and unsupervised learning methods. Different approaches to determining the parameters are presented. A CLI application was developed to perform regression analysis, which enables loading of a dataset, network parameter configuration, and which provides results for inspection to the user. Experimental analysis was performed using the application for regression analysis of five real-world datasets, whose results were shown and commented. The results show the importance of clustering algorithm usage and radial basis function width scaling when determining network parameters

The relaxation method for learning in artificial neural networks

A new mathematical approach for deriving learning algorithms for various neural network models including the Hopfield model, Bidirectional Associative Memory, Dynamic Heteroassociative Neural Memory, and Radial Basis Function Networks is presented. The mathematical approach is based on the relaxation method for solving systems of linear inequalities. The newly developed learning algorithms are fast and they guarantee convergence to a solution in a finite number of steps. The new algorithms are highly insensitive to choice of parameters and the initial set of weights. They also exhibit high scalability on binary random patterns. Rigorous mathematical foundations for the new algorithms and their simulation studies are included

Uma abordagem para parametrização de Redes Neurais de Função de Base Radial baseada na combinação de procedimentos não supervisionados e de uma nova proposição de escalonamento de parâmetros.

Neste trabalho será apresentada uma abordagem para parametrização de redes RBF (Radial Basis Function) baseada na combinação de procedimentos não supervisionados e uma nova proposição de escalonamento de parâmetros. A metodologia consiste em combinar procedimentos referenciados na literatura com o objetivo de obter modelos de redes RBF com melhores exatidões e algoritmos computacionais mais compactos. Alguns exemplos serão utilizados para ilustrar o emprego da abordagem proposta e também servirão para realizar comparações de resultados com os principais procedimentos referenciados em textos da área. As redes neurais com funções de base radial (RBF) são modelos não lineares que podem realizar um mapeamento (interpolação) eficiente de dados de entrada e saída de diversos tipos de sistemas, resultando em boa capacidade de generalização aliada a processamentos de informações de forma compacta, possibilitando na representação eficiente de sistemas dinâmicos complexos e de séries temporais, por exemplo. Os bons resultados na capacidade de interpolação de uma RBF dependem de alguns parâmetros que devem ser adequadamente ajustados. Algumas abordagens foram desenvolvidas nesse contexto. O procedimento proposto neste trabalho mostrou-se ser uma alternativa promissora, com aplicação direta e que apresenta uma exatidão adequada para várias aplicações práticas. Exemplos como aproximações de funções, modelagem de sistemas dinâmicos não lineares, previsão de série temporal e classificação de padrões serão discutidos com a finalidade de exemplificar os procedimentos propostos, além de servir de comparações com os resultados obtidos por outras técnicas utilizadas em redes RBF

Exchange rate forecasting: an application of radial basis function neural networks

The purpose of this research is to investigate the forecasting performance of Artificial Neural Network models applied to foreign exchange rates. The study concentrates on the behavior of forecasts of exchange rates generated from the radial basis function (RBF) network models where little previous work exists;Exchange rates examined are the German mark/US dollar, Japanese yen/US dollar, and Italian lira/US dollar. One-step-ahead forecasts from univariate and multivariate RBF models are compared with those generated from ARIMA models, random walk forecasts and the forward rates. Interest rates and the money supply (M1) are used as explanatory variables in the multivariate analyses;Out-of-sample evaluation criteria include root mean squared error, correct direction , and speculative direction . Hypothesis tests are used to assess if differences in forecast accuracy from different models are significant and to assess if models can predict the direction of change with statistical significance. The tests employed are the Modified Diebold Marino test [Harvey et al. (1997)], the Pesaran-Timmerman (1992, 1994) non-parametric market timing test, and the chi2 test of independence [see Swanson and White (1997)];The main results of the study indicate that RBF models may be a useful alternative to the other models considered for forecasting exchange rates. In particular, out-of-sample forecasting results indicate that some multivariate RBF models using interest rates as economic variables do have forecasting value for some exchange rates. In the presence of interest rates, the M1 variable does not seem to possess much explanatory power for forecasting the three exchange rates