Representation of Functional Data in Neural Networks

Functional Data Analysis (FDA) is an extension of traditional data analysis to functional data, for example spectra, temporal series, spatio-temporal images, gesture recognition data, etc. Functional data are rarely known in practice; usually a regular or irregular sampling is known. For this reason, some processing is needed in order to benefit from the smooth character of functional data in the analysis methods. This paper shows how to extend the Radial-Basis Function Networks (RBFN) and Multi-Layer Perceptron (MLP) models to functional data inputs, in particular when the latter are known through lists of input-output pairs. Various possibilities for functional processing are discussed, including the projection on smooth bases, Functional Principal Component Analysis, functional centering and reduction, and the use of differential operators. It is shown how to incorporate these functional processing into the RBFN and MLP models. The functional approach is illustrated on a benchmark of spectrometric data analysis.Comment: Also available online from: http://www.sciencedirect.com/science/journal/0925231

Travel time measure specification by functional approximation: application of radial basis function neural networks

AbstractIn this study, in the purpose of providing a dynamic procedure for reliable travel time specification, the performance of a neural functional approximation method is analysed. The numerical analyses are carried out on the succeeding sections of a freeway segment inputting data obtained from microwave radar sensor units located successively at the cross-sections of a freeway segment of approximately 4km. Measurements on traffic variables, i.e., vehicle counts, speed, and occupancy, for the reference time periods are processed. The structure of the employed radial basis function neural networks are configured considering the data of a three-lane freeway segment obtained by succeeding sensors located in side-fired position. Travel time measures approximated by the neural models are compared with the corresponding field measurements obtained by probe vehicle. Results prove neural model's performance in representing spatiotemporal variation of flow dynamics as well as travel times. Adaptability of the proposed travel time specification procedure to real-time intelligent control systems is a possible future extension

Extending the functional equivalence of radial basis functionnetworks and fuzzy inference systems

We establish the functional equivalence of a generalized class of Gaussian radial basis function (RBFs) networks and the full Takagi-Sugeno model (1983) of fuzzy inference. This generalizes an existing result which applies to the standard Gaussian RBF network and a restricted form of the Takagi-Sugeno fuzzy system. The more general framework allows the removal of some of the restrictive conditions of the previous result

Comments on 'functional equivalence between radial basis function networks and fuzzy inference systems' [and author's reply]

The above paper claims that under a set of minor restrictions radial basis function networks and fuzzy inference systems are functionally equivalent. The purpose of this letter is to show that this set of restrictions is incomplete and that, when it is completed, the said functional equivalence applies only to a small range of fuzzy inference systems. In addition, a modified set of restrictions is proposed which is applicable for a much wider range of fuzzy inference systems

Local Training for Radial Basis Function Networks: Towards Solving the Hidden Unit Problem

This work examines training methods for radial basis function networks (RBFNs). First, the theoretical and practical motivation for RBFNs is reviewed, as are two currently popular training methods. Next a new training method is developed using well known results from functional analysis. This method trains each kidden unit individually, and is thus called the local training method. The structure of the method allows analysis of individual hidden units; moreover a covariance-related quantity is defined that gives insight into how many hidden units to employ. Two examples illustrate the usefulness of the method. Lastly, an ad hoc method to further improve RBFN performance is demonstrated

A new class of wavelet networks for nonlinear system identification

A new class of wavelet networks (WNs) is proposed for nonlinear system identification. In the new networks, the model structure for a high-dimensional system is chosen to be a superimposition of a number of functions with fewer variables. By expanding each function using truncated wavelet decompositions, the multivariate nonlinear networks can be converted into linear-in-the-parameter regressions, which can be solved using least-squares type methods. An efficient model term selection approach based upon a forward orthogonal least squares (OLS) algorithm and the error reduction ratio (ERR) is applied to solve the linear-in-the-parameters problem in the present study. The main advantage of the new WN is that it exploits the attractive features of multiscale wavelet decompositions and the capability of traditional neural networks. By adopting the analysis of variance (ANOVA) expansion, WNs can now handle nonlinear identification problems in high dimensions

A unified wavelet-based modelling framework for non-linear system identification: the WANARX model structure

A new unified modelling framework based on the superposition of additive submodels, functional components, and wavelet decompositions is proposed for non-linear system identification. A non-linear model, which is often represented using a multivariate non-linear function, is initially decomposed into a number of functional components via the wellknown analysis of variance (ANOVA) expression, which can be viewed as a special form of the NARX (non-linear autoregressive with exogenous inputs) model for representing dynamic input–output systems. By expanding each functional component using wavelet decompositions including the regular lattice frame decomposition, wavelet series and multiresolution wavelet decompositions, the multivariate non-linear model can then be converted into a linear-in-theparameters problem, which can be solved using least-squares type methods. An efficient model structure determination approach based upon a forward orthogonal least squares (OLS) algorithm, which involves a stepwise orthogonalization of the regressors and a forward selection of the relevant model terms based on the error reduction ratio (ERR), is employed to solve the linear-in-the-parameters problem in the present study. The new modelling structure is referred to as a wavelet-based ANOVA decomposition of the NARX model or simply WANARX model, and can be applied to represent high-order and high dimensional non-linear systems

Theoretical Interpretations and Applications of Radial Basis Function Networks

Medical applications usually used Radial Basis Function Networks just as Artificial Neural Networks. However, RBFNs are Knowledge-Based Networks that can be interpreted in several way: Artificial Neural Networks, Regularization Networks, Support Vector Machines, Wavelet Networks, Fuzzy Controllers, Kernel Estimators, Instanced-Based Learners. A survey of their interpretations and of their corresponding learning algorithms is provided as well as a brief survey on dynamic learning algorithms. RBFNs' interpretations can suggest applications that are particularly interesting in medical domains