Comparative performance of some popular ANN algorithms on benchmark and function approximation problems

We report an inter-comparison of some popular algorithms within the artificial neural network domain (viz., Local search algorithms, global search algorithms, higher order algorithms and the hybrid algorithms) by applying them to the standard benchmarking problems like the IRIS data, XOR/N-Bit parity and Two Spiral. Apart from giving a brief description of these algorithms, the results obtained for the above benchmark problems are presented in the paper. The results suggest that while Levenberg-Marquardt algorithm yields the lowest RMS error for the N-bit Parity and the Two Spiral problems, Higher Order Neurons algorithm gives the best results for the IRIS data problem. The best results for the XOR problem are obtained with the Neuro Fuzzy algorithm. The above algorithms were also applied for solving several regression problems such as cos(x) and a few special functions like the Gamma function, the complimentary Error function and the upper tail cumulative $\chi^2$-distribution function. The results of these regression problems indicate that, among all the ANN algorithms used in the present study, Levenberg-Marquardt algorithm yields the best results. Keeping in view the highly non-linear behaviour and the wide dynamic range of these functions, it is suggested that these functions can be also considered as standard benchmark problems for function approximation using artificial neural networks.Comment: 18 pages 5 figures. Accepted in Pramana- Journal of Physic

Generation of Fuzzy Rules Based on Complex-valued Neuro-Fuzzy Learning Algorithm

In order to generate or tune fuzzy rules, Neuro-Fuzzy learning algorithms with Gaussian type membership functions based on gradient-descent method are well known. In this paper, we propose a new learning approach, the Complex-valued Neuro-Fuzzy learning algorithm. This method is an extension of the conventional method to complex domain by using a complex-valued neural network that maps complex values to real values. Input, antecedent membership functions and consequent singleton are complex, and output is real. Two-dimensional input can be better represented by complex numbers than by real values. We compared it with the conventional method by several function identification problems, and revealed that the proposed method outperformed the counterpart, and that it is a useful tool for learning a fuzzy system model.The 3rd International Workshop on Advanced Computational Intelligence and Intelligent Informatics (IWACIII 2013) will be held in Shanghai, China from October 18 to 21 in 2013

Feature selection using genetic algorithms and probabilistic neural networks

Selection of input variables is a key stage in building predictive models, and an important form of data mining. As exhaustive evaluation of potential input sets using full non-linear models is impractical, it is necessary to use simple fast-evaluating models and heuristic selection strategies. This paper discusses a fast, efficient, and powerful nonlinear input selection procedure using a combination of Probabilistic Neural Networks and repeated bitwise gradient descent. The algorithm is compared with forward elimination, backward elimination and genetic algorithms using a selection of real-world data sets. The algorithm has comparative performance and greatly reduced execution time with respect to these alternative approaches. It is demonstrated empirically that reliable results cannot be gained using any of these approaches without the use of resampling

From monitoring data to remaining useful life : an evolving approach including uncertainty.

International audienceAlthough prognostic activity is nowadays recognized as a key feature in maintenance strategies, real prognostic systems are scarce in industry. That can be explained from different aspects, one of them being the lack of knowledge on the monitored system that impedes the development of classical dependability analysis (based on statistical data for example). Within this frame, the general purpose of the work is to propose a prognostic system that starts from monitoring data and goes through provisional reliability and remaining useful life by characterizing the uncertainty following from the degradation process. More precisely, the paper emphasizes on the development of an evolving neuro-fuzzy predictor that, not only "gives" an approximation of the degradation of an equipment but also associates to it a confidence measure

Computational intelligence techniques for maximum energy efficiency of cogeneration processes based on internal combustion engines

153 p.El objeto de la tesis consiste en desarrollar estrategias de modelado y optimización del rendimiento energético de plantas de cogeneración basadas en motores de combustión interna (MCI), mediante el uso de las últimas tecnologías de inteligencia computacional. Con esta finalidad se cuenta con datos reales de una planta de cogeneración de energía, propiedad de la compañía EnergyWorks, situada en la localidad de Monzón (provincia de Huesca). La tesis se realiza en el marco de trabajo conjunto del Grupo de Diseño en Electrónica Digital (GDED) de la Universidad del País Vasco UPV/EHU y la empresa Optimitive S.L., empresa dedicada al software avanzado para la mejora en tiempo real de procesos industriale

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Novel Levenberg–Marquardt based learning algorithm for unmanned aerial vehicles

In this paper, Levenberg–Marquardt inspired sliding mode control theory based adaptation laws are proposed to train an intelligent fuzzy neural network controller for a quadrotor aircraft. The proposed controller is used to control and stabilize a quadrotor unmanned aerial vehicle in the presence of periodic wind gust. A proportional-derivative controller is firstly introduced based on which fuzzy neural network is able to learn the quadrotor's control model on-line. The proposed design allows handling uncertainties and lack of modelling at a computationally inexpensive cost. The parameter update rules of the learning algorithms are derived based on a Levenberg–Marquardt inspired approach, and the proof of the stability of two proposed control laws are verified by using the Lyapunov stability theory. In order to evaluate the performance of the proposed controllers extensive simulations and real-time experiments are conducted. The 3D trajectory tracking problem for a quadrotor is considered in the presence of time-varying wind conditions