Comparative performance of some popular ANN algorithms on benchmark and function approximation problems

We report an inter-comparison of some popular algorithms within the artificial neural network domain (viz., Local search algorithms, global search algorithms, higher order algorithms and the hybrid algorithms) by applying them to the standard benchmarking problems like the IRIS data, XOR/N-Bit parity and Two Spiral. Apart from giving a brief description of these algorithms, the results obtained for the above benchmark problems are presented in the paper. The results suggest that while Levenberg-Marquardt algorithm yields the lowest RMS error for the N-bit Parity and the Two Spiral problems, Higher Order Neurons algorithm gives the best results for the IRIS data problem. The best results for the XOR problem are obtained with the Neuro Fuzzy algorithm. The above algorithms were also applied for solving several regression problems such as cos(x) and a few special functions like the Gamma function, the complimentary Error function and the upper tail cumulative $\chi^2$-distribution function. The results of these regression problems indicate that, among all the ANN algorithms used in the present study, Levenberg-Marquardt algorithm yields the best results. Keeping in view the highly non-linear behaviour and the wide dynamic range of these functions, it is suggested that these functions can be also considered as standard benchmark problems for function approximation using artificial neural networks.Comment: 18 pages 5 figures. Accepted in Pramana- Journal of Physic

Impact of noise on a dynamical system: prediction and uncertainties from a swarm-optimized neural network

In this study, an artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. The hybrid ANN+PSO algorithm was applied on Mackey--Glass chaotic time series in the short-term $x(t+6)$. The performance prediction was evaluated and compared with another studies available in the literature. Also, we presented properties of the dynamical system via the study of chaotic behaviour obtained from the predicted time series. Next, the hybrid ANN+PSO algorithm was complemented with a Gaussian stochastic procedure (called {\it stochastic} hybrid ANN+PSO) in order to obtain a new estimator of the predictions, which also allowed us to compute uncertainties of predictions for noisy Mackey--Glass chaotic time series. Thus, we studied the impact of noise for several cases with a white noise level ($\sigma_{N}$) from 0.01 to 0.1.Comment: 11 pages, 8 figure

Gaussian process based model predictive control : a thesis submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Engineering, School of Engineering and Advanced Technology, Massey University, New Zealand

The performance of using Model Predictive Control (MPC) techniques is highly dependent on a model that is able to accurately represent the dynamical system. The datadriven modelling techniques are usually used as an alternative approach to obtain such a model when first principle techniques are not applicable. However, it is not easy to assess the quality of learnt models when using the traditional data-driven models, such as Artificial Neural Network (ANN) and Fuzzy Model (FM). This issue is addressed in this thesis by using probabilistic Gaussian Process (GP) models. One key issue of using the GP models is accurately learning the hyperparameters. The Conjugate Gradient (CG) algorithms are conventionally used in the problem of maximizing the Log-Likelihood (LL) function to obtain these hyperparameters. In this thesis, we proposed a hybrid Particle Swarm Optimization (PSO) algorithm to cope with the problem of learning hyperparameters. In addition, we also explored using the Mean Squared Error (MSE) of outputs as the fitness function in the optimization problem. This will provide us a quality indication of intermediate solutions. The GP based MPC approaches for unknown systems have been studied in the past decade. However, most of them are not generally formulated. In addition, the optimization solutions in existing GP based MPC algorithms are not clearly given or are computationally demanding. In this thesis, we first study the use of GP based MPC approaches in the unconstrained problems. Compared to the existing works, the proposed approach is generally formulated and the corresponding optimization problem is eff- ciently solved by using the analytical gradients of GP models w.r.t. outputs and control inputs. The GPMPC1 and GPMPC2 algorithms are subsequently proposed to handle the general constrained problems. In addition, through using the proposed basic and extended GP based local dynamical models, the constrained MPC problem is effectively solved in the GPMPC1 and GPMPC2 algorithms. The proposed algorithms are verified in the trajectory tracking problem of the quadrotor. The issue of closed-loop stability in the proposed GPMPC algorithm is addressed by means of the terminal cost and constraint technique in this thesis. The stability guaranteed GPMPC algorithm is subsequently proposed for the constrained problem. By using the extended GP based local dynamical model, the corresponding MPC problem is effectively solved

Brain image clustering by wavelet energy and CBSSO optimization algorithm

Previously, the diagnosis of brain abnormality was significantly important in the saving of social and hospital resources. Wavelet energy is known as an effective feature detection which has great efficiency in different utilities. This paper suggests a new method based on wavelet energy to automatically classify magnetic resonance imaging (MRI) brain images into two groups (normal and abnormal), utilizing support vector machine (SVM) classification based on chaotic binary shark smell optimization (CBSSO) to optimize the SVM weights. The results of the suggested CBSSO-based KSVM are compared favorably to several other methods in terms of better sensitivity and authenticity. The proposed CAD system can additionally be utilized to categorize the images with various pathological conditions, types, and illness modes

Advanced theoretical and experimental studies in automatic control and information systems

A series of research projects is briefly summarized which includes investigations in the following areas: (1) mathematical programming problems for large system and infinite-dimensional spaces, (2) bounded-input bounded-output stability, (3) non-parametric approximations, and (4) differential games. A list of reports and papers which were published over the ten year period of research is included

Fuzzy heterogeneous neural networks for signal forecasting

Fuzzy heterogeneous neural networks are recently introduced models based on neurons accepting heterogeneous inputs (i.e. mixtures of numerical and non-numerical information possibly with missing data) with either crisp or imprecise character, which can be coupled with classical neurons. This paper compares the effectiveness of this kind of networks with time-delay and recurrent architectures that use classical neuron models and training algorithms in a signal forecasting problem, in the context of finding models of the central nervous system controllers.Peer ReviewedPostprint (author's final draft