Empirical learning aided by weak domain knowledge in the form of feature importance

Standard hybrid learners that use domain knowledge require stronger knowledge that is hard and expensive to acquire. However, weaker domain knowledge can benefit from prior knowledge while being cost effective. Weak knowledge in the form of feature relative importance (FRI) is presented and explained. Feature relative importance is a real valued approximation of a feature’s importance provided by experts. Advantage of using this knowledge is demonstrated by IANN, a modified multilayer neural network algorithm. IANN is a very simple modification of standard neural network algorithm but attains significant performance gains. Experimental results in the field of molecular biology show higher performance over other empirical learning algorithms including standard backpropagation and support vector machines. IANN performance is even comparable to a theory refinement system KBANN that uses stronger domain knowledge. This shows Feature relative importance can improve performance of existing empirical learning algorithms significantly with minimal effort

Forecasting foreign exchange rates with adaptive neural networks using radial basis functions and particle swarm optimization

The motivation for this paper is to introduce a hybrid Neural Network architecture of Particle Swarm Optimization and Adaptive Radial Basis Function (ARBF-PSO), a time varying leverage trading strategy based on Glosten, Jagannathan and Runkle (GJR) volatility forecasts and a Neural Network fitness function for financial forecasting purposes. This is done by benchmarking the ARBF-PSO results with those of three different Neural Networks architectures, a Nearest Neighbors algorithm (k-NN), an autoregressive moving average model (ARMA), a moving average convergence/divergence model (MACD) plus a naïve strategy. More specifically, the trading and statistical performance of all models is investigated in a forecast simulation of the EUR/USD, EUR/GBP and EUR/JPY ECB exchange rate fixing time series over the period January 1999 to March 2011 using the last two years for out-of-sample testing

A Regression-based Training Algorithm for Multilayer Neural Networks

Artificial neural networks (ANNs) are powerful tools for machine learning with applications in many areas including speech recognition, image classification, medical diagnosis, and spam filtering. It has been shown that ANNs can approximate any function to any degree of accuracy given enough neurons and training time. However, there is no guarantee on the number of neurons required or the time it will take to train them. These are the main disadvantages of using ANNs. This thesis develops an algorithm which uses regression-based techniques to decrease the number of training epochs. A modification of the Delta Rule, combined with techniques established for regression training of single-layer networks, has resulted in much faster training than standard gradient descent in many cases. The algorithm showed statistically significant improvements over standard backpropagation in the number of iterations, the total training time, the resulting error, and the accuracy of the resulting classifier in most cases. The algorithm was tested on several datasets of varying complexity and the results are presented