Orthogonal Least Square Algorithm Applied to the Initialization of Multi-Layer Perceptrons

An efficient procedure is proposed for initializing two-layer perceptrons and for determining the optimal number of hidden neurons. This is based on the Orthogonal Least Squares method, which is typical of RBF as well as Wavelet networks. Some experiments are discussed, in which the proposed method is coupled with standard backpropagation training and compared with random initialization. 1 Introduction A suitable and efficient initialization procedure allows to set the initial values of the weights of a network not far from the optimal values determined by training: so doing, the training procedure takes a shorter time to reach the optimal values and therefore considerable saving of computation time is achieved. In networks design, great importance must be attributed also to a correct choice of the number of hidden neurons, which helps avoiding problems of overfitting and contributes to reduce the time required for the training without significantly affecting the network perform..