Comparison of Stochastic Global Optimization Methods: Estimating Neural Network Weights

Agricultural Economic

On Training Neural Nets through Stochastic Minimization

The revival of multilayer neural networks in the mid 80's originated from the discovery of the backpropagation technique as a feasible training procedure. In spite of its shortcomings, it is probably the most widespread technique for training feedforward nets. In recent years, several deterministic methods more efficient than back-propagation have been proposed. In this paper a stochastic minimization algorithm, Iterated Adaptive Memory Stochastic Search, is described which does not use gradient information and is found to perform better than back-propagation on the encoder and parity problems 1 . Keywords: stochastic optimization, learning algorithms, back-propagation 1. Introduction Learning from examples, the problem which neural networks were created to solve, is one of the most important research topics in the AI community. A possible way to formalize learning from examples is to hypothesize the existence of a function that captures the underlying mapping, thereby enabling gen..