Matrix algebras in quasi-newtonian algorithms for optimal learning in multi-layer perceptrons

Abstract

In this work the authors implement in a Multi-Layer Perceptron (MLP) environment a new class of quasi-newtonian (QN) methods. The algorithms proposed in the present paper use in the iterative scheme of a generalized BFGS-method a family of matrix algebras, recently introduced for displacement decompositions and for optimal preconditioning. This novel approach allows to construct methods having an O(n log_2 n) complexity. Numerical experiences compared with the performances of the best QN-algorithms known in the literature confirm the effectiveness of these new optimization techniques