Considerations of the Gain Spectrum

Abstract

We propose the gain spectrum as an efficient means to understand the learning dynamics of MLP gradient based learning and to roughly estimate an upper bound of the necessary network complexity. The feasibility of the approach will be shown by a number of experiments. 1 Introduction The investigation of the learning process of artificial neural networks is an area that still receives a great deal of attention in the research community. Due to the nonlinearity of the governing equations and finite size effects there is still no such thing as a fully automated efficient learning algorithm, if there can be any at all. Recent theoretical results provide guide lines for the construction of learning algorithms and permit some insight into the nature of the adaptation mechanism, but even apparently simple on-line algorithms are far from being sufficiently well understood. Our goal is to contribute to the explanation of the learning process. The approach we pursue differs from some of the prev..