Analysis of Natural Gradient Descent for Multilayer Neural Networks

Natural gradient descent is a principled method for adapting the parameters of a statistical model on-line using an underlying Riemannian parameter space to redefine the direction of steepest descent. The algorithm is examined via methods of statistical physics which accurately characterize both transient and asymptotic behavior. A solution of the learning dynamics is obtained for the case of multilayer neural network training in the limit of large input dimension. We find that natural gradient learning leads to optimal asymptotic performance and outperforms gradient descent in the transient, significantly shortening or even removing plateaus in the transient generalization performance which typically hamper gradient descent training.Comment: 14 pages including figures. To appear in Physical Review

How to optimize nonlinear force-free coronal magnetic field extrapolations from SDO/HMI vector magnetograms?

The SDO/HMI instruments provide photospheric vector magnetograms with a high spatial and temporal resolution. Our intention is to model the coronal magnetic field above active regions with the help of a nonlinear force-free extrapolation code. Our code is based on an optimization principle and has been tested extensively with semi-analytic and numeric equilibria and been applied before to vector magnetograms from Hinode and ground based observations. Recently we implemented a new version which takes measurement errors in photospheric vector magnetograms into account. Photospheric field measurements are often due to measurement errors and finite nonmagnetic forces inconsistent as a boundary for a force-free field in the corona. In order to deal with these uncertainties, we developed two improvements: 1.) Preprocessing of the surface measurements in order to make them compatible with a force-free field 2.) The new code keeps a balance between the force-free constraint and deviation from the photospheric field measurements. Both methods contain free parameters, which have to be optimized for use with data from SDO/HMI. Within this work we describe the corresponding analysis method and evaluate the force-free equilibria by means of how well force-freeness and solenoidal conditions are fulfilled, the angle between magnetic field and electric current and by comparing projections of magnetic field lines with coronal images from SDO/AIA. We also compute the available free magnetic energy and discuss the potential influence of control parameters.Comment: 17 Pages, 6 Figures, Sol. Phys., accepte

A Comparative Study of Sparse Associative Memories

We study various models of associative memories with sparse information, i.e. a pattern to be stored is a random string of $0$s and $1$s with about $\log N$ $1$s, only. We compare different synaptic weights, architectures and retrieval mechanisms to shed light on the influence of the various parameters on the storage capacity.Comment: 28 pages, 2 figure

Testing non-linear force-free coronal magnetic field extrapolations with the Titov-Demoulin equilibrium

CONTEXT: As the coronal magnetic field can usually not be measured directly, it has to be extrapolated from photospheric measurements into the corona. AIMS: We test the quality of a non-linear force-free coronal magnetic field extrapolation code with the help of a known analytical solution. METHODS: The non-linear force-free equations are numerically solved with the help of an optimization principle. The method minimizes an integral over the force-free and solenoidal condition. As boundary condition we use either the magnetic field components on all six sides of the computational box in Case I or only on the bottom boundary in Case II. We check the quality of the reconstruction by computing how well force-freeness and divergence-freeness are fulfilled and by comparing the numerical solution with the analytical solution. The comparison is done with magnetic field line plots and several quantitative measures, like the vector correlation, Cauchy Schwarz, normalized vector error, mean vector error and magnetic energy. RESULTS: For Case I the reconstructed magnetic field shows good agreement with the original magnetic field topology, whereas in Case II there are considerable deviations from the exact solution. This is corroborated by the quantitative measures, which are significantly better for Case I. CONCLUSIONS: Despite the strong nonlinearity of the considered force-free equilibrium, the optimization method of extrapolation is able to reconstruct it; however, the quality of reconstruction depends significantly on the consistency of the input data, which is given only if the known solution is provided also at the lateral and top boundaries, and on the presence or absence of flux concentrations near the boundaries of the magnetogram.Comment: 6 pages, 2 figures, Research Not