Storage capacity of ultrametric committee machines

The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K

The Approach to Ergodicity in Monte Carlo Simulations

The approach to the ergodic limit in Monte Carlo simulations is studied using both analytic and numerical methods. With the help of a stochastic model, a metric is defined that enables the examination of a simulation in both the ergodic and non-ergodic regimes. In the non-ergodic regime, the model implies how the simulation is expected to approach ergodic behavior analytically, and the analytically inferred decay law of the metric allows the monitoring of the onset of ergodic behavior. The metric is related to previously defined measures developed for molecular dynamics simulations, and the metric enables the comparison of the relative efficiencies of different Monte Carlo schemes. Applications to Lennard-Jones 13-particle clusters are shown to match the model for Metropolis, J-walking and parallel tempering based approaches. The relative efficiencies of these three Monte Carlo approaches are compared, and the decay law is shown to be useful in determining needed high temperature parameters in parallel tempering and J-walking studies of atomic clusters.Comment: 17 Pages, 7 Figure