Replica-symmetry breaking in noise-optimal neural networks

Abstract

Recent studies of optimization in neural networks trained with noisy data have shown that replica-symmetric solutions are unstable in the low-noise region of parameter space. We calculate the 1-step replica-symmetry broken solution in this region, which joins the replica-symmetric solution continuously at the dr Almeida-Thouless line. These solutions yield satisfactory agreement with simulations for the aligning field distribution, better than those given by the replica-symmetric ansatz