Supervised online learning with an ensemble of students randomized by the
choice of initial conditions is analyzed. For the case of the perceptron
learning rule, asymptotically the same improvement in the generalization error
of the ensemble compared to the performance of a single student is found as in
Gibbs learning. For more optimized learning rules, however, using an ensemble
yields no improvement. This is explained by showing that for any learning rule
f a transform f~​ exists, such that a single student using
f~​ has the same generalization behaviour as an ensemble of
f-students.Comment: 8 pages, 1 figure. Submitted to J.Phys.