103,798 research outputs found
Phase transitions in optimal unsupervised learning
We determine the optimal performance of learning the orientation of the
symmetry axis of a set of P = alpha N points that are uniformly distributed in
all the directions but one on the N-dimensional sphere. The components along
the symmetry breaking direction, of unitary vector B, are sampled from a
mixture of two gaussians of variable separation and width. The typical optimal
performance is measured through the overlap Ropt=B.J* where J* is the optimal
guess of the symmetry breaking direction. Within this general scenario, the
learning curves Ropt(alpha) may present first order transitions if the clusters
are narrow enough. Close to these transitions, high performance states can be
obtained through the minimization of the corresponding optimal potential,
although these solutions are metastable, and therefore not learnable, within
the usual bayesian scenario.Comment: 9 pages, 8 figures, submitted to PRE, This new version of the paper
contains one new section, Bayesian versus optimal solutions, where we explain
in detail the results supporting our claim that bayesian learning may not be
optimal. Figures 4 of the first submission was difficult to understand. We
replaced it by two new figures (Figs. 4 and 5 in this new version) containing
more detail
- …