4,678 research outputs found
MMSE of probabilistic low-rank matrix estimation: Universality with respect to the output channel
This paper considers probabilistic estimation of a low-rank matrix from
non-linear element-wise measurements of its elements. We derive the
corresponding approximate message passing (AMP) algorithm and its state
evolution. Relying on non-rigorous but standard assumptions motivated by
statistical physics, we characterize the minimum mean squared error (MMSE)
achievable information theoretically and with the AMP algorithm. Unlike in
related problems of linear estimation, in the present setting the MMSE depends
on the output channel only trough a single parameter - its Fisher information.
We illustrate this striking finding by analysis of submatrix localization, and
of detection of communities hidden in a dense stochastic block model. For this
example we locate the computational and statistical boundaries that are not
equal for rank larger than four.Comment: 10 pages, Allerton Conference on Communication, Control, and
Computing 201
Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap
A simple model to study subspace clustering is the high-dimensional
-Gaussian mixture model where the cluster means are sparse vectors. Here we
provide an exact asymptotic characterization of the statistically optimal
reconstruction error in this model in the high-dimensional regime with
extensive sparsity, i.e. when the fraction of non-zero components of the
cluster means , as well as the ratio between the number of
samples and the dimension are fixed, while the dimension diverges. We identify
the information-theoretic threshold below which obtaining a positive
correlation with the true cluster means is statistically impossible.
Additionally, we investigate the performance of the approximate message passing
(AMP) algorithm analyzed via its state evolution, which is conjectured to be
optimal among polynomial algorithm for this task. We identify in particular the
existence of a statistical-to-computational gap between the algorithm that
require a signal-to-noise ratio
to perform better than random, and the information theoretic threshold at
.
Finally, we discuss the case of sub-extensive sparsity by comparing the
performance of the AMP with other sparsity-enhancing algorithms, such as
sparse-PCA and diagonal thresholding.Comment: NeurIPS camera-ready versio
Phase Transitions in Sparse PCA
We study optimal estimation for sparse principal component analysis when the
number of non-zero elements is small but on the same order as the dimension of
the data. We employ approximate message passing (AMP) algorithm and its state
evolution to analyze what is the information theoretically minimal mean-squared
error and the one achieved by AMP in the limit of large sizes. For a special
case of rank one and large enough density of non-zeros Deshpande and Montanari
[1] proved that AMP is asymptotically optimal. We show that both for low
density and for large rank the problem undergoes a series of phase transitions
suggesting existence of a region of parameters where estimation is information
theoretically possible, but AMP (and presumably every other polynomial
algorithm) fails. The analysis of the large rank limit is particularly
instructive.Comment: 6 pages, 3 figure
- âŠ