2,349 research outputs found
Comparing disorder and adaptability in stochasticity
In the literature, there are various notions of stochasticity which measure
how well an algorithmically random set satisfies the law of large numbers. Such
notions can be categorized by disorder and adaptability: adaptive strategies
may use information observed about the set when deciding how to act, and
disorderly strategies may act out of order. In the disorderly setting, adaptive
strategies are more powerful than non-adaptive ones. In the adaptive setting,
Merkle et al. showed that disorderly strategies are more powerful than orderly
ones. This leaves open the question of how disorderly, non-adaptive strategies
compare to orderly, adaptive strategies, as well as how both relate to orderly,
non-adaptive strategies. In this paper, we show that orderly, adaptive
strategies and disorderly, non-adaptive strategies are both strictly more
powerful than orderly, non-adaptive strategies. Using the techniques developed
to prove this, we also make progress towards the former open question by
introducing a notion of orderly, ``weakly adaptable'' strategies which we prove
is incomparable with disorderly, non-adaptive strategies
TAP variational principle for the constrained overlap multiple spherical Sherrington-Kirkpatrick model
Spin glass models involving multiple replicas with constrained overlaps have
been studied in [FPV92; PT07; Pan18a]. For the spherical versions of these
models [Ko19; Ko20] showed that the limiting free energy is given by a Parisi
type minimization. In this work we show that for Sherrington-Kirkpatrick (i.e.
2-spin) interactions, it can also be expressed in terms of a
Thouless-Andersson-Palmer (TAP) variational principle. This is only the second
spin glass model where a mathematically rigorous TAP computation of the free
energy at all temperatures and external fields has been achieved. The
variational formula we derive here also confirms that the model is replica
symmetric, a fact which is natural but not obviously deducible from its Parisi
formula.Comment: 47 page
Optimal Algorithms for the Inhomogeneous Spiked Wigner Model
In this paper, we study a spiked Wigner problem with an inhomogeneous noise
profile. Our aim in this problem is to recover the signal passed through an
inhomogeneous low-rank matrix channel. While the information-theoretic
performances are well-known, we focus on the algorithmic problem. We derive an
approximate message-passing algorithm (AMP) for the inhomogeneous problem and
show that its rigorous state evolution coincides with the information-theoretic
optimal Bayes fixed-point equations. We identify in particular the existence of
a statistical-to-computational gap where known algorithms require a
signal-to-noise ratio bigger than the information-theoretic threshold to
perform better than random. Finally, from the adapted AMP iteration we deduce a
simple and efficient spectral method that can be used to recover the transition
for matrices with general variance profiles. This spectral method matches the
conjectured optimal computational phase transition.Comment: 17 pages, 5 figure
A multiscale cavity method for sublinear-rank symmetric matrix factorization
We consider a statistical model for symmetric matrix factorization with
additive Gaussian noise in the high-dimensional regime where the rank of
the signal matrix to infer scales with its size as .
Allowing for a -dependent rank offers new challenges and requires new
methods. Working in the Bayesian-optimal setting, we show that whenever the
signal has i.i.d. entries the limiting mutual information between signal and
data is given by a variational formula involving a rank-one replica symmetric
potential. In other words, from the information-theoretic perspective, the case
of a (slowly) growing rank is the same as when (namely, the standard
spiked Wigner model). The proof is primarily based on a novel multiscale cavity
method allowing for growing rank along with some information-theoretic
identities on worst noise for the Gaussian vector channel. We believe that the
cavity method developed here will play a role in the analysis of a broader
class of inference and spin models where the degrees of freedom are large
arrays instead of vectors
Fundamental limits of Non-Linear Low-Rank Matrix Estimation
We consider the task of estimating a low-rank matrix from non-linear and
noisy observations. We prove a strong universality result showing that
Bayes-optimal performances are characterized by an equivalent Gaussian model
with an effective prior, whose parameters are entirely determined by an
expansion of the non-linear function. In particular, we show that to
reconstruct the signal accurately, one requires a signal-to-noise ratio growing
as , where is the first non-zero Fisher
information coefficient of the function. We provide asymptotic characterization
for the minimal achievable mean squared error (MMSE) and an approximate
message-passing algorithm that reaches the MMSE under conditions analogous to
the linear version of the problem. We also provide asymptotic errors achieved
by methods such as principal component analysis combined with Bayesian
denoising, and compare them with Bayes-optimal MMSE.Comment: 42 pages, 2 figure
Spectral Phase Transitions in Non-Linear Wigner Spiked Models
We study the asymptotic behavior of the spectrum of a random matrix where a
non-linearity is applied entry-wise to a Wigner matrix perturbed by a rank-one
spike with independent and identically distributed entries. In this setting, we
show that when the signal-to-noise ratio scale as , where is the first non-zero generalized information
coefficient of the function, the non-linear spike model effectively behaves as
an equivalent spiked Wigner matrix, where the former spike before the
non-linearity is now raised to a power . This allows us to study the
phase transition of the leading eigenvalues, generalizing part of the work of
Baik, Ben Arous and Pech\'e to these non-linear models.Comment: 27 page
- …