Comparing disorder and adaptability in stochasticity

In the literature, there are various notions of stochasticity which measure how well an algorithmically random set satisfies the law of large numbers. Such notions can be categorized by disorder and adaptability: adaptive strategies may use information observed about the set when deciding how to act, and disorderly strategies may act out of order. In the disorderly setting, adaptive strategies are more powerful than non-adaptive ones. In the adaptive setting, Merkle et al. showed that disorderly strategies are more powerful than orderly ones. This leaves open the question of how disorderly, non-adaptive strategies compare to orderly, adaptive strategies, as well as how both relate to orderly, non-adaptive strategies. In this paper, we show that orderly, adaptive strategies and disorderly, non-adaptive strategies are both strictly more powerful than orderly, non-adaptive strategies. Using the techniques developed to prove this, we also make progress towards the former open question by introducing a notion of orderly, ``weakly adaptable'' strategies which we prove is incomparable with disorderly, non-adaptive strategies

TAP variational principle for the constrained overlap multiple spherical Sherrington-Kirkpatrick model

Spin glass models involving multiple replicas with constrained overlaps have been studied in [FPV92; PT07; Pan18a]. For the spherical versions of these models [Ko19; Ko20] showed that the limiting free energy is given by a Parisi type minimization. In this work we show that for Sherrington-Kirkpatrick (i.e. 2-spin) interactions, it can also be expressed in terms of a Thouless-Andersson-Palmer (TAP) variational principle. This is only the second spin glass model where a mathematically rigorous TAP computation of the free energy at all temperatures and external fields has been achieved. The variational formula we derive here also confirms that the model is replica symmetric, a fact which is natural but not obviously deducible from its Parisi formula.Comment: 47 page

Optimal Algorithms for the Inhomogeneous Spiked Wigner Model

In this paper, we study a spiked Wigner problem with an inhomogeneous noise profile. Our aim in this problem is to recover the signal passed through an inhomogeneous low-rank matrix channel. While the information-theoretic performances are well-known, we focus on the algorithmic problem. We derive an approximate message-passing algorithm (AMP) for the inhomogeneous problem and show that its rigorous state evolution coincides with the information-theoretic optimal Bayes fixed-point equations. We identify in particular the existence of a statistical-to-computational gap where known algorithms require a signal-to-noise ratio bigger than the information-theoretic threshold to perform better than random. Finally, from the adapted AMP iteration we deduce a simple and efficient spectral method that can be used to recover the transition for matrices with general variance profiles. This spectral method matches the conjectured optimal computational phase transition.Comment: 17 pages, 5 figure

A multiscale cavity method for sublinear-rank symmetric matrix factorization

We consider a statistical model for symmetric matrix factorization with additive Gaussian noise in the high-dimensional regime where the rank $M$ of the signal matrix to infer scales with its size $N$ as $M = o(N^{1/10})$. Allowing for a $N$-dependent rank offers new challenges and requires new methods. Working in the Bayesian-optimal setting, we show that whenever the signal has i.i.d. entries the limiting mutual information between signal and data is given by a variational formula involving a rank-one replica symmetric potential. In other words, from the information-theoretic perspective, the case of a (slowly) growing rank is the same as when $M = 1$ (namely, the standard spiked Wigner model). The proof is primarily based on a novel multiscale cavity method allowing for growing rank along with some information-theoretic identities on worst noise for the Gaussian vector channel. We believe that the cavity method developed here will play a role in the analysis of a broader class of inference and spin models where the degrees of freedom are large arrays instead of vectors

Fundamental limits of Non-Linear Low-Rank Matrix Estimation

We consider the task of estimating a low-rank matrix from non-linear and noisy observations. We prove a strong universality result showing that Bayes-optimal performances are characterized by an equivalent Gaussian model with an effective prior, whose parameters are entirely determined by an expansion of the non-linear function. In particular, we show that to reconstruct the signal accurately, one requires a signal-to-noise ratio growing as $N^{\frac 12 (1-1/k_F)}$, where $k_F$ is the first non-zero Fisher information coefficient of the function. We provide asymptotic characterization for the minimal achievable mean squared error (MMSE) and an approximate message-passing algorithm that reaches the MMSE under conditions analogous to the linear version of the problem. We also provide asymptotic errors achieved by methods such as principal component analysis combined with Bayesian denoising, and compare them with Bayes-optimal MMSE.Comment: 42 pages, 2 figure

Spectral Phase Transitions in Non-Linear Wigner Spiked Models

We study the asymptotic behavior of the spectrum of a random matrix where a non-linearity is applied entry-wise to a Wigner matrix perturbed by a rank-one spike with independent and identically distributed entries. In this setting, we show that when the signal-to-noise ratio scale as $N^{\frac{1}{2} (1-1/k_\star)}$, where $k_\star$ is the first non-zero generalized information coefficient of the function, the non-linear spike model effectively behaves as an equivalent spiked Wigner matrix, where the former spike before the non-linearity is now raised to a power $k_\star$. This allows us to study the phase transition of the leading eigenvalues, generalizing part of the work of Baik, Ben Arous and Pech\'e to these non-linear models.Comment: 27 page