The mean field theory of spin glasses: the heuristic replica approach and recent rigorous results

The mathematically correct computation of the spin glasses free energy in the infinite range limit crowns 25 years of mathematic efforts in solving this model. The exact solution of the model was found many years ago by using a heuristic approach; the results coming from the heuristic approach were crucial in deriving the mathematical results. The mathematical tools used in the rigorous approach are quite different from those of the heuristic approach. In this note we will review the heuristic approach to spin glasses in the light of the rigorous results; we will also discuss some conjectures that may be useful to derive the solution of the model in an alternative way.Comment: 12 pages, 1 figure; lecture at the Flato Colloquia Day, Thursday 27 November, 200

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdu. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.Comment: 22 pages; added details on the replica symmetry assumptio

Support Recovery with Sparsely Sampled Free Random Matrices

Consider a Bernoulli-Gaussian complex $n$-vector whose components are $V_i = X_i B_i$, with X_i \sim \Cc\Nc(0,\Pc_x) and binary $B_i$ mutually independent and iid across $i$. This random $q$-sparse vector is multiplied by a square random matrix \Um, and a randomly chosen subset, of average size $n p$, $p \in [0,1]$, of the resulting vector components is then observed in additive Gaussian noise. We extend the scope of conventional noisy compressive sampling models where \Um is typically %A16 the identity or a matrix with iid components, to allow \Um satisfying a certain freeness condition. This class of matrices encompasses Haar matrices and other unitarily invariant matrices. We use the replica method and the decoupling principle of Guo and Verd\'u, as well as a number of information theoretic bounds, to study the input-output mutual information and the support recovery error rate in the limit of $n \to \infty$. We also extend the scope of the large deviation approach of Rangan, Fletcher and Goyal and characterize the performance of a class of estimators encompassing thresholded linear MMSE and $\ell_1$ relaxation