RSB Decoupling Property of MAP Estimators

The large-system decoupling property of a MAP estimator is studied when it estimates the i.i.d. vector $\boldsymbol{x}$ from the observation $\boldsymbol{y}=\mathbf{A}\boldsymbol{x}+\boldsymbol{z}$ with $\mathbf{A}$ being chosen from a wide range of matrix ensembles, and the noise vector $\boldsymbol{z}$ being i.i.d. and Gaussian. Using the replica method, we show that the marginal joint distribution of any two corresponding input and output symbols converges to a deterministic distribution which describes the input-output distribution of a single user system followed by a MAP estimator. Under the $b$RSB assumption, the single user system is a scalar channel with additive noise where the noise term is given by the sum of an independent Gaussian random variable and $b$ correlated interference terms. As the $b$RSB assumption reduces to RS, the interference terms vanish which results in the formerly studied RS decoupling principle.Comment: 5 pages, presented in Information Theory Workshop 201

Asymptotics of Nonlinear LSE Precoders with Applications to Transmit Antenna Selection

This paper studies the large-system performance of Least Square Error (LSE) precoders which~minimize~the~input-output distortion over an arbitrary support subject to a general penalty function. The asymptotics are determined via the replica method in a general form which encloses the Replica Symmetric (RS) and Replica Symmetry Breaking (RSB) ans\"atze. As a result, the "marginal decoupling property" of LSE precoders for $b$-steps of RSB is derived. The generality of the studied setup enables us to address special cases in which the number of active transmit antennas are constrained. Our numerical investigations depict that the computationally efficient forms of LSE precoders based on "$\ell_1$-norm" minimization perform close to the cases with "zero-norm" penalty function which have a considerable improvements compared to the random antenna selection. For the case with BPSK signals and restricted number of active antennas, the results show that RS fails to predict the performance while the RSB ansatz is consistent with theoretical bounds.Comment: 5 pages; 2 figures; to be presented at ISIT 201

Replica Symmetry Breaking in Compressive Sensing

For noisy compressive sensing systems, the asymptotic distortion with respect to an arbitrary distortion function is determined when a general class of least-square based reconstruction schemes is employed. The sampling matrix is considered to belong to a large ensemble of random matrices including i.i.d. and projector matrices, and the source vector is assumed to be i.i.d. with a desired distribution. We take a statistical mechanical approach by representing the asymptotic distortion as a macroscopic parameter of a spin glass and employing the replica method for the large-system analysis. In contrast to earlier studies, we evaluate the general replica ansatz which includes the RS ansatz as well as RSB. The generality of the solution enables us to study the impact of symmetry breaking. Our numerical investigations depict that for the reconstruction scheme with the "zero-norm" penalty function, the RS fails to predict the asymptotic distortion for relatively large compression rates; however, the one-step RSB ansatz gives a valid prediction of the performance within a larger regime of compression rates.Comment: 7 pages, 3 figures, presented at ITA 201

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

The replica method is a non-rigorous but well-known technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method, under the assumption of replica symmetry, to study estimators that are maximum a posteriori (MAP) under a postulated prior distribution. It is shown that with random linear measurements and Gaussian noise, the replica-symmetric prediction of the asymptotic behavior of the postulated MAP estimate of an n-dimensional vector "decouples" as n scalar postulated MAP estimators. The result is based on applying a hardening argument to the replica analysis of postulated posterior mean estimators of Tanaka and of Guo and Verdu. The replica-symmetric postulated MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero norm-regularized estimation it reduces to a hard-threshold. Among other benefits, the replica method provides a computationally-tractable method for precisely predicting various performance metrics including mean-squared error and sparsity pattern recovery probability.Comment: 22 pages; added details on the replica symmetry assumptio

Compressed Sensing Performance Analysis via Replica Method using Bayesian framework

Compressive sensing (CS) is a new methodology to capture signals at lower rate than the Nyquist sampling rate when the signals are sparse or sparse in some domain. The performance of CS estimators is analyzed in this paper using tools from statistical mechanics, especially called replica method. This method has been used to analyze communication systems like Code Division Multiple Access (CDMA) and multiple input multi- ple output (MIMO) systems with large size. Replica analysis, now days rigorously proved, is an efficient tool to analyze large systems in general. Specifically, we analyze the performance of some of the estimators used in CS like LASSO (the Least Absolute Shrinkage and Selection Operator) estimator and Zero-Norm regularizing estimator as a special case of maximum a posteriori (MAP) estimator by using Bayesian framework to connect the CS estimators and replica method. We use both replica symmetric (RS) ansatz and one-step replica symmetry breaking (1RSB) ansatz, clamming the latter is efficient when the problem is not convex. This work is more analytical in its form. It is deferred for next step to focus on the numerical results.Comment: The analytical work and results were presented at the 2012 IEEE European School of Information Theory in Antalya, Turkey between the 16th and the 20th of Apri