1,320 research outputs found
Orthogonal Matching Pursuit: A Brownian Motion Analysis
A well-known analysis of Tropp and Gilbert shows that orthogonal matching
pursuit (OMP) can recover a k-sparse n-dimensional real vector from 4 k log(n)
noise-free linear measurements obtained through a random Gaussian measurement
matrix with a probability that approaches one as n approaches infinity. This
work strengthens this result by showing that a lower number of measurements, 2
k log(n - k), is in fact sufficient for asymptotic recovery. More generally,
when the sparsity level satisfies kmin <= k <= kmax but is unknown, 2 kmax
log(n - kmin) measurements is sufficient. Furthermore, this number of
measurements is also sufficient for detection of the sparsity pattern (support)
of the vector with measurement errors provided the signal-to-noise ratio (SNR)
scales to infinity. The scaling 2 k log(n - k) exactly matches the number of
measurements required by the more complex lasso method for signal recovery with
a similar SNR scaling.Comment: 11 pages, 2 figure
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
The replica method is a non-rigorous but well-known technique from
statistical physics used in the asymptotic analysis of large, random, nonlinear
problems. This paper applies the replica method, under the assumption of
replica symmetry, to study estimators that are maximum a posteriori (MAP) under
a postulated prior distribution. It is shown that with random linear
measurements and Gaussian noise, the replica-symmetric prediction of the
asymptotic behavior of the postulated MAP estimate of an n-dimensional vector
"decouples" as n scalar postulated MAP estimators. The result is based on
applying a hardening argument to the replica analysis of postulated posterior
mean estimators of Tanaka and of Guo and Verdu.
The replica-symmetric postulated MAP analysis can be readily applied to many
estimators used in compressed sensing, including basis pursuit, lasso, linear
estimation with thresholding, and zero norm-regularized estimation. In the case
of lasso estimation the scalar estimator reduces to a soft-thresholding
operator, and for zero norm-regularized estimation it reduces to a
hard-threshold. Among other benefits, the replica method provides a
computationally-tractable method for precisely predicting various performance
metrics including mean-squared error and sparsity pattern recovery probability.Comment: 22 pages; added details on the replica symmetry assumptio
Vector Approximate Message Passing for the Generalized Linear Model
The generalized linear model (GLM), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
output , arises in a range of applications such
as robust regression, binary classification, quantized compressed sensing,
phase retrieval, photon-limited imaging, and inference from neural spike
trains. When is large and i.i.d. Gaussian, the generalized
approximate message passing (GAMP) algorithm is an efficient means of MAP or
marginal inference, and its performance can be rigorously characterized by a
scalar state evolution. For general , though, GAMP can
misbehave. Damping and sequential-updating help to robustify GAMP, but their
effects are limited. Recently, a "vector AMP" (VAMP) algorithm was proposed for
additive white Gaussian noise channels. VAMP extends AMP's guarantees from
i.i.d. Gaussian to the larger class of rotationally invariant
. In this paper, we show how VAMP can be extended to the GLM.
Numerical experiments show that the proposed GLM-VAMP is much more robust to
ill-conditioning in than damped GAMP
Optimal Quantization for Compressive Sensing under Message Passing Reconstruction
We consider the optimal quantization of compressive sensing measurements
following the work on generalization of relaxed belief propagation (BP) for
arbitrary measurement channels. Relaxed BP is an iterative reconstruction
scheme inspired by message passing algorithms on bipartite graphs. Its
asymptotic error performance can be accurately predicted and tracked through
the state evolution formalism. We utilize these results to design mean-square
optimal scalar quantizers for relaxed BP signal reconstruction and empirically
demonstrate the superior error performance of the resulting quantizers.Comment: 5 pages, 3 figures, submitted to IEEE International Symposium on
Information Theory (ISIT) 2011; minor corrections in v
- …