570 research outputs found
Replica Symmetry Breaking in Compressive Sensing
For noisy compressive sensing systems, the asymptotic distortion with respect
to an arbitrary distortion function is determined when a general class of
least-square based reconstruction schemes is employed. The sampling matrix is
considered to belong to a large ensemble of random matrices including i.i.d.
and projector matrices, and the source vector is assumed to be i.i.d. with a
desired distribution. We take a statistical mechanical approach by representing
the asymptotic distortion as a macroscopic parameter of a spin glass and
employing the replica method for the large-system analysis. In contrast to
earlier studies, we evaluate the general replica ansatz which includes the RS
ansatz as well as RSB. The generality of the solution enables us to study the
impact of symmetry breaking. Our numerical investigations depict that for the
reconstruction scheme with the "zero-norm" penalty function, the RS fails to
predict the asymptotic distortion for relatively large compression rates;
however, the one-step RSB ansatz gives a valid prediction of the performance
within a larger regime of compression rates.Comment: 7 pages, 3 figures, presented at ITA 201
Compressed Sensing Performance Analysis via Replica Method using Bayesian framework
Compressive sensing (CS) is a new methodology to capture signals at lower
rate than the Nyquist sampling rate when the signals are sparse or sparse in
some domain. The performance of CS estimators is analyzed in this paper using
tools from statistical mechanics, especially called replica method. This method
has been used to analyze communication systems like Code Division Multiple
Access (CDMA) and multiple input multi- ple output (MIMO) systems with large
size. Replica analysis, now days rigorously proved, is an efficient tool to
analyze large systems in general. Specifically, we analyze the performance of
some of the estimators used in CS like LASSO (the Least Absolute Shrinkage and
Selection Operator) estimator and Zero-Norm regularizing estimator as a special
case of maximum a posteriori (MAP) estimator by using Bayesian framework to
connect the CS estimators and replica method. We use both replica symmetric
(RS) ansatz and one-step replica symmetry breaking (1RSB) ansatz, clamming the
latter is efficient when the problem is not convex. This work is more
analytical in its form. It is deferred for next step to focus on the numerical
results.Comment: The analytical work and results were presented at the 2012 IEEE
European School of Information Theory in Antalya, Turkey between the 16th and
the 20th of Apri
A typical reconstruction limit of compressed sensing based on Lp-norm minimization
We consider the problem of reconstructing an -dimensional continuous
vector \bx from constraints which are generated by its linear
transformation under the assumption that the number of non-zero elements of
\bx is typically limited to (). Problems of this
type can be solved by minimizing a cost function with respect to the -norm
||\bx||_p=\lim_{\epsilon \to +0}\sum_{i=1}^N |x_i|^{p+\epsilon}, subject to
the constraints under an appropriate condition. For several , we assess a
typical case limit , which represents a critical relation
between and for successfully reconstructing the original
vector by minimization for typical situations in the limit
with keeping finite, utilizing the replica method. For ,
is considerably smaller than its worst case counterpart, which
has been rigorously derived by existing literature of information theory.Comment: 12 pages, 2 figure
Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery
Noiseless compressive sensing is a protocol that enables undersampling and
later recovery of a signal without loss of information. This compression is
possible because the signal is usually sufficiently sparse in a given basis.
Currently, the algorithm offering the best tradeoff between compression rate,
robustness, and speed for compressive sensing is the LASSO (l1-norm bias)
algorithm. However, many studies have pointed out the possibility that the
implementation of lp-norms biases, with p smaller than one, could give better
performance while sacrificing convexity. In this work, we focus specifically on
the extreme case of the l0-based reconstruction, a task that is complicated by
the discontinuity of the loss. In the first part of the paper, we describe via
statistical physics methods, and in particular the replica method, how the
solutions to this optimization problem are arranged in a clustered structure.
We observe two distinct regimes: one at low compression rate where the signal
can be recovered exactly, and one at high compression rate where the signal
cannot be recovered accurately. In the second part, we present two
message-passing algorithms based on our first results for the l0-norm
optimization problem. The proposed algorithms are able to recover the signal at
compression rates higher than the ones achieved by LASSO while being
computationally efficient
- …