2,490 research outputs found
The dynamics of message passing on dense graphs, with applications to compressed sensing
Approximate message passing algorithms proved to be extremely effective in
reconstructing sparse signals from a small number of incoherent linear
measurements. Extensive numerical experiments further showed that their
dynamics is accurately tracked by a simple one-dimensional iteration termed
state evolution. In this paper we provide the first rigorous foundation to
state evolution. We prove that indeed it holds asymptotically in the large
system limit for sensing matrices with independent and identically distributed
gaussian entries.
While our focus is on message passing algorithms for compressed sensing, the
analysis extends beyond this setting, to a general class of algorithms on dense
graphs. In this context, state evolution plays the role that density evolution
has for sparse graphs.
The proof technique is fundamentally different from the standard approach to
density evolution, in that it copes with large number of short loops in the
underlying factor graph. It relies instead on a conditioning technique recently
developed by Erwin Bolthausen in the context of spin glass theory.Comment: 41 page
Optimal Quantization for Compressive Sensing under Message Passing Reconstruction
We consider the optimal quantization of compressive sensing measurements
following the work on generalization of relaxed belief propagation (BP) for
arbitrary measurement channels. Relaxed BP is an iterative reconstruction
scheme inspired by message passing algorithms on bipartite graphs. Its
asymptotic error performance can be accurately predicted and tracked through
the state evolution formalism. We utilize these results to design mean-square
optimal scalar quantizers for relaxed BP signal reconstruction and empirically
demonstrate the superior error performance of the resulting quantizers.Comment: 5 pages, 3 figures, submitted to IEEE International Symposium on
Information Theory (ISIT) 2011; minor corrections in v
Generalized Approximate Message-Passing Decoder for Universal Sparse Superposition Codes
Sparse superposition (SS) codes were originally proposed as a
capacity-achieving communication scheme over the additive white Gaussian noise
channel (AWGNC) [1]. Very recently, it was discovered that these codes are
universal, in the sense that they achieve capacity over any memoryless channel
under generalized approximate message-passing (GAMP) decoding [2], although
this decoder has never been stated for SS codes. In this contribution we
introduce the GAMP decoder for SS codes, we confirm empirically the
universality of this communication scheme through its study on various channels
and we provide the main analysis tools: state evolution and potential. We also
compare the performance of GAMP with the Bayes-optimal MMSE decoder. We
empirically illustrate that despite the presence of a phase transition
preventing GAMP to reach the optimal performance, spatial coupling allows to
boost the performance that eventually tends to capacity in a proper limit. We
also prove that, in contrast with the AWGNC case, SS codes for binary input
channels have a vanishing error floor in the limit of large codewords.
Moreover, the performance of Hadamard-based encoders is assessed for practical
implementations
Sparse Estimation with the Swept Approximated Message-Passing Algorithm
Approximate Message Passing (AMP) has been shown to be a superior method for
inference problems, such as the recovery of signals from sets of noisy,
lower-dimensionality measurements, both in terms of reconstruction accuracy and
in computational efficiency. However, AMP suffers from serious convergence
issues in contexts that do not exactly match its assumptions. We propose a new
approach to stabilizing AMP in these contexts by applying AMP updates to
individual coefficients rather than in parallel. Our results show that this
change to the AMP iteration can provide theoretically expected, but hitherto
unobtainable, performance for problems on which the standard AMP iteration
diverges. Additionally, we find that the computational costs of this swept
coefficient update scheme is not unduly burdensome, allowing it to be applied
efficiently to signals of large dimensionality.Comment: 11 pages, 3 figures, implementation available at
https://github.com/eric-tramel/SwAMP-Dem
Dynamical Functional Theory for Compressed Sensing
We introduce a theoretical approach for designing generalizations of the
approximate message passing (AMP) algorithm for compressed sensing which are
valid for large observation matrices that are drawn from an invariant random
matrix ensemble. By design, the fixed points of the algorithm obey the
Thouless-Anderson-Palmer (TAP) equations corresponding to the ensemble. Using a
dynamical functional approach we are able to derive an effective stochastic
process for the marginal statistics of a single component of the dynamics. This
allows us to design memory terms in the algorithm in such a way that the
resulting fields become Gaussian random variables allowing for an explicit
analysis. The asymptotic statistics of these fields are consistent with the
replica ansatz of the compressed sensing problem.Comment: 5 pages, accepted for ISIT 201
On Convergence of Approximate Message Passing
Approximate message passing is an iterative algorithm for compressed sensing
and related applications. A solid theory about the performance and convergence
of the algorithm exists for measurement matrices having iid entries of zero
mean. However, it was observed by several authors that for more general
matrices the algorithm often encounters convergence problems. In this paper we
identify the reason of the non-convergence for measurement matrices with iid
entries and non-zero mean in the context of Bayes optimal inference. Finally we
demonstrate numerically that when the iterative update is changed from parallel
to sequential the convergence is restored.Comment: 5 pages, 3 figure
- …