70 research outputs found
Universality in polytope phase transitions and message passing algorithms
We consider a class of nonlinear mappings in
indexed by symmetric random matrices with independent entries. Within spin glass theory, special cases of these
mappings correspond to iterating the TAP equations and were studied by
Bolthausen [Comm. Math. Phys. 325 (2014) 333-366]. Within information theory,
they are known as "approximate message passing" algorithms. We study the
high-dimensional (large ) behavior of the iterates of for
polynomial functions , and prove that it is universal; that is, it
depends only on the first two moments of the entries of , under a
sub-Gaussian tail condition. As an application, we prove the universality of a
certain phase transition arising in polytope geometry and compressed sensing.
This solves, for a broad class of random projections, a conjecture by David
Donoho and Jared Tanner.Comment: Published in at http://dx.doi.org/10.1214/14-AAP1010 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Dynamical Functional Theory for Compressed Sensing
We introduce a theoretical approach for designing generalizations of the
approximate message passing (AMP) algorithm for compressed sensing which are
valid for large observation matrices that are drawn from an invariant random
matrix ensemble. By design, the fixed points of the algorithm obey the
Thouless-Anderson-Palmer (TAP) equations corresponding to the ensemble. Using a
dynamical functional approach we are able to derive an effective stochastic
process for the marginal statistics of a single component of the dynamics. This
allows us to design memory terms in the algorithm in such a way that the
resulting fields become Gaussian random variables allowing for an explicit
analysis. The asymptotic statistics of these fields are consistent with the
replica ansatz of the compressed sensing problem.Comment: 5 pages, accepted for ISIT 201
Properties of spatial coupling in compressed sensing
In this paper we address a series of open questions about the construction of
spatially coupled measurement matrices in compressed sensing. For hardware
implementations one is forced to depart from the limiting regime of parameters
in which the proofs of the so-called threshold saturation work. We investigate
quantitatively the behavior under finite coupling range, the dependence on the
shape of the coupling interaction, and optimization of the so-called seed to
minimize distance from optimality. Our analysis explains some of the properties
observed empirically in previous works and provides new insight on spatially
coupled compressed sensing.Comment: 5 pages, 6 figure
On Convergence of Approximate Message Passing
Approximate message passing is an iterative algorithm for compressed sensing
and related applications. A solid theory about the performance and convergence
of the algorithm exists for measurement matrices having iid entries of zero
mean. However, it was observed by several authors that for more general
matrices the algorithm often encounters convergence problems. In this paper we
identify the reason of the non-convergence for measurement matrices with iid
entries and non-zero mean in the context of Bayes optimal inference. Finally we
demonstrate numerically that when the iterative update is changed from parallel
to sequential the convergence is restored.Comment: 5 pages, 3 figure
An Approximate Message Passing Algorithm for Rapid Parameter-Free Compressed Sensing MRI
For certain sensing matrices, the Approximate Message Passing (AMP) algorithm
efficiently reconstructs undersampled signals. However, in Magnetic Resonance
Imaging (MRI), where Fourier coefficients of a natural image are sampled with
variable density, AMP encounters convergence problems. In response we present
an algorithm based on Orthogonal AMP constructed specifically for variable
density partial Fourier sensing matrices. For the first time in this setting a
state evolution has been observed. A practical advantage of state evolution is
that Stein's Unbiased Risk Estimate (SURE) can be effectively implemented,
yielding an algorithm with no free parameters. We empirically evaluate the
effectiveness of the parameter-free algorithm on simulated data and find that
it converges over 5x faster and to a lower mean-squared error solution than
Fast Iterative Shrinkage-Thresholding (FISTA).Comment: 5 pages, 5 figures, IEEE International Conference on Image Processing
(ICIP) 202
Convexity in source separation: Models, geometry, and algorithms
Source separation or demixing is the process of extracting multiple
components entangled within a signal. Contemporary signal processing presents a
host of difficult source separation problems, from interference cancellation to
background subtraction, blind deconvolution, and even dictionary learning.
Despite the recent progress in each of these applications, advances in
high-throughput sensor technology place demixing algorithms under pressure to
accommodate extremely high-dimensional signals, separate an ever larger number
of sources, and cope with more sophisticated signal and mixing models. These
difficulties are exacerbated by the need for real-time action in automated
decision-making systems.
Recent advances in convex optimization provide a simple framework for
efficiently solving numerous difficult demixing problems. This article provides
an overview of the emerging field, explains the theory that governs the
underlying procedures, and surveys algorithms that solve them efficiently. We
aim to equip practitioners with a toolkit for constructing their own demixing
algorithms that work, as well as concrete intuition for why they work
- …