138 research outputs found
On the linear independence of spikes and sines
The purpose of this work is to survey what is known about the linear
independence of spikes and sines. The paper provides new results for the case
where the locations of the spikes and the frequencies of the sines are chosen
at random. This problem is equivalent to studying the spectral norm of a random
submatrix drawn from the discrete Fourier transform matrix. The proof involves
depends on an extrapolation argument of Bourgain and Tzafriri.Comment: 16 pages, 4 figures. Revision with new proof of major theorem
Analysis of Basis Pursuit Via Capacity Sets
Finding the sparsest solution for an under-determined linear system
of equations is of interest in many applications. This problem is
known to be NP-hard. Recent work studied conditions on the support size of
that allow its recovery using L1-minimization, via the Basis Pursuit
algorithm. These conditions are often relying on a scalar property of
called the mutual-coherence. In this work we introduce an alternative set of
features of an arbitrarily given , called the "capacity sets". We show how
those could be used to analyze the performance of the basis pursuit, leading to
improved bounds and predictions of performance. Both theoretical and numerical
methods are presented, all using the capacity values, and shown to lead to
improved assessments of the basis pursuit success in finding the sparest
solution of
Templates for Convex Cone Problems with Applications to Sparse Signal Recovery
This paper develops a general framework for solving a variety of convex cone
problems that frequently arise in signal processing, machine learning,
statistics, and other fields. The approach works as follows: first, determine a
conic formulation of the problem; second, determine its dual; third, apply
smoothing; and fourth, solve using an optimal first-order method. A merit of
this approach is its flexibility: for example, all compressed sensing problems
can be solved via this approach. These include models with objective
functionals such as the total-variation norm, ||Wx||_1 where W is arbitrary, or
a combination thereof. In addition, the paper also introduces a number of
technical contributions such as a novel continuation scheme, a novel approach
for controlling the step size, and some new results showing that the smooth and
unsmoothed problems are sometimes formally equivalent. Combined with our
framework, these lead to novel, stable and computationally efficient
algorithms. For instance, our general implementation is competitive with
state-of-the-art methods for solving intensively studied problems such as the
LASSO. Further, numerical experiments show that one can solve the Dantzig
selector problem, for which no efficient large-scale solvers exist, in a few
hundred iterations. Finally, the paper is accompanied with a software release.
This software is not a single, monolithic solver; rather, it is a suite of
programs and routines designed to serve as building blocks for constructing
complete algorithms.Comment: The TFOCS software is available at http://tfocs.stanford.edu This
version has updated reference
On Model-Based RIP-1 Matrices
The Restricted Isometry Property (RIP) is a fundamental property of a matrix
enabling sparse recovery. Informally, an m x n matrix satisfies RIP of order k
in the l_p norm if ||Ax||_p \approx ||x||_p for any vector x that is k-sparse,
i.e., that has at most k non-zeros. The minimal number of rows m necessary for
the property to hold has been extensively investigated, and tight bounds are
known. Motivated by signal processing models, a recent work of Baraniuk et al
has generalized this notion to the case where the support of x must belong to a
given model, i.e., a given family of supports. This more general notion is much
less understood, especially for norms other than l_2. In this paper we present
tight bounds for the model-based RIP property in the l_1 norm. Our bounds hold
for the two most frequently investigated models: tree-sparsity and
block-sparsity. We also show implications of our results to sparse recovery
problems.Comment: Version 3 corrects a few errors present in the earlier version. In
particular, it states and proves correct upper and lower bounds for the
number of rows in RIP-1 matrices for the block-sparse model. The bounds are
of the form k log_b n, not k log_k n as stated in the earlier versio
Recent Progress in Shearlet Theory: Systematic Construction of Shearlet Dilation Groups, Characterization of Wavefront Sets, and New Embeddings
The class of generalized shearlet dilation groups has recently been developed
to allow the unified treatment of various shearlet groups and associated
shearlet transforms that had previously been studied on a case-by-case basis.
We consider several aspects of these groups: First, their systematic
construction from associative algebras, secondly, their suitability for the
characterization of wavefront sets, and finally, the question of constructing
embeddings into the symplectic group in a way that intertwines the
quasi-regular representation with the metaplectic one. For all questions, it is
possible to treat the full class of generalized shearlet groups in a
comprehensive and unified way, thus generalizing known results to an infinity
of new cases. Our presentation emphasizes the interplay between the algebraic
structure underlying the construction of the shearlet dilation groups, the
geometric properties of the dual action, and the analytic properties of the
associated shearlet transforms.Comment: 28 page
Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints
Regularization of ill-posed linear inverse problems via penalization
has been proposed for cases where the solution is known to be (almost) sparse.
One way to obtain the minimizer of such an penalized functional is via
an iterative soft-thresholding algorithm. We propose an alternative
implementation to -constraints, using a gradient method, with
projection on -balls. The corresponding algorithm uses again iterative
soft-thresholding, now with a variable thresholding parameter. We also propose
accelerated versions of this iterative method, using ingredients of the
(linear) steepest descent method. We prove convergence in norm for one of these
projected gradient methods, without and with acceleration.Comment: 24 pages, 5 figures. v2: added reference, some amendments, 27 page
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary
signal processing, where the goal is to reconstruct an unknown signal from
partial indirect, and possibly noisy, measurements of it. A now standard method
for recovering the unknown signal is to solve a convex optimization problem
that enforces some prior knowledge about its structure. This has proved
efficient in many problems routinely encountered in imaging sciences,
statistics and machine learning. This chapter delivers a review of recent
advances in the field where the regularization prior promotes solutions
conforming to some notion of simplicity/low-complexity. These priors encompass
as popular examples sparsity and group sparsity (to capture the compressibility
of natural signals and images), total variation and analysis sparsity (to
promote piecewise regularity), and low-rank (as natural extension of sparsity
to matrix-valued data). Our aim is to provide a unified treatment of all these
regularizations under a single umbrella, namely the theory of partial
smoothness. This framework is very general and accommodates all low-complexity
regularizers just mentioned, as well as many others. Partial smoothness turns
out to be the canonical way to encode low-dimensional models that can be linear
spaces or more general smooth manifolds. This review is intended to serve as a
one stop shop toward the understanding of the theoretical properties of the
so-regularized solutions. It covers a large spectrum including: (i) recovery
guarantees and stability to noise, both in terms of -stability and
model (manifold) identification; (ii) sensitivity analysis to perturbations of
the parameters involved (in particular the observations), with applications to
unbiased risk estimation ; (iii) convergence properties of the forward-backward
proximal splitting scheme, that is particularly well suited to solve the
corresponding large-scale regularized optimization problem
Restricted Isometries for Partial Random Circulant Matrices
In the theory of compressed sensing, restricted isometry analysis has become
a standard tool for studying how efficiently a measurement matrix acquires
information about sparse and compressible signals. Many recovery algorithms are
known to succeed when the restricted isometry constants of the sampling matrix
are small. Many potential applications of compressed sensing involve a
data-acquisition process that proceeds by convolution with a random pulse
followed by (nonrandom) subsampling. At present, the theoretical analysis of
this measurement technique is lacking. This paper demonstrates that the th
order restricted isometry constant is small when the number of samples
satisfies , where is the length of the pulse.
This bound improves on previous estimates, which exhibit quadratic scaling
The road to deterministic matrices with the restricted isometry property
The restricted isometry property (RIP) is a well-known matrix condition that
provides state-of-the-art reconstruction guarantees for compressed sensing.
While random matrices are known to satisfy this property with high probability,
deterministic constructions have found less success. In this paper, we consider
various techniques for demonstrating RIP deterministically, some popular and
some novel, and we evaluate their performance. In evaluating some techniques,
we apply random matrix theory and inadvertently find a simple alternative proof
that certain random matrices are RIP. Later, we propose a particular class of
matrices as candidates for being RIP, namely, equiangular tight frames (ETFs).
Using the known correspondence between real ETFs and strongly regular graphs,
we investigate certain combinatorial implications of a real ETF being RIP.
Specifically, we give probabilistic intuition for a new bound on the clique
number of Paley graphs of prime order, and we conjecture that the corresponding
ETFs are RIP in a manner similar to random matrices.Comment: 24 page
- …