11,773 research outputs found
Convex recovery of a structured signal from independent random linear measurements
This chapter develops a theoretical analysis of the convex programming method
for recovering a structured signal from independent random linear measurements.
This technique delivers bounds for the sampling complexity that are similar
with recent results for standard Gaussian measurements, but the argument
applies to a much wider class of measurement ensembles. To demonstrate the
power of this approach, the paper presents a short analysis of phase retrieval
by trace-norm minimization. The key technical tool is a framework, due to
Mendelson and coauthors, for bounding a nonnegative empirical process.Comment: 18 pages, 1 figure. To appear in "Sampling Theory, a Renaissance."
v2: minor corrections. v3: updated citations and increased emphasis on
Mendelson's contribution
Simultaneously Structured Models with Application to Sparse and Low-rank Matrices
The topic of recovery of a structured model given a small number of linear
observations has been well-studied in recent years. Examples include recovering
sparse or group-sparse vectors, low-rank matrices, and the sum of sparse and
low-rank matrices, among others. In various applications in signal processing
and machine learning, the model of interest is known to be structured in
several ways at the same time, for example, a matrix that is simultaneously
sparse and low-rank.
Often norms that promote each individual structure are known, and allow for
recovery using an order-wise optimal number of measurements (e.g.,
norm for sparsity, nuclear norm for matrix rank). Hence, it is reasonable to
minimize a combination of such norms. We show that, surprisingly, if we use
multi-objective optimization with these norms, then we can do no better,
order-wise, than an algorithm that exploits only one of the present structures.
This result suggests that to fully exploit the multiple structures, we need an
entirely new convex relaxation, i.e. not one that is a function of the convex
relaxations used for each structure. We then specialize our results to the case
of sparse and low-rank matrices. We show that a nonconvex formulation of the
problem can recover the model from very few measurements, which is on the order
of the degrees of freedom of the matrix, whereas the convex problem obtained
from a combination of the and nuclear norms requires many more
measurements. This proves an order-wise gap between the performance of the
convex and nonconvex recovery problems in this case. Our framework applies to
arbitrary structure-inducing norms as well as to a wide range of measurement
ensembles. This allows us to give performance bounds for problems such as
sparse phase retrieval and low-rank tensor completion.Comment: 38 pages, 9 figure
Structured random measurements in signal processing
Compressed sensing and its extensions have recently triggered interest in
randomized signal acquisition. A key finding is that random measurements
provide sparse signal reconstruction guarantees for efficient and stable
algorithms with a minimal number of samples. While this was first shown for
(unstructured) Gaussian random measurement matrices, applications require
certain structure of the measurements leading to structured random measurement
matrices. Near optimal recovery guarantees for such structured measurements
have been developed over the past years in a variety of contexts. This article
surveys the theory in three scenarios: compressed sensing (sparse recovery),
low rank matrix recovery, and phaseless estimation. The random measurement
matrices to be considered include random partial Fourier matrices, partial
random circulant matrices (subsampled convolutions), matrix completion, and
phase estimation from magnitudes of Fourier type measurements. The article
concludes with a brief discussion of the mathematical techniques for the
analysis of such structured random measurements.Comment: 22 pages, 2 figure
Improving compressed sensing with the diamond norm
In low-rank matrix recovery, one aims to reconstruct a low-rank matrix from a
minimal number of linear measurements. Within the paradigm of compressed
sensing, this is made computationally efficient by minimizing the nuclear norm
as a convex surrogate for rank.
In this work, we identify an improved regularizer based on the so-called
diamond norm, a concept imported from quantum information theory. We show that
-for a class of matrices saturating a certain norm inequality- the descent cone
of the diamond norm is contained in that of the nuclear norm. This suggests
superior reconstruction properties for these matrices. We explicitly
characterize this set of matrices. Moreover, we demonstrate numerically that
the diamond norm indeed outperforms the nuclear norm in a number of relevant
applications: These include signal analysis tasks such as blind matrix
deconvolution or the retrieval of certain unitary basis changes, as well as the
quantum information problem of process tomography with random measurements.
The diamond norm is defined for matrices that can be interpreted as order-4
tensors and it turns out that the above condition depends crucially on that
tensorial structure. In this sense, this work touches on an aspect of the
notoriously difficult tensor completion problem.Comment: 25 pages + Appendix, 7 Figures, published versio
Sharp Time--Data Tradeoffs for Linear Inverse Problems
In this paper we characterize sharp time-data tradeoffs for optimization
problems used for solving linear inverse problems. We focus on the minimization
of a least-squares objective subject to a constraint defined as the sub-level
set of a penalty function. We present a unified convergence analysis of the
gradient projection algorithm applied to such problems. We sharply characterize
the convergence rate associated with a wide variety of random measurement
ensembles in terms of the number of measurements and structural complexity of
the signal with respect to the chosen penalty function. The results apply to
both convex and nonconvex constraints, demonstrating that a linear convergence
rate is attainable even though the least squares objective is not strongly
convex in these settings. When specialized to Gaussian measurements our results
show that such linear convergence occurs when the number of measurements is
merely 4 times the minimal number required to recover the desired signal at all
(a.k.a. the phase transition). We also achieve a slower but geometric rate of
convergence precisely above the phase transition point. Extensive numerical
results suggest that the derived rates exactly match the empirical performance
Phase Retrieval via Matrix Completion
This paper develops a novel framework for phase retrieval, a problem which
arises in X-ray crystallography, diffraction imaging, astronomical imaging and
many other applications. Our approach combines multiple structured
illuminations together with ideas from convex programming to recover the phase
from intensity measurements, typically from the modulus of the diffracted wave.
We demonstrate empirically that any complex-valued object can be recovered from
the knowledge of the magnitude of just a few diffracted patterns by solving a
simple convex optimization problem inspired by the recent literature on matrix
completion. More importantly, we also demonstrate that our noise-aware
algorithms are stable in the sense that the reconstruction degrades gracefully
as the signal-to-noise ratio decreases. Finally, we introduce some theory
showing that one can design very simple structured illumination patterns such
that three diffracted figures uniquely determine the phase of the object we
wish to recover
Robust 1-Bit Compressed Sensing via Hinge Loss Minimization
This work theoretically studies the problem of estimating a structured
high-dimensional signal from noisy -bit Gaussian
measurements. Our recovery approach is based on a simple convex program which
uses the hinge loss function as data fidelity term. While such a risk
minimization strategy is very natural to learn binary output models, such as in
classification, its capacity to estimate a specific signal vector is largely
unexplored. A major difficulty is that the hinge loss is just piecewise linear,
so that its "curvature energy" is concentrated in a single point. This is
substantially different from other popular loss functions considered in signal
estimation, e.g., the square or logistic loss, which are at least locally
strongly convex. It is therefore somewhat unexpected that we can still prove
very similar types of recovery guarantees for the hinge loss estimator, even in
the presence of strong noise. More specifically, our non-asymptotic error
bounds show that stable and robust reconstruction of can be achieved with
the optimal oversampling rate in terms of the number of
measurements . Moreover, we permit a wide class of structural assumptions on
the ground truth signal, in the sense that can belong to an arbitrary
bounded convex set . The proofs of our main results
rely on some recent advances in statistical learning theory due to Mendelson.
In particular, we invoke an adapted version of Mendelson's small ball method
that allows us to establish a quadratic lower bound on the error of the first
order Taylor approximation of the empirical hinge loss function
- …