4,368 research outputs found
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary
signal processing, where the goal is to reconstruct an unknown signal from
partial indirect, and possibly noisy, measurements of it. A now standard method
for recovering the unknown signal is to solve a convex optimization problem
that enforces some prior knowledge about its structure. This has proved
efficient in many problems routinely encountered in imaging sciences,
statistics and machine learning. This chapter delivers a review of recent
advances in the field where the regularization prior promotes solutions
conforming to some notion of simplicity/low-complexity. These priors encompass
as popular examples sparsity and group sparsity (to capture the compressibility
of natural signals and images), total variation and analysis sparsity (to
promote piecewise regularity), and low-rank (as natural extension of sparsity
to matrix-valued data). Our aim is to provide a unified treatment of all these
regularizations under a single umbrella, namely the theory of partial
smoothness. This framework is very general and accommodates all low-complexity
regularizers just mentioned, as well as many others. Partial smoothness turns
out to be the canonical way to encode low-dimensional models that can be linear
spaces or more general smooth manifolds. This review is intended to serve as a
one stop shop toward the understanding of the theoretical properties of the
so-regularized solutions. It covers a large spectrum including: (i) recovery
guarantees and stability to noise, both in terms of -stability and
model (manifold) identification; (ii) sensitivity analysis to perturbations of
the parameters involved (in particular the observations), with applications to
unbiased risk estimation ; (iii) convergence properties of the forward-backward
proximal splitting scheme, that is particularly well suited to solve the
corresponding large-scale regularized optimization problem
Designing structured tight frames via an alternating projection method
Tight frames, also known as general Welch-bound- equality sequences, generalize orthonormal systems. Numerous applications - including communications, coding, and sparse approximation- require finite-dimensional tight frames that possess additional structural properties. This paper proposes an alternating projection method that is versatile enough to solve a huge class of inverse eigenvalue problems (IEPs), which includes the frame design problem. To apply this method, one needs only to solve a matrix nearness problem that arises naturally from the design specifications. Therefore, it is the fast and easy to develop versions of the algorithm that target new design problems. Alternating projection will often succeed even if algebraic constructions are unavailable. To demonstrate that alternating projection is an effective tool for frame design, the paper studies some important structural properties in detail. First, it addresses the most basic design problem: constructing tight frames with prescribed vector norms. Then, it discusses equiangular tight frames, which are natural dictionaries for sparse approximation. Finally, it examines tight frames whose individual vectors have low peak-to-average-power ratio (PAR), which is a valuable property for code-division multiple-access (CDMA) applications. Numerical experiments show that the proposed algorithm succeeds in each of these three cases. The appendices investigate the convergence properties of the algorithm
Parameter Identification in a Probabilistic Setting
Parameter identification problems are formulated in a probabilistic language,
where the randomness reflects the uncertainty about the knowledge of the true
values. This setting allows conceptually easily to incorporate new information,
e.g. through a measurement, by connecting it to Bayes's theorem. The unknown
quantity is modelled as a (may be high-dimensional) random variable. Such a
description has two constituents, the measurable function and the measure. One
group of methods is identified as updating the measure, the other group changes
the measurable function. We connect both groups with the relatively recent
methods of functional approximation of stochastic problems, and introduce
especially in combination with the second group of methods a new procedure
which does not need any sampling, hence works completely deterministically. It
also seems to be the fastest and more reliable when compared with other
methods. We show by example that it also works for highly nonlinear non-smooth
problems with non-Gaussian measures.Comment: 29 pages, 16 figure
Multi-resolution Low-rank Tensor Formats
We describe a simple, black-box compression format for tensors with a
multiscale structure. By representing the tensor as a sum of compressed tensors
defined on increasingly coarse grids, we capture low-rank structures on each
grid-scale, and we show how this leads to an increase in compression for a
fixed accuracy. We devise an alternating algorithm to represent a given tensor
in the multiresolution format and prove local convergence guarantees. In two
dimensions, we provide examples that show that this approach can beat the
Eckart-Young theorem, and for dimensions higher than two, we achieve higher
compression than the tensor-train format on six real-world datasets. We also
provide results on the closedness and stability of the tensor format and
discuss how to perform common linear algebra operations on the level of the
compressed tensors.Comment: 29 pages, 9 figure
- …