4,264 research outputs found
Matrix Recipes for Hard Thresholding Methods
In this paper, we present and analyze a new set of low-rank recovery
algorithms for linear inverse problems within the class of hard thresholding
methods. We provide strategies on how to set up these algorithms via basic
ingredients for different configurations to achieve complexity vs. accuracy
tradeoffs. Moreover, we study acceleration schemes via memory-based techniques
and randomized, -approximate matrix projections to decrease the
computational costs in the recovery process. For most of the configurations, we
present theoretical analysis that guarantees convergence under mild problem
conditions. Simulation results demonstrate notable performance improvements as
compared to state-of-the-art algorithms both in terms of reconstruction
accuracy and computational complexity.Comment: 26 page
Deep Network Classification by Scattering and Homotopy Dictionary Learning
We introduce a sparse scattering deep convolutional neural network, which
provides a simple model to analyze properties of deep representation learning
for classification. Learning a single dictionary matrix with a classifier
yields a higher classification accuracy than AlexNet over the ImageNet 2012
dataset. The network first applies a scattering transform that linearizes
variabilities due to geometric transformations such as translations and small
deformations. A sparse dictionary coding reduces intra-class
variability while preserving class separation through projections over unions
of linear spaces. It is implemented in a deep convolutional network with a
homotopy algorithm having an exponential convergence. A convergence proof is
given in a general framework that includes ALISTA. Classification results are
analyzed on ImageNet
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary
signal processing, where the goal is to reconstruct an unknown signal from
partial indirect, and possibly noisy, measurements of it. A now standard method
for recovering the unknown signal is to solve a convex optimization problem
that enforces some prior knowledge about its structure. This has proved
efficient in many problems routinely encountered in imaging sciences,
statistics and machine learning. This chapter delivers a review of recent
advances in the field where the regularization prior promotes solutions
conforming to some notion of simplicity/low-complexity. These priors encompass
as popular examples sparsity and group sparsity (to capture the compressibility
of natural signals and images), total variation and analysis sparsity (to
promote piecewise regularity), and low-rank (as natural extension of sparsity
to matrix-valued data). Our aim is to provide a unified treatment of all these
regularizations under a single umbrella, namely the theory of partial
smoothness. This framework is very general and accommodates all low-complexity
regularizers just mentioned, as well as many others. Partial smoothness turns
out to be the canonical way to encode low-dimensional models that can be linear
spaces or more general smooth manifolds. This review is intended to serve as a
one stop shop toward the understanding of the theoretical properties of the
so-regularized solutions. It covers a large spectrum including: (i) recovery
guarantees and stability to noise, both in terms of -stability and
model (manifold) identification; (ii) sensitivity analysis to perturbations of
the parameters involved (in particular the observations), with applications to
unbiased risk estimation ; (iii) convergence properties of the forward-backward
proximal splitting scheme, that is particularly well suited to solve the
corresponding large-scale regularized optimization problem
- …