1,160 research outputs found
Statistical Compressive Sensing of Gaussian Mixture Models
A new framework of compressive sensing (CS), namely statistical compressive
sensing (SCS), that aims at efficiently sampling a collection of signals that
follow a statistical distribution and achieving accurate reconstruction on
average, is introduced. For signals following a Gaussian distribution, with
Gaussian or Bernoulli sensing matrices of O(k) measurements, considerably
smaller than the O(k log(N/k)) required by conventional CS, where N is the
signal dimension, and with an optimal decoder implemented with linear
filtering, significantly faster than the pursuit decoders applied in
conventional CS, the error of SCS is shown tightly upper bounded by a constant
times the k-best term approximation error, with overwhelming probability. The
failure probability is also significantly smaller than that of conventional CS.
Stronger yet simpler results further show that for any sensing matrix, the
error of Gaussian SCS is upper bounded by a constant times the k-best term
approximation with probability one, and the bound constant can be efficiently
calculated. For signals following Gaussian mixture models, SCS with a piecewise
linear decoder is introduced and shown to produce for real images better
results than conventional CS based on sparse models
C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework
Sparse modeling is a powerful framework for data analysis and processing.
Traditionally, encoding in this framework is performed by solving an
L1-regularized linear regression problem, commonly referred to as Lasso or
Basis Pursuit. In this work we combine the sparsity-inducing property of the
Lasso model at the individual feature level, with the block-sparsity property
of the Group Lasso model, where sparse groups of features are jointly encoded,
obtaining a sparsity pattern hierarchically structured. This results in the
Hierarchical Lasso (HiLasso), which shows important practical modeling
advantages. We then extend this approach to the collaborative case, where a set
of simultaneously coded signals share the same sparsity pattern at the higher
(group) level, but not necessarily at the lower (inside the group) level,
obtaining the collaborative HiLasso model (C-HiLasso). Such signals then share
the same active groups, or classes, but not necessarily the same active set.
This model is very well suited for applications such as source identification
and separation. An efficient optimization procedure, which guarantees
convergence to the global optimum, is developed for these new models. The
underlying presentation of the new framework and optimization approach is
complemented with experimental examples and theoretical results regarding
recovery guarantees for the proposed models
Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering
State-of-the-art subspace clustering methods are based on expressing each
data point as a linear combination of other data points while regularizing the
matrix of coefficients with , or nuclear norms.
regularization is guaranteed to give a subspace-preserving affinity (i.e.,
there are no connections between points from different subspaces) under broad
theoretical conditions, but the clusters may not be connected. and
nuclear norm regularization often improve connectivity, but give a
subspace-preserving affinity only for independent subspaces. Mixed ,
and nuclear norm regularizations offer a balance between the
subspace-preserving and connectedness properties, but this comes at the cost of
increased computational complexity. This paper studies the geometry of the
elastic net regularizer (a mixture of the and norms) and uses
it to derive a provably correct and scalable active set method for finding the
optimal coefficients. Our geometric analysis also provides a theoretical
justification and a geometric interpretation for the balance between the
connectedness (due to regularization) and subspace-preserving (due to
regularization) properties for elastic net subspace clustering. Our
experiments show that the proposed active set method not only achieves
state-of-the-art clustering performance, but also efficiently handles
large-scale datasets.Comment: 15 pages, 6 figures, accepted to CVPR 2016 for oral presentatio
- β¦