7,454 research outputs found
Recovery of Sparse Probability Measures via Convex Programming
We consider the problem of cardinality penalized optimization of a convex function
over the probability simplex with additional convex constraints. The classical
â„“_1 regularizer fails to promote sparsity on the probability simplex since â„“_1 norm
on the probability simplex is trivially constant. We propose a direct relaxation of
the minimum cardinality problem and show that it can be efficiently solved using
convex programming. As a first application we consider recovering a sparse probability
measure given moment constraints, in which our formulation becomes linear
programming, hence can be solved very efficiently. A sufficient condition for
exact recovery of the minimum cardinality solution is derived for arbitrary affine
constraints. We then develop a penalized version for the noisy setting which can
be solved using second order cone programs. The proposed method outperforms
known rescaling heuristics based on â„“_1 norm. As a second application we consider
convex clustering using a sparse Gaussian mixture and compare our results with
the well known soft k-means algorithm
Estimation in high dimensions: a geometric perspective
This tutorial provides an exposition of a flexible geometric framework for
high dimensional estimation problems with constraints. The tutorial develops
geometric intuition about high dimensional sets, justifies it with some results
of asymptotic convex geometry, and demonstrates connections between geometric
results and estimation problems. The theory is illustrated with applications to
sparse recovery, matrix completion, quantization, linear and logistic
regression and generalized linear models.Comment: 56 pages, 9 figures. Multiple minor change
Performance Analysis of Sparse Recovery Based on Constrained Minimal Singular Values
The stability of sparse signal reconstruction is investigated in this paper.
We design efficient algorithms to verify the sufficient condition for unique
sparse recovery. One of our algorithm produces comparable results with
the state-of-the-art technique and performs orders of magnitude faster. We show
that the -constrained minimal singular value (-CMSV) of the
measurement matrix determines, in a very concise manner, the recovery
performance of -based algorithms such as the Basis Pursuit, the Dantzig
selector, and the LASSO estimator. Compared with performance analysis involving
the Restricted Isometry Constant, the arguments in this paper are much less
complicated and provide more intuition on the stability of sparse signal
recovery. We show also that, with high probability, the subgaussian ensemble
generates measurement matrices with -CMSVs bounded away from zero, as
long as the number of measurements is relatively large. To compute the
-CMSV and its lower bound, we design two algorithms based on the
interior point algorithm and the semi-definite relaxation
Computational Complexity versus Statistical Performance on Sparse Recovery Problems
We show that several classical quantities controlling compressed sensing
performance directly match classical parameters controlling algorithmic
complexity. We first describe linearly convergent restart schemes on
first-order methods solving a broad range of compressed sensing problems, where
sharpness at the optimum controls convergence speed. We show that for sparse
recovery problems, this sharpness can be written as a condition number, given
by the ratio between true signal sparsity and the largest signal size that can
be recovered by the observation matrix. In a similar vein, Renegar's condition
number is a data-driven complexity measure for convex programs, generalizing
classical condition numbers for linear systems. We show that for a broad class
of compressed sensing problems, the worst case value of this algorithmic
complexity measure taken over all signals matches the restricted singular value
of the observation matrix which controls robust recovery performance. Overall,
this means in both cases that, in compressed sensing problems, a single
parameter directly controls both computational complexity and recovery
performance. Numerical experiments illustrate these points using several
classical algorithms.Comment: Final version, to appear in information and Inferenc
Convex Clustering via Optimal Mass Transport
We consider approximating distributions within the framework of optimal mass
transport and specialize to the problem of clustering data sets. Distances
between distributions are measured in the Wasserstein metric. The main problem
we consider is that of approximating sample distributions by ones with sparse
support. This provides a new viewpoint to clustering. We propose different
relaxations of a cardinality function which penalizes the size of the support
set. We establish that a certain relaxation provides the tightest convex lower
approximation to the cardinality penalty. We compare the performance of
alternative relaxations on a numerical study on clustering.Comment: 12 pages, 12 figure
- …