Polar Polytopes and Recovery of Sparse Representations

Suppose we have a signal y which we wish to represent using a linear combination of a number of basis atoms a_i, y=sum_i x_i a_i = Ax. The problem of finding the minimum L0 norm representation for y is a hard problem. The Basis Pursuit (BP) approach proposes to find the minimum L1 norm representation instead, which corresponds to a linear program (LP) that can be solved using modern LP techniques, and several recent authors have given conditions for the BP (minimum L1 norm) and sparse (minimum L0 solutions) representations to be identical. In this paper, we explore this sparse representation problem} using the geometry of convex polytopes, as recently introduced into the field by Donoho. By considering the dual LP we find that the so-called polar polytope P of the centrally-symmetric polytope P whose vertices are the atom pairs +-a_i is particularly helpful in providing us with geometrical insight into optimality conditions given by Fuchs and Tropp for non-unit-norm atom sets. In exploring this geometry we are able to tighten some of these earlier results, showing for example that the Fuchs condition is both necessary and sufficient for L1-unique-optimality, and that there are situations where Orthogonal Matching Pursuit (OMP) can eventually find all L1-unique-optimal solutions with m nonzeros even if ERC fails for m, if allowed to run for more than m steps

Group polytope faces pursuit for recovery of block-sparse signals

This is the accepted version of the article. The final publication is available at link.springer.com. http://www.springerlink.com/content/e0r61416446277w0

The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation

We provide a necessary and sufficient condition for the uniqueness of penalized least-squares estimators whose penalty term is given by a norm with a polytope unit ball, covering a wide range of methods including SLOPE and LASSO, as well as the related method of basis pursuit. We consider a strong type of uniqueness that is relevant for statistical problems. The uniqueness condition is geometric and involves how the row span of the design matrix intersects the faces of the dual norm unit ball, which for SLOPE is given by the sign permutahedron. Further considerations based this condition also allow to derive results on sparsity and clustering features. In particular, we define the notion of a SLOPE model to describe both sparsity and clustering properties of this method and also provide a geometric characterization of accessible SLOPE models.Comment: new title, minor change

On Modified l_1-Minimization Problems in Compressed Sensing

Sparse signal modeling has received much attention recently because of its application in medical imaging, group testing and radar technology, among others. Compressed sensing, a recently coined term, has showed us, both in theory and practice, that various signals of interest which are sparse or approximately sparse can be efficiently recovered by using far fewer samples than suggested by Shannon sampling theorem. Sparsity is the only prior information about an unknown signal assumed in traditional compressed sensing techniques. But in many applications, other kinds of prior information are also available, such as partial knowledge of the support, tree structure of signal and clustering of large coefficients around a small set of coefficients. In this thesis, we consider compressed sensing problems with prior information on the support of the signal, together with sparsity. We modify regular l_1 -minimization problems considered in compressed sensing, using this extra information. We call these modified l_1 -minimization problems. We show that partial knowledge of the support helps us to weaken sufficient conditions for the recovery of sparse signals using modified ` 1 minimization problems. In case of deterministic compressed sensing, we show that a sharp condition for sparse recovery can be improved using modified ` 1 minimization problems. We also derive algebraic necessary and sufficient condition for modified basis pursuit problem and use an open source algorithm known as l_1 -homotopy algorithm to perform some numerical experiments and compare the performance of modified Basis Pursuit Denoising with the regular Basis Pursuit Denoising