On the solution uniqueness characterization in the L1 norm and polyhedral gauge recovery

Paper to appear.International audienceThis paper first proposes another proof of the necessary and sufficient conditions of solution uniqueness in 1-norm minimization given recently by H. Zhang, W. Yin, and L. Cheng. The analysis avoids the need of the surjectivity assumption made by these authors and should be mainly appealing by its short length (it can therefore be proposed to students exercising in convex optimization). In the second part of the paper, the previous existence and uniqueness characterization is extended to the recovery problem where the ℓ 1 norm is substituted by a polyhedral gauge. In addition to present interest for a number of practical problems, this extension clarifies the geometrical aspect of the previous uniqueness characterization. Numerical techniques are proposed to compute a solution to the polyhedral gauge recovery problem in polynomial time and to check its possible uniqueness by a simple linear algebra test

RSP-Based Analysis for Sparsest and Least ℓ1\ell_1-Norm Solutions to Underdetermined Linear Systems

Recently, the worse-case analysis, probabilistic analysis and empirical justification have been employed to address the fundamental question: When does $\ell_1$-minimization find the sparsest solution to an underdetermined linear system? In this paper, a deterministic analysis, rooted in the classic linear programming theory, is carried out to further address this question. We first identify a necessary and sufficient condition for the uniqueness of least $\ell_1$-norm solutions to linear systems. From this condition, we deduce that a sparsest solution coincides with the unique least $\ell_1$-norm solution to a linear system if and only if the so-called \emph{range space property} (RSP) holds at this solution. This yields a broad understanding of the relationship between $\ell_0$- and $\ell_1$-minimization problems. Our analysis indicates that the RSP truly lies at the heart of the relationship between these two problems. Through RSP-based analysis, several important questions in this field can be largely addressed. For instance, how to efficiently interpret the gap between the current theory and the actual numerical performance of $\ell_1$-minimization by a deterministic analysis, and if a linear system has multiple sparsest solutions, when does $\ell_1$-minimization guarantee to find one of them? Moreover, new matrix properties (such as the \emph{RSP of order $K$} and the \emph{Weak-RSP of order $K$}) are introduced in this paper, and a new theory for sparse signal recovery based on the RSP of order $K$ is established

The Lasso Problem and Uniqueness

The lasso is a popular tool for sparse linear regression, especially for problems in which the number of variables p exceeds the number of observations n. But when p>n, the lasso criterion is not strictly convex, and hence it may not have a unique minimum. An important question is: when is the lasso solution well-defined (unique)? We review results from the literature, which show that if the predictor variables are drawn from a continuous probability distribution, then there is a unique lasso solution with probability one, regardless of the sizes of n and p. We also show that this result extends easily to $\ell_1$ penalized minimization problems over a wide range of loss functions. A second important question is: how can we deal with the case of non-uniqueness in lasso solutions? In light of the aforementioned result, this case really only arises when some of the predictor variables are discrete, or when some post-processing has been performed on continuous predictor measurements. Though we certainly cannot claim to provide a complete answer to such a broad question, we do present progress towards understanding some aspects of non-uniqueness. First, we extend the LARS algorithm for computing the lasso solution path to cover the non-unique case, so that this path algorithm works for any predictor matrix. Next, we derive a simple method for computing the component-wise uncertainty in lasso solutions of any given problem instance, based on linear programming. Finally, we review results from the literature on some of the unifying properties of lasso solutions, and also point out particular forms of solutions that have distinctive properties.Comment: 25 pages, 0 figure

A Unique "Nonnegative" Solution to an Underdetermined System: from Vectors to Matrices

This paper investigates the uniqueness of a nonnegative vector solution and the uniqueness of a positive semidefinite matrix solution to underdetermined linear systems. A vector solution is the unique solution to an underdetermined linear system only if the measurement matrix has a row-span intersecting the positive orthant. Focusing on two types of binary measurement matrices, Bernoulli 0-1 matrices and adjacency matrices of general expander graphs, we show that, in both cases, the support size of a unique nonnegative solution can grow linearly, namely O(n), with the problem dimension n. We also provide closed-form characterizations of the ratio of this support size to the signal dimension. For the matrix case, we show that under a necessary and sufficient condition for the linear compressed observations operator, there will be a unique positive semidefinite matrix solution to the compressed linear observations. We further show that a randomly generated Gaussian linear compressed observations operator will satisfy this condition with overwhelmingly high probability