8,430 research outputs found
Dense Error Correction via L1-Minimization
This paper studies the problem of recovering a non-negative sparse signal \x
\in \Re^n from highly corrupted linear measurements \y = A\x + \e \in \Re^m,
where \e is an unknown error vector whose nonzero entries may be unbounded.
Motivated by an observation from face recognition in computer vision, this
paper proves that for highly correlated (and possibly overcomplete)
dictionaries , any non-negative, sufficiently sparse signal \x can be
recovered by solving an -minimization problem: \min \|\x\|_1 +
\|\e\|_1 \quad {subject to} \quad \y = A\x + \e. More precisely, if the
fraction of errors is bounded away from one and the support of \x
grows sublinearly in the dimension of the observation, then as goes to
infinity, the above -minimization succeeds for all signals \x and
almost all sign-and-support patterns of \e. This result suggests that
accurate recovery of sparse signals is possible and computationally feasible
even with nearly 100% of the observations corrupted. The proof relies on a
careful characterization of the faces of a convex polytope spanned together by
the standard crosspolytope and a set of iid Gaussian vectors with nonzero mean
and small variance, which we call the ``cross-and-bouquet'' model. Simulations
and experimental results corroborate the findings, and suggest extensions to
the result.Comment: 40 pages, 9 figure
Worst Configurations (Instantons) for Compressed Sensing over Reals: a Channel Coding Approach
We consider the Linear Programming (LP) solution of the Compressed Sensing
(CS) problem over reals, also known as the Basis Pursuit (BasP) algorithm. The
BasP allows interpretation as a channel-coding problem, and it guarantees
error-free reconstruction with a properly chosen measurement matrix and
sufficiently sparse error vectors. In this manuscript, we examine how the BasP
performs on a given measurement matrix and develop an algorithm to discover the
sparsest vectors for which the BasP fails. The resulting algorithm is a
generalization of our previous results on finding the most probable
error-patterns degrading performance of a finite size Low-Density Parity-Check
(LDPC) code in the error-floor regime. The BasP fails when its output is
different from the actual error-pattern. We design a CS-Instanton Search
Algorithm (ISA) generating a sparse vector, called a CS-instanton, such that
the BasP fails on the CS-instanton, while the BasP recovery is successful for
any modification of the CS-instanton replacing a nonzero element by zero. We
also prove that, given a sufficiently dense random input for the error-vector,
the CS-ISA converges to an instanton in a small finite number of steps. The
performance of the CS-ISA is illustrated on a randomly generated matrix. For this example, the CS-ISA outputs the shortest instanton (error
vector) pattern of length 11.Comment: Accepted to be presented at the IEEE International Symposium on
Information Theory (ISIT 2010). 5 pages, 2 Figures. Minor edits from previous
version. Added a new reference
Robust Regression via Hard Thresholding
We study the problem of Robust Least Squares Regression (RLSR) where several
response variables can be adversarially corrupted. More specifically, for a
data matrix X \in R^{p x n} and an underlying model w*, the response vector is
generated as y = X'w* + b where b \in R^n is the corruption vector supported
over at most C.n coordinates. Existing exact recovery results for RLSR focus
solely on L1-penalty based convex formulations and impose relatively strict
model assumptions such as requiring the corruptions b to be selected
independently of X.
In this work, we study a simple hard-thresholding algorithm called TORRENT
which, under mild conditions on X, can recover w* exactly even if b corrupts
the response variables in an adversarial manner, i.e. both the support and
entries of b are selected adversarially after observing X and w*. Our results
hold under deterministic assumptions which are satisfied if X is sampled from
any sub-Gaussian distribution. Finally unlike existing results that apply only
to a fixed w*, generated independently of X, our results are universal and hold
for any w* \in R^p.
Next, we propose gradient descent-based extensions of TORRENT that can scale
efficiently to large scale problems, such as high dimensional sparse recovery
and prove similar recovery guarantees for these extensions. Empirically we find
TORRENT, and more so its extensions, offering significantly faster recovery
than the state-of-the-art L1 solvers. For instance, even on moderate-sized
datasets (with p = 50K) with around 40% corrupted responses, a variant of our
proposed method called TORRENT-HYB is more than 20x faster than the best L1
solver.Comment: 24 pages, 3 figure
- β¦