4,417 research outputs found
Improving A*OMP: Theoretical and Empirical Analyses With a Novel Dynamic Cost Model
Best-first search has been recently utilized for compressed sensing (CS) by
the A* orthogonal matching pursuit (A*OMP) algorithm. In this work, we
concentrate on theoretical and empirical analyses of A*OMP. We present a
restricted isometry property (RIP) based general condition for exact recovery
of sparse signals via A*OMP. In addition, we develop online guarantees which
promise improved recovery performance with the residue-based termination
instead of the sparsity-based one. We demonstrate the recovery capabilities of
A*OMP with extensive recovery simulations using the adaptive-multiplicative
(AMul) cost model, which effectively compensates for the path length
differences in the search tree. The presented results, involving phase
transitions for different nonzero element distributions as well as recovery
rates and average error, reveal not only the superior recovery accuracy of
A*OMP, but also the improvements with the residue-based termination and the
AMul cost model. Comparison of the run times indicate the speed up by the AMul
cost model. We also demonstrate a hybrid of OMP and A?OMP to accelerate the
search further. Finally, we run A*OMP on a sparse image to illustrate its
recovery performance for more realistic coefcient distributions
Ultra-high Dimensional Multiple Output Learning With Simultaneous Orthogonal Matching Pursuit: A Sure Screening Approach
We propose a novel application of the Simultaneous Orthogonal Matching
Pursuit (S-OMP) procedure for sparsistant variable selection in ultra-high
dimensional multi-task regression problems. Screening of variables, as
introduced in \cite{fan08sis}, is an efficient and highly scalable way to
remove many irrelevant variables from the set of all variables, while retaining
all the relevant variables. S-OMP can be applied to problems with hundreds of
thousands of variables and once the number of variables is reduced to a
manageable size, a more computationally demanding procedure can be used to
identify the relevant variables for each of the regression outputs. To our
knowledge, this is the first attempt to utilize relatedness of multiple outputs
to perform fast screening of relevant variables. As our main theoretical
contribution, we prove that, asymptotically, S-OMP is guaranteed to reduce an
ultra-high number of variables to below the sample size without losing true
relevant variables. We also provide formal evidence that a modified Bayesian
information criterion (BIC) can be used to efficiently determine the number of
iterations in S-OMP. We further provide empirical evidence on the benefit of
variable selection using multiple regression outputs jointly, as opposed to
performing variable selection for each output separately. The finite sample
performance of S-OMP is demonstrated on extensive simulation studies, and on a
genetic association mapping problem. Adaptive Lasso; Greedy forward
regression; Orthogonal matching pursuit; Multi-output regression; Multi-task
learning; Simultaneous orthogonal matching pursuit; Sure screening; Variable
selectio
Self-Dictionary Sparse Regression for Hyperspectral Unmixing: Greedy Pursuit and Pure Pixel Search are Related
This paper considers a recently emerged hyperspectral unmixing formulation
based on sparse regression of a self-dictionary multiple measurement vector
(SD-MMV) model, wherein the measured hyperspectral pixels are used as the
dictionary. Operating under the pure pixel assumption, this SD-MMV formalism is
special in that it allows simultaneous identification of the endmember spectral
signatures and the number of endmembers. Previous SD-MMV studies mainly focus
on convex relaxations. In this study, we explore the alternative of greedy
pursuit, which generally provides efficient and simple algorithms. In
particular, we design a greedy SD-MMV algorithm using simultaneous orthogonal
matching pursuit. Intriguingly, the proposed greedy algorithm is shown to be
closely related to some existing pure pixel search algorithms, especially, the
successive projection algorithm (SPA). Thus, a link between SD-MMV and pure
pixel search is revealed. We then perform exact recovery analyses, and prove
that the proposed greedy algorithm is robust to noise---including its
identification of the (unknown) number of endmembers---under a sufficiently low
noise level. The identification performance of the proposed greedy algorithm is
demonstrated through both synthetic and real-data experiments
- …