7 research outputs found
Conditioning of Random Block Subdictionaries with Applications to Block-Sparse Recovery and Regression
The linear model, in which a set of observations is assumed to be given by a
linear combination of columns of a matrix, has long been the mainstay of the
statistics and signal processing literature. One particular challenge for
inference under linear models is understanding the conditions on the dictionary
under which reliable inference is possible. This challenge has attracted
renewed attention in recent years since many modern inference problems deal
with the "underdetermined" setting, in which the number of observations is much
smaller than the number of columns in the dictionary. This paper makes several
contributions for this setting when the set of observations is given by a
linear combination of a small number of groups of columns of the dictionary,
termed the "block-sparse" case. First, it specifies conditions on the
dictionary under which most block subdictionaries are well conditioned. This
result is fundamentally different from prior work on block-sparse inference
because (i) it provides conditions that can be explicitly computed in
polynomial time, (ii) the given conditions translate into near-optimal scaling
of the number of columns of the block subdictionaries as a function of the
number of observations for a large class of dictionaries, and (iii) it suggests
that the spectral norm and the quadratic-mean block coherence of the dictionary
(rather than the worst-case coherences) fundamentally limit the scaling of
dimensions of the well-conditioned block subdictionaries. Second, this paper
investigates the problems of block-sparse recovery and block-sparse regression
in underdetermined settings. Near-optimal block-sparse recovery and regression
are possible for certain dictionaries as long as the dictionary satisfies
easily computable conditions and the coefficients describing the linear
combination of groups of columns can be modeled through a mild statistical
prior.Comment: 39 pages, 3 figures. A revised and expanded version of the paper
published in IEEE Transactions on Information Theory (DOI:
10.1109/TIT.2015.2429632); this revision includes corrections in the proofs
of some of the result
A Multiple Hypothesis Testing Approach to Low-Complexity Subspace Unmixing
Subspace-based signal processing traditionally focuses on problems involving
a few subspaces. Recently, a number of problems in different application areas
have emerged that involve a significantly larger number of subspaces relative
to the ambient dimension. It becomes imperative in such settings to first
identify a smaller set of active subspaces that contribute to the observation
before further processing can be carried out. This problem of identification of
a small set of active subspaces among a huge collection of subspaces from a
single (noisy) observation in the ambient space is termed subspace unmixing.
This paper formally poses the subspace unmixing problem under the parsimonious
subspace-sum (PS3) model, discusses connections of the PS3 model to problems in
wireless communications, hyperspectral imaging, high-dimensional statistics and
compressed sensing, and proposes a low-complexity algorithm, termed marginal
subspace detection (MSD), for subspace unmixing. The MSD algorithm turns the
subspace unmixing problem for the PS3 model into a multiple hypothesis testing
(MHT) problem and its analysis in the paper helps control the family-wise error
rate of this MHT problem at any level under two random
signal generation models. Some other highlights of the analysis of the MSD
algorithm include: (i) it is applicable to an arbitrary collection of subspaces
on the Grassmann manifold; (ii) it relies on properties of the collection of
subspaces that are computable in polynomial time; and () it allows for
linear scaling of the number of active subspaces as a function of the ambient
dimension. Finally, numerical results are presented in the paper to better
understand the performance of the MSD algorithm.Comment: Submitted for journal publication; 33 pages, 14 figure
A Computational and Statistical Study of Convex and Nonconvex Optimization with Applications to Structured Source Demixing and Matrix Factorization Problems
University of Minnesota Ph.D. dissertation. September 2017. Major: Electrical/Computer Engineering. Advisor: Jarvis Haupt. 1 computer file (PDF); ix, 153 pages.Modern machine learning problems that emerge from real-world applications typically involve estimating high dimensional model parameters, whose number may be of the same order as or even significantly larger than the number of measurements. In such high dimensional settings, statistically-consistent estimation of true underlying models via classical approaches is often impossible, due to the lack of identifiability. A recent solution to this issue is through incorporating regularization functions into estimation procedures to promote intrinsic low-complexity structure of the underlying models. Statistical studies have established successful recovery of model parameters via structure-exploiting regularized estimators and computational efforts have examined efficient numerical procedures to accurately solve the associated optimization problems. In this dissertation, we study the statistical and computational aspects of some regularized estimators that are successful in reconstructing high dimensional models. The investigated estimation frameworks are motivated by their applications in different areas of engineering, such as structural health monitoring and recommendation systems. In particular, the group Lasso recovery guarantees provided in Chapter 2 will bring insight into the application of this estimator for localizing material defects in the context of a structural diagnostics problem. Chapter 3 describes the convergence study of an accelerated variant of the well-known alternating direction method of multipliers (ADMM) for minimizing strongly convex functions. The analysis is followed by several experimental evidence into the algorithm's applicability to a ranking problem. Finally, Chapter 4 presents a local convergence analysis of regularized factorization-based estimators for reconstructing low-rank matrices. Interestingly, the analysis of this chapter reveals the interplay between statistical and computational aspects of such (non-convex) estimators. Therefore, it can be useful in a wide variety of problems that involve low-rank matrix estimation