919 research outputs found
Homomorphic Sensing of Subspace Arrangements
Homomorphic sensing is a recent algebraic-geometric framework that studies
the unique recovery of points in a linear subspace from their images under a
given collection of linear maps. It has been successful in interpreting such a
recovery in the case of permutations composed by coordinate projections, an
important instance in applications known as unlabeled sensing, which models
data that are out of order and have missing values. In this paper, we provide
tighter and simpler conditions that guarantee the unique recovery for the
single-subspace case, extend the result to the case of a subspace arrangement,
and show that the unique recovery in a single subspace is locally stable under
noise. We specialize our results to several examples of homomorphic sensing
such as real phase retrieval and unlabeled sensing. In so doing, in a unified
way, we obtain conditions that guarantee the unique recovery for those
examples, typically known via diverse techniques in the literature, as well as
novel conditions for sparse and unsigned versions of unlabeled sensing.
Similarly, our noise result also implies that the unique recovery in unlabeled
sensing is locally stable.Comment: 18 page
Recommended from our members
Mixture Models in Machine Learning
Modeling with mixtures is a powerful method in the statistical toolkit that can be used for representing the presence of sub-populations within an overall population. In many applications ranging from financial models to genetics, a mixture model is used to fit the data. The primary difficulty in learning mixture models is that the observed data set does not identify the sub-population to which an individual observation belongs. Despite being studied for more than a century, the theoretical guarantees of mixture models remain unknown for several important settings.
In this thesis, we look at three groups of problems. The first part is aimed at estimating the parameters of a mixture of simple distributions. We ask the following question: How many samples are necessary and sufficient to learn the latent parameters? We propose several approaches for this problem that include complex analytic tools to connect statistical distances between pairs of mixtures with the characteristic function. We show sufficient sample complexity guarantees for mixtures of popular distributions (including Gaussian, Poisson and Geometric). For many distributions, our results provide the first sample complexity guarantees for parameter estimation in the corresponding mixture. Using these techniques, we also provide improved lower bounds on the Total Variation distance between Gaussian mixtures with two components and demonstrate new results in some sequence reconstruction problems.
In the second part, we study Mixtures of Sparse Linear Regressions where the goal is to learn the best set of linear relationships between the scalar responses (i.e., labels) and the explanatory variables (i.e., features). We focus on a scenario where a learner is able to choose the features to get the labels. To tackle the high dimensionality of data, we further assume that the linear maps are also sparse , i.e., have only few prominent features among many. For this setting, we devise algorithms with sub-linear (as a function of the dimension) sample complexity guarantees that are also robust to noise.
In the final part, we study Mixtures of Sparse Linear Classifiers in the same setting as above. Given a set of features and the binary labels, the objective of this task is to find a set of hyperplanes in the space of features such that for any (feature, label) pair, there exists a hyperplane in the set that justifies the mapping. We devise efficient algorithms with sub-linear sample complexity guarantees for learning the unknown hyperplanes under similar sparsity assumptions as above. To that end, we propose several novel techniques that include tensor decomposition methods and combinatorial designs
- …