8,124 research outputs found
Sketching for Large-Scale Learning of Mixture Models
Learning parameters from voluminous data can be prohibitive in terms of
memory and computational requirements. We propose a "compressive learning"
framework where we estimate model parameters from a sketch of the training
data. This sketch is a collection of generalized moments of the underlying
probability distribution of the data. It can be computed in a single pass on
the training set, and is easily computable on streams or distributed datasets.
The proposed framework shares similarities with compressive sensing, which aims
at drastically reducing the dimension of high-dimensional signals while
preserving the ability to reconstruct them. To perform the estimation task, we
derive an iterative algorithm analogous to sparse reconstruction algorithms in
the context of linear inverse problems. We exemplify our framework with the
compressive estimation of a Gaussian Mixture Model (GMM), providing heuristics
on the choice of the sketching procedure and theoretical guarantees of
reconstruction. We experimentally show on synthetic data that the proposed
algorithm yields results comparable to the classical Expectation-Maximization
(EM) technique while requiring significantly less memory and fewer computations
when the number of database elements is large. We further demonstrate the
potential of the approach on real large-scale data (over 10 8 training samples)
for the task of model-based speaker verification. Finally, we draw some
connections between the proposed framework and approximate Hilbert space
embedding of probability distributions using random features. We show that the
proposed sketching operator can be seen as an innovative method to design
translation-invariant kernels adapted to the analysis of GMMs. We also use this
theoretical framework to derive information preservation guarantees, in the
spirit of infinite-dimensional compressive sensing
Adaptive Resonance Theory
SyNAPSE program of the Defense Advanced Projects Research Agency (Hewlett-Packard Company, subcontract under DARPA prime contract HR0011-09-3-0001, and HRL Laboratories LLC, subcontract #801881-BS under DARPA prime contract HR0011-09-C-0001); CELEST, an NSF Science of Learning Center (SBE-0354378
A Multiple Hypothesis Testing Approach to Low-Complexity Subspace Unmixing
Subspace-based signal processing traditionally focuses on problems involving
a few subspaces. Recently, a number of problems in different application areas
have emerged that involve a significantly larger number of subspaces relative
to the ambient dimension. It becomes imperative in such settings to first
identify a smaller set of active subspaces that contribute to the observation
before further processing can be carried out. This problem of identification of
a small set of active subspaces among a huge collection of subspaces from a
single (noisy) observation in the ambient space is termed subspace unmixing.
This paper formally poses the subspace unmixing problem under the parsimonious
subspace-sum (PS3) model, discusses connections of the PS3 model to problems in
wireless communications, hyperspectral imaging, high-dimensional statistics and
compressed sensing, and proposes a low-complexity algorithm, termed marginal
subspace detection (MSD), for subspace unmixing. The MSD algorithm turns the
subspace unmixing problem for the PS3 model into a multiple hypothesis testing
(MHT) problem and its analysis in the paper helps control the family-wise error
rate of this MHT problem at any level under two random
signal generation models. Some other highlights of the analysis of the MSD
algorithm include: (i) it is applicable to an arbitrary collection of subspaces
on the Grassmann manifold; (ii) it relies on properties of the collection of
subspaces that are computable in polynomial time; and () it allows for
linear scaling of the number of active subspaces as a function of the ambient
dimension. Finally, numerical results are presented in the paper to better
understand the performance of the MSD algorithm.Comment: Submitted for journal publication; 33 pages, 14 figure
Multiband Spectrum Access: Great Promises for Future Cognitive Radio Networks
Cognitive radio has been widely considered as one of the prominent solutions
to tackle the spectrum scarcity. While the majority of existing research has
focused on single-band cognitive radio, multiband cognitive radio represents
great promises towards implementing efficient cognitive networks compared to
single-based networks. Multiband cognitive radio networks (MB-CRNs) are
expected to significantly enhance the network's throughput and provide better
channel maintenance by reducing handoff frequency. Nevertheless, the wideband
front-end and the multiband spectrum access impose a number of challenges yet
to overcome. This paper provides an in-depth analysis on the recent
advancements in multiband spectrum sensing techniques, their limitations, and
possible future directions to improve them. We study cooperative communications
for MB-CRNs to tackle a fundamental limit on diversity and sampling. We also
investigate several limits and tradeoffs of various design parameters for
MB-CRNs. In addition, we explore the key MB-CRNs performance metrics that
differ from the conventional metrics used for single-band based networks.Comment: 22 pages, 13 figures; published in the Proceedings of the IEEE
Journal, Special Issue on Future Radio Spectrum Access, March 201
- âŠ