12,935 research outputs found
Spectral Methods for Learning Multivariate Latent Tree Structure
This work considers the problem of learning the structure of multivariate
linear tree models, which include a variety of directed tree graphical models
with continuous, discrete, and mixed latent variables such as linear-Gaussian
models, hidden Markov models, Gaussian mixture models, and Markov evolutionary
trees. The setting is one where we only have samples from certain observed
variables in the tree, and our goal is to estimate the tree structure (i.e.,
the graph of how the underlying hidden variables are connected to each other
and to the observed variables). We propose the Spectral Recursive Grouping
algorithm, an efficient and simple bottom-up procedure for recovering the tree
structure from independent samples of the observed variables. Our finite sample
size bounds for exact recovery of the tree structure reveal certain natural
dependencies on underlying statistical and structural properties of the
underlying joint distribution. Furthermore, our sample complexity guarantees
have no explicit dependence on the dimensionality of the observed variables,
making the algorithm applicable to many high-dimensional settings. At the heart
of our algorithm is a spectral quartet test for determining the relative
topology of a quartet of variables from second-order statistics
Hyper-Spectral Image Analysis with Partially-Latent Regression and Spatial Markov Dependencies
Hyper-spectral data can be analyzed to recover physical properties at large
planetary scales. This involves resolving inverse problems which can be
addressed within machine learning, with the advantage that, once a relationship
between physical parameters and spectra has been established in a data-driven
fashion, the learned relationship can be used to estimate physical parameters
for new hyper-spectral observations. Within this framework, we propose a
spatially-constrained and partially-latent regression method which maps
high-dimensional inputs (hyper-spectral images) onto low-dimensional responses
(physical parameters such as the local chemical composition of the soil). The
proposed regression model comprises two key features. Firstly, it combines a
Gaussian mixture of locally-linear mappings (GLLiM) with a partially-latent
response model. While the former makes high-dimensional regression tractable,
the latter enables to deal with physical parameters that cannot be observed or,
more generally, with data contaminated by experimental artifacts that cannot be
explained with noise models. Secondly, spatial constraints are introduced in
the model through a Markov random field (MRF) prior which provides a spatial
structure to the Gaussian-mixture hidden variables. Experiments conducted on a
database composed of remotely sensed observations collected from the Mars
planet by the Mars Express orbiter demonstrate the effectiveness of the
proposed model.Comment: 12 pages, 4 figures, 3 table
A Method of Moments for Mixture Models and Hidden Markov Models
Mixture models are a fundamental tool in applied statistics and machine
learning for treating data taken from multiple subpopulations. The current
practice for estimating the parameters of such models relies on local search
heuristics (e.g., the EM algorithm) which are prone to failure, and existing
consistent methods are unfavorable due to their high computational and sample
complexity which typically scale exponentially with the number of mixture
components. This work develops an efficient method of moments approach to
parameter estimation for a broad class of high-dimensional mixture models with
many components, including multi-view mixtures of Gaussians (such as mixtures
of axis-aligned Gaussians) and hidden Markov models. The new method leads to
rigorous unsupervised learning results for mixture models that were not
achieved by previous works; and, because of its simplicity, it offers a viable
alternative to EM for practical deployment
Spectral Sequence Motif Discovery
Sequence discovery tools play a central role in several fields of
computational biology. In the framework of Transcription Factor binding
studies, motif finding algorithms of increasingly high performance are required
to process the big datasets produced by new high-throughput sequencing
technologies. Most existing algorithms are computationally demanding and often
cannot support the large size of new experimental data. We present a new motif
discovery algorithm that is built on a recent machine learning technique,
referred to as Method of Moments. Based on spectral decompositions, this method
is robust under model misspecification and is not prone to locally optimal
solutions. We obtain an algorithm that is extremely fast and designed for the
analysis of big sequencing data. In a few minutes, we can process datasets of
hundreds of thousand sequences and extract motif profiles that match those
computed by various state-of-the-art algorithms.Comment: 20 pages, 3 figures, 1 tabl
- …