2,481 research outputs found
Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models
Conventional deep neural networks (DNN) for speech acoustic modeling rely on
Gaussian mixture models (GMM) and hidden Markov model (HMM) to obtain binary
class labels as the targets for DNN training. Subword classes in speech
recognition systems correspond to context-dependent tied states or senones. The
present work addresses some limitations of GMM-HMM senone alignments for DNN
training. We hypothesize that the senone probabilities obtained from a DNN
trained with binary labels can provide more accurate targets to learn better
acoustic models. However, DNN outputs bear inaccuracies which are exhibited as
high dimensional unstructured noise, whereas the informative components are
structured and low-dimensional. We exploit principle component analysis (PCA)
and sparse coding to characterize the senone subspaces. Enhanced probabilities
obtained from low-rank and sparse reconstructions are used as soft-targets for
DNN acoustic modeling, that also enables training with untranscribed data.
Experiments conducted on AMI corpus shows 4.6% relative reduction in word error
rate
Dissecting high-dimensional phenotypes with bayesian sparse factor analysis of genetic covariance matrices.
Quantitative genetic studies that model complex, multivariate phenotypes are important for both evolutionary prediction and artificial selection. For example, changes in gene expression can provide insight into developmental and physiological mechanisms that link genotype and phenotype. However, classical analytical techniques are poorly suited to quantitative genetic studies of gene expression where the number of traits assayed per individual can reach many thousand. Here, we derive a Bayesian genetic sparse factor model for estimating the genetic covariance matrix (G-matrix) of high-dimensional traits, such as gene expression, in a mixed-effects model. The key idea of our model is that we need consider only G-matrices that are biologically plausible. An organism's entire phenotype is the result of processes that are modular and have limited complexity. This implies that the G-matrix will be highly structured. In particular, we assume that a limited number of intermediate traits (or factors, e.g., variations in development or physiology) control the variation in the high-dimensional phenotype, and that each of these intermediate traits is sparse - affecting only a few observed traits. The advantages of this approach are twofold. First, sparse factors are interpretable and provide biological insight into mechanisms underlying the genetic architecture. Second, enforcing sparsity helps prevent sampling errors from swamping out the true signal in high-dimensional data. We demonstrate the advantages of our model on simulated data and in an analysis of a published Drosophila melanogaster gene expression data set
Bayesian Sparse Factor Analysis of Genetic Covariance Matrices
Quantitative genetic studies that model complex, multivariate phenotypes are
important for both evolutionary prediction and artificial selection. For
example, changes in gene expression can provide insight into developmental and
physiological mechanisms that link genotype and phenotype. However, classical
analytical techniques are poorly suited to quantitative genetic studies of gene
expression where the number of traits assayed per individual can reach many
thousand. Here, we derive a Bayesian genetic sparse factor model for estimating
the genetic covariance matrix (G-matrix) of high-dimensional traits, such as
gene expression, in a mixed effects model. The key idea of our model is that we
need only consider G-matrices that are biologically plausible. An organism's
entire phenotype is the result of processes that are modular and have limited
complexity. This implies that the G-matrix will be highly structured. In
particular, we assume that a limited number of intermediate traits (or factors,
e.g., variations in development or physiology) control the variation in the
high-dimensional phenotype, and that each of these intermediate traits is
sparse -- affecting only a few observed traits. The advantages of this approach
are two-fold. First, sparse factors are interpretable and provide biological
insight into mechanisms underlying the genetic architecture. Second, enforcing
sparsity helps prevent sampling errors from swamping out the true signal in
high-dimensional data. We demonstrate the advantages of our model on simulated
data and in an analysis of a published Drosophila melanogaster gene expression
data set.Comment: 35 pages, 7 figure
Multi-view Learning as a Nonparametric Nonlinear Inter-Battery Factor Analysis
Factor analysis aims to determine latent factors, or traits, which summarize
a given data set. Inter-battery factor analysis extends this notion to multiple
views of the data. In this paper we show how a nonlinear, nonparametric version
of these models can be recovered through the Gaussian process latent variable
model. This gives us a flexible formalism for multi-view learning where the
latent variables can be used both for exploratory purposes and for learning
representations that enable efficient inference for ambiguous estimation tasks.
Learning is performed in a Bayesian manner through the formulation of a
variational compression scheme which gives a rigorous lower bound on the log
likelihood. Our Bayesian framework provides strong regularization during
training, allowing the structure of the latent space to be determined
efficiently and automatically. We demonstrate this by producing the first (to
our knowledge) published results of learning from dozens of views, even when
data is scarce. We further show experimental results on several different types
of multi-view data sets and for different kinds of tasks, including exploratory
data analysis, generation, ambiguity modelling through latent priors and
classification.Comment: 49 pages including appendi
Maximally Divergent Intervals for Anomaly Detection
We present new methods for batch anomaly detection in multivariate time
series. Our methods are based on maximizing the Kullback-Leibler divergence
between the data distribution within and outside an interval of the time
series. An empirical analysis shows the benefits of our algorithms compared to
methods that treat each time step independently from each other without
optimizing with respect to all possible intervals.Comment: ICML Workshop on Anomaly Detectio
Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model
Multi-label classification is the challenging task of predicting the presence
and absence of multiple targets, involving representation learning and label
correlation modeling. We propose a novel framework for multi-label
classification, Multivariate Probit Variational AutoEncoder (MPVAE), that
effectively learns latent embedding spaces as well as label correlations. MPVAE
learns and aligns two probabilistic embedding spaces for labels and features
respectively. The decoder of MPVAE takes in the samples from the embedding
spaces and models the joint distribution of output targets under a Multivariate
Probit model by learning a shared covariance matrix. We show that MPVAE
outperforms the existing state-of-the-art methods on a variety of application
domains, using public real-world datasets. MPVAE is further shown to remain
robust under noisy settings. Lastly, we demonstrate the interpretability of the
learned covariance by a case study on a bird observation dataset
- …