11,408 research outputs found
A Quantum Many-body Wave Function Inspired Language Modeling Approach
The recently proposed quantum language model (QLM) aimed at a principled
approach to modeling term dependency by applying the quantum probability
theory. The latest development for a more effective QLM has adopted word
embeddings as a kind of global dependency information and integrated the
quantum-inspired idea in a neural network architecture. While these
quantum-inspired LMs are theoretically more general and also practically
effective, they have two major limitations. First, they have not taken into
account the interaction among words with multiple meanings, which is common and
important in understanding natural language text. Second, the integration of
the quantum-inspired LM with the neural network was mainly for effective
training of parameters, yet lacking a theoretical foundation accounting for
such integration. To address these two issues, in this paper, we propose a
Quantum Many-body Wave Function (QMWF) inspired language modeling approach. The
QMWF inspired LM can adopt the tensor product to model the aforesaid
interaction among words. It also enables us to reveal the inherent necessity of
using Convolutional Neural Network (CNN) in QMWF language modeling.
Furthermore, our approach delivers a simple algorithm to represent and match
text/sentence pairs. Systematic evaluation shows the effectiveness of the
proposed QMWF-LM algorithm, in comparison with the state of the art
quantum-inspired LMs and a couple of CNN-based methods, on three typical
Question Answering (QA) datasets.Comment: 10 pages,4 figures,CIK
Modeling Covariate Effects in Group Independent Component Analysis with Applications to Functional Magnetic Resonance Imaging
Independent component analysis (ICA) is a powerful computational tool for
separating independent source signals from their linear mixtures. ICA has been
widely applied in neuroimaging studies to identify and characterize underlying
brain functional networks. An important goal in such studies is to assess the
effects of subjects' clinical and demographic covariates on the spatial
distributions of the functional networks. Currently, covariate effects are not
incorporated in existing group ICA decomposition methods. Hence, they can only
be evaluated through ad-hoc approaches which may not be accurate in many cases.
In this paper, we propose a hierarchical covariate ICA model that provides a
formal statistical framework for estimating and testing covariate effects in
ICA decomposition. A maximum likelihood method is proposed for estimating the
covariate ICA model. We develop two expectation-maximization (EM) algorithms to
obtain maximum likelihood estimates. The first is an exact EM algorithm, which
has analytically tractable E-step and M-step. Additionally, we propose a
subspace-based approximate EM, which can significantly reduce computational
time while still retain high model-fitting accuracy. Furthermore, to test
covariate effects on the functional networks, we develop a voxel-wise
approximate inference procedure which eliminates the needs of computationally
expensive covariance estimation. The performance of the proposed methods is
evaluated via simulation studies. The application is illustrated through an
fMRI study of Zen meditation.Comment: 36 pages, 5 figure
Exploiting Low-dimensional Structures to Enhance DNN Based Acoustic Modeling in Speech Recognition
We propose to model the acoustic space of deep neural network (DNN)
class-conditional posterior probabilities as a union of low-dimensional
subspaces. To that end, the training posteriors are used for dictionary
learning and sparse coding. Sparse representation of the test posteriors using
this dictionary enables projection to the space of training data. Relying on
the fact that the intrinsic dimensions of the posterior subspaces are indeed
very small and the matrix of all posteriors belonging to a class has a very low
rank, we demonstrate how low-dimensional structures enable further enhancement
of the posteriors and rectify the spurious errors due to mismatch conditions.
The enhanced acoustic modeling method leads to improvements in continuous
speech recognition task using hybrid DNN-HMM (hidden Markov model) framework in
both clean and noisy conditions, where upto 15.4% relative reduction in word
error rate (WER) is achieved
- …