58 research outputs found
Inference in Hidden Markov Models with Explicit State Duration Distributions
In this letter we borrow from the inference techniques developed for
unbounded state-cardinality (nonparametric) variants of the HMM and use them to
develop a tuning-parameter free, black-box inference procedure for
Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have
latent states consisting of both discrete state-indicator and discrete
state-duration random variables. In contrast to the implicit geometric state
duration distribution possessed by the standard HMM, EDHMMs allow the direct
parameterisation and estimation of per-state duration distributions. As most
duration distributions are defined over the positive integers, truncation or
other approximations are usually required to perform EDHMM inference
Stochastic Collapsed Variational Inference for Sequential Data
Stochastic variational inference for collapsed models has recently been
successfully applied to large scale topic modelling. In this paper, we propose
a stochastic collapsed variational inference algorithm in the sequential data
setting. Our algorithm is applicable to both finite hidden Markov models and
hierarchical Dirichlet process hidden Markov models, and to any datasets
generated by emission distributions in the exponential family. Our experiment
results on two discrete datasets show that our inference is both more efficient
and more accurate than its uncollapsed version, stochastic variational
inference.Comment: NIPS Workshop on Advances in Approximate Bayesian Inference, 201
Learning Tree Distributions by Hidden Markov Models
Hidden tree Markov models allow learning distributions for tree structured
data while being interpretable as nondeterministic automata. We provide a
concise summary of the main approaches in literature, focusing in particular on
the causality assumptions introduced by the choice of a specific tree visit
direction. We will then sketch a novel non-parametric generalization of the
bottom-up hidden tree Markov model with its interpretation as a
nondeterministic tree automaton with infinite states.Comment: Accepted in LearnAut2018 worksho
Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection
Speech recognition systems have achieved high recognition performance for
several tasks. However, the performance of such systems is dependent on the
tremendously costly development work of preparing vast amounts of task-matched
transcribed speech data for supervised training. The key problem here is the
cost of transcribing speech data. The cost is repeatedly required to support
new languages and new tasks. Assuming broad network services for transcribing
speech data for many users, a system would become more self-sufficient and more
useful if it possessed the ability to learn from very light feedback from the
users without annoying them. In this paper, we propose a general reinforcement
learning framework for speech recognition systems based on the policy gradient
method. As a particular instance of the framework, we also propose a hypothesis
selection-based reinforcement learning method. The proposed framework provides
a new view for several existing training and adaptation methods. The
experimental results show that the proposed method improves the recognition
performance compared to unsupervised adaptation.Comment: 5 pages, 6 figure
A reversible infinite HMM using normalised random measures
We present a nonparametric prior over reversible Markov chains. We use
completely random measures, specifically gamma processes, to construct a
countably infinite graph with weighted edges. By enforcing symmetry to make the
edges undirected we define a prior over random walks on graphs that results in
a reversible Markov chain. The resulting prior over infinite transition
matrices is closely related to the hierarchical Dirichlet process but enforces
reversibility. A reinforcement scheme has recently been proposed with similar
properties, but the de Finetti measure is not well characterised. We take the
alternative approach of explicitly constructing the mixing measure, which
allows more straightforward and efficient inference at the cost of no longer
having a closed form predictive distribution. We use our process to construct a
reversible infinite HMM which we apply to two real datasets, one from
epigenomics and one ion channel recording.Comment: 9 pages, 6 figure
A Nonparametric Bayesian Approach to Uncovering Rat Hippocampal Population Codes During Spatial Navigation
Rodent hippocampal population codes represent important spatial information
about the environment during navigation. Several computational methods have
been developed to uncover the neural representation of spatial topology
embedded in rodent hippocampal ensemble spike activity. Here we extend our
previous work and propose a nonparametric Bayesian approach to infer rat
hippocampal population codes during spatial navigation. To tackle the model
selection problem, we leverage a nonparametric Bayesian model. Specifically, to
analyze rat hippocampal ensemble spiking activity, we apply a hierarchical
Dirichlet process-hidden Markov model (HDP-HMM) using two Bayesian inference
methods, one based on Markov chain Monte Carlo (MCMC) and the other based on
variational Bayes (VB). We demonstrate the effectiveness of our Bayesian
approaches on recordings from a freely-behaving rat navigating in an open field
environment. We find that MCMC-based inference with Hamiltonian Monte Carlo
(HMC) hyperparameter sampling is flexible and efficient, and outperforms VB and
MCMC approaches with hyperparameters set by empirical Bayes
- …