4 research outputs found
A Spectral Algorithm for Learning Hidden Markov Models
Hidden Markov Models (HMMs) are one of the most fundamental and widely used
statistical tools for modeling discrete time series. In general, learning HMMs
from data is computationally hard (under cryptographic assumptions), and
practitioners typically resort to search heuristics which suffer from the usual
local optima issues. We prove that under a natural separation condition (bounds
on the smallest singular value of the HMM parameters), there is an efficient
and provably correct algorithm for learning HMMs. The sample complexity of the
algorithm does not explicitly depend on the number of distinct (discrete)
observations---it implicitly depends on this quantity through spectral
properties of the underlying HMM. This makes the algorithm particularly
applicable to settings with a large number of observations, such as those in
natural language processing where the space of observation is sometimes the
words in a language. The algorithm is also simple, employing only a singular
value decomposition and matrix multiplications.Comment: Published in JCSS Special Issue "Learning Theory 2009
Consistent estimation of the filtering and marginal smoothing distributions in nonparametric hidden Markov models
In this paper, we consider the filtering and smoothing recursions in
nonparametric finite state space hidden Markov models (HMMs) when the
parameters of the model are unknown and replaced by estimators. We provide an
explicit and time uniform control of the filtering and smoothing errors in
total variation norm as a function of the parameter estimation errors. We prove
that the risk for the filtering and smoothing errors may be uniformly upper
bounded by the risk of the estimators. It has been proved very recently that
statistical inference for finite state space nonparametric HMMs is possible. We
study how the recent spectral methods developed in the parametric setting may
be extended to the nonparametric framework and we give explicit upper bounds
for the L2-risk of the nonparametric spectral estimators. When the observation
space is compact, this provides explicit rates for the filtering and smoothing
errors in total variation norm. The performance of the spectral method is
assessed with simulated data for both the estimation of the (nonparametric)
conditional distribution of the observations and the estimation of the marginal
smoothing distributions.Comment: 27 pages, 2 figures. arXiv admin note: text overlap with
arXiv:1501.0478
The Value of Observation for Monitoring Dynamic Systems
We consider the fundamental problem of monitoring (i.e. tracking) the belief state in a dynamic system, when the model is only approximately correct and when the initial belief state might be unknown. In this general setting where the model is (perhaps only slightly) mis-specified, monitoring (and consequently planning) may be impossible as errors might accumulate over time. We provide a new characterization, the value of observation, which allows us to bound the error accumulation. The value of observation is a parameter that governs how much information the observation provides. For instance, in Partially Observable MDPs when it is 1 the POMDP is an MDP while for an unobservable Markov Decision Process the parameter is 0. Thus, the new parameter characterizes a spectrum from MDPs to unobservable MDPs depending on the amount of information conveyed in the observations.