4,400 research outputs found
Nonparametric inference in hidden Markov models using P-splines
Hidden Markov models (HMMs) are flexible time series models in which the
distributions of the observations depend on unobserved serially correlated
states. The state-dependent distributions in HMMs are usually taken from some
class of parametrically specified distributions. The choice of this class can
be difficult, and an unfortunate choice can have serious consequences for
example on state estimates, on forecasts and generally on the resulting model
complexity and interpretation, in particular with respect to the number of
states. We develop a novel approach for estimating the state-dependent
distributions of an HMM in a nonparametric way, which is based on the idea of
representing the corresponding densities as linear combinations of a large
number of standardized B-spline basis functions, imposing a penalty term on
non-smoothness in order to maintain a good balance between goodness-of-fit and
smoothness. We illustrate the nonparametric modeling approach in a real data
application concerned with vertical speeds of a diving beaked whale,
demonstrating that compared to parametric counterparts it can lead to models
that are more parsimonious in terms of the number of states yet fit the data
equally well
Regularized Maximum Likelihood Estimation and Feature Selection in Mixtures-of-Experts Models
Mixture of Experts (MoE) are successful models for modeling heterogeneous
data in many statistical learning problems including regression, clustering and
classification. Generally fitted by maximum likelihood estimation via the
well-known EM algorithm, their application to high-dimensional problems is
still therefore challenging. We consider the problem of fitting and feature
selection in MoE models, and propose a regularized maximum likelihood
estimation approach that encourages sparse solutions for heterogeneous
regression data models with potentially high-dimensional predictors. Unlike
state-of-the art regularized MLE for MoE, the proposed modelings do not require
an approximate of the penalty function. We develop two hybrid EM algorithms: an
Expectation-Majorization-Maximization (EM/MM) algorithm, and an EM algorithm
with coordinate ascent algorithm. The proposed algorithms allow to
automatically obtaining sparse solutions without thresholding, and avoid matrix
inversion by allowing univariate parameter updates. An experimental study shows
the good performance of the algorithms in terms of recovering the actual sparse
solutions, parameter estimation, and clustering of heterogeneous regression
data
- …