7,982 research outputs found
Mixture-of-Parents Maximum Entropy Markov Models
We present the mixture-of-parents maximum entropy Markov model (MoP-MEMM), a class of directed graphical models extending MEMMs. The MoP-MEMM allows tractable incorporation of long-range dependencies be- tween nodes by restricting the conditional distribution of each node to be a mixture of distributions given the parents. We show how to efficiently compute the exact marginal posterior node distributions, regardless of the range of the dependencies. This enables us to model non-sequential correlations present within text documents, as well as between in- terconnected documents, such as hyperlinked web pages. We apply the MoP-MEMM to a named entity recognition task and a web page classification task. In each, our model shows significant improvement over the basic MEMM, and is competitive with other long- range sequence models that use approximate inference. 1 Introductio
Probabilistic Methodology and Techniques for Artefact Conception and Development
The purpose of this paper is to make a state of the art on probabilistic methodology and techniques for artefact conception and development. It is the 8th deliverable of the BIBA (Bayesian Inspired Brain and Artefacts) project. We first present the incompletness problem as the central difficulty that both living creatures and artefacts have to face: how can they perceive, infer, decide and act efficiently with incomplete and uncertain knowledge?. We then introduce a generic probabilistic formalism called Bayesian Programming. This formalism is then used to review the main probabilistic methodology
and techniques. This review is organized in 3 parts: first the probabilistic models from Bayesian networks to Kalman filters and from sensor fusion to CAD systems, second the inference techniques and finally the learning and model acquisition and comparison methodologies. We conclude with the perspectives of the BIBA project as they rise from this state of the art
Localizing the Latent Structure Canonical Uncertainty: Entropy Profiles for Hidden Markov Models
This report addresses state inference for hidden Markov models. These models
rely on unobserved states, which often have a meaningful interpretation. This
makes it necessary to develop diagnostic tools for quantification of state
uncertainty. The entropy of the state sequence that explains an observed
sequence for a given hidden Markov chain model can be considered as the
canonical measure of state sequence uncertainty. This canonical measure of
state sequence uncertainty is not reflected by the classic multivariate state
profiles computed by the smoothing algorithm, which summarizes the possible
state sequences. Here, we introduce a new type of profiles which have the
following properties: (i) these profiles of conditional entropies are a
decomposition of the canonical measure of state sequence uncertainty along the
sequence and makes it possible to localize this uncertainty, (ii) these
profiles are univariate and thus remain easily interpretable on tree
structures. We show how to extend the smoothing algorithms for hidden Markov
chain and tree models to compute these entropy profiles efficiently.Comment: Submitted to Journal of Machine Learning Research; No RR-7896 (2012
Entropic Priors for Discrete Probabilistic Networks and for Mixtures of Gaussians Models
The ongoing unprecedented exponential explosion of available computing power,
has radically transformed the methods of statistical inference. What used to be
a small minority of statisticians advocating for the use of priors and a strict
adherence to bayes theorem, it is now becoming the norm across disciplines. The
evolutionary direction is now clear. The trend is towards more realistic,
flexible and complex likelihoods characterized by an ever increasing number of
parameters. This makes the old question of: What should the prior be? to
acquire a new central importance in the modern bayesian theory of inference.
Entropic priors provide one answer to the problem of prior selection. The
general definition of an entropic prior has existed since 1988, but it was not
until 1998 that it was found that they provide a new notion of complete
ignorance. This paper re-introduces the family of entropic priors as minimizers
of mutual information between the data and the parameters, as in
[rodriguez98b], but with a small change and a correction. The general formalism
is then applied to two large classes of models: Discrete probabilistic networks
and univariate finite mixtures of gaussians. It is also shown how to perform
inference by efficiently sampling the corresponding posterior distributions.Comment: 24 pages, 3 figures, Presented at MaxEnt2001, APL Johns Hopkins
University, August 4-9 2001. See also http://omega.albany.edu:8008
- …