22,550 research outputs found
Minimum Conditional Description Length Estimation for Markov Random Fields
In this paper we discuss a method, which we call Minimum Conditional
Description Length (MCDL), for estimating the parameters of a subset of sites
within a Markov random field. We assume that the edges are known for the entire
graph . Then, for a subset , we estimate the parameters
for nodes and edges in as well as for edges incident to a node in , by
finding the exponential parameter for that subset that yields the best
compression conditioned on the values on the boundary . Our
estimate is derived from a temporally stationary sequence of observations on
the set . We discuss how this method can also be applied to estimate a
spatially invariant parameter from a single configuration, and in so doing,
derive the Maximum Pseudo-Likelihood (MPL) estimate.Comment: Information Theory and Applications (ITA) workshop, February 201
On Graphical Models via Univariate Exponential Family Distributions
Undirected graphical models, or Markov networks, are a popular class of
statistical models, used in a wide variety of applications. Popular instances
of this class include Gaussian graphical models and Ising models. In many
settings, however, it might not be clear which subclass of graphical models to
use, particularly for non-Gaussian and non-categorical data. In this paper, we
consider a general sub-class of graphical models where the node-wise
conditional distributions arise from exponential families. This allows us to
derive multivariate graphical model distributions from univariate exponential
family distributions, such as the Poisson, negative binomial, and exponential
distributions. Our key contributions include a class of M-estimators to fit
these graphical model distributions; and rigorous statistical analysis showing
that these M-estimators recover the true graphical model structure exactly,
with high probability. We provide examples of genomic and proteomic networks
learned via instances of our class of graphical models derived from Poisson and
exponential distributions.Comment: Journal of Machine Learning Researc
Conjugate Projective Limits
We characterize conjugate nonparametric Bayesian models as projective limits
of conjugate, finite-dimensional Bayesian models. In particular, we identify a
large class of nonparametric models representable as infinite-dimensional
analogues of exponential family distributions and their canonical conjugate
priors. This class contains most models studied in the literature, including
Dirichlet processes and Gaussian process regression models. To derive these
results, we introduce a representation of infinite-dimensional Bayesian models
by projective limits of regular conditional probabilities. We show under which
conditions the nonparametric model itself, its sufficient statistics, and -- if
they exist -- conjugate updates of the posterior are projective limits of their
respective finite-dimensional counterparts. We illustrate our results both by
application to existing nonparametric models and by construction of a model on
infinite permutations.Comment: 49 pages; improved version: revised proof of theorem 3 (results
unchanged), discussion added, exposition revise
Algebraic statistical models
Many statistical models are algebraic in that they are defined in terms of
polynomial constraints, or in terms of polynomial or rational parametrizations.
The parameter spaces of such models are typically semi-algebraic subsets of the
parameter space of a reference model with nice properties, such as for example
a regular exponential family. This observation leads to the definition of an
`algebraic exponential family'. This new definition provides a unified
framework for the study of statistical models with algebraic structure. In this
paper we review the ingredients to this definition and illustrate in examples
how computational algebraic geometry can be used to solve problems arising in
statistical inference in algebraic models
Parsimonious Description of Generalized Gibbs Measures : Decimation of the 2d-Ising Model
In this paper, we detail and complete the existing characterizations of the
decimation of the Ising model on in the generalized Gibbs context. We
first recall a few features of the Dobrushin program of restoration of
Gibbsianness and present the construction of global specifications consistent
with the extremal decimated measures. We use them to consider these
renormalized measures as almost Gibbsian measures and to precise its convex set
of DLR measures. We also recall the weakly Gibbsian description and complete it
using a potential that admits a quenched correlation decay, i.e. a well-defined
configuration-dependent length beyond which this potential decays
exponentially. We use these results to incorporate these decimated measures in
the new framework of parsimonious random fields that has been recently
developed to investigate probability aspects related to neurosciences.Comment: 32 pages, preliminary versio
- …