2,930 research outputs found
Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
In recent years a number of methods have been developed for automatically
learning the (sparse) connectivity structure of Markov Random Fields. These
methods are mostly based on L1-regularized optimization which has a number of
disadvantages such as the inability to assess model uncertainty and expensive
cross-validation to find the optimal regularization parameter. Moreover, the
model's predictive performance may degrade dramatically with a suboptimal value
of the regularization parameter (which is sometimes desirable to induce
sparseness). We propose a fully Bayesian approach based on a "spike and slab"
prior (similar to L0 regularization) that does not suffer from these
shortcomings. We develop an approximate MCMC method combining Langevin dynamics
and reversible jump MCMC to conduct inference in this model. Experiments show
that the proposed model learns a good combination of the structure and
parameter values without the need for separate hyper-parameter tuning.
Moreover, the model's predictive performance is much more robust than L1-based
methods with hyper-parameter settings that induce highly sparse model
structures.Comment: Accepted in the Conference on Uncertainty in Artificial Intelligence
(UAI), 201
Maximum Likelihood Estimation in Gaussian Chain Graph Models under the Alternative Markov Property
The AMP Markov property is a recently proposed alternative Markov property
for chain graphs. In the case of continuous variables with a joint multivariate
Gaussian distribution, it is the AMP rather than the earlier introduced LWF
Markov property that is coherent with data-generation by natural
block-recursive regressions. In this paper, we show that maximum likelihood
estimates in Gaussian AMP chain graph models can be obtained by combining
generalized least squares and iterative proportional fitting to an iterative
algorithm. In an appendix, we give useful convergence results for iterative
partial maximization algorithms that apply in particular to the described
algorithm.Comment: 15 pages, article will appear in Scandinavian Journal of Statistic
Efficient likelihood estimation in state space models
Motivated by studying asymptotic properties of the maximum likelihood
estimator (MLE) in stochastic volatility (SV) models, in this paper we
investigate likelihood estimation in state space models. We first prove, under
some regularity conditions, there is a consistent sequence of roots of the
likelihood equation that is asymptotically normal with the inverse of the
Fisher information as its variance. With an extra assumption that the
likelihood equation has a unique root for each , then there is a consistent
sequence of estimators of the unknown parameters. If, in addition, the supremum
of the log likelihood function is integrable, the MLE exists and is strongly
consistent. Edgeworth expansion of the approximate solution of likelihood
equation is also established. Several examples, including Markov switching
models, ARMA models, (G)ARCH models and stochastic volatility (SV) models, are
given for illustration.Comment: With the comments by Jens Ledet Jensen and reply to the comments.
Published at http://dx.doi.org/10.1214/009053606000000614;
http://dx.doi.org/10.1214/09-AOS748A; http://dx.doi.org/10.1214/09-AOS748B in
the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Hierarchical Models for Independence Structures of Networks
We introduce a new family of network models, called hierarchical network
models, that allow us to represent in an explicit manner the stochastic
dependence among the dyads (random ties) of the network. In particular, each
member of this family can be associated with a graphical model defining
conditional independence clauses among the dyads of the network, called the
dependency graph. Every network model with dyadic independence assumption can
be generalized to construct members of this new family. Using this new
framework, we generalize the Erd\"os-R\'enyi and beta-models to create
hierarchical Erd\"os-R\'enyi and beta-models. We describe various methods for
parameter estimation as well as simulation studies for models with sparse
dependency graphs.Comment: 19 pages, 7 figure
Unsupervised Bilingual POS Tagging with Markov Random Fields
In this paper, we give a treatment to the problem of bilingual part-of-speech induction with parallel data. We demonstrate that naïve optimization of log-likelihood with joint MRFs suffers from a severe problem of local maxima, and suggest an alternative – using contrastive estimation for estimation of the parameters. Our experiments show that estimating the parameters this way, using overlapping features with joint MRFs performs better than previous work on the 1984 dataset.
Binary Models for Marginal Independence
Log-linear models are a classical tool for the analysis of contingency
tables. In particular, the subclass of graphical log-linear models provides a
general framework for modelling conditional independences. However, with the
exception of special structures, marginal independence hypotheses cannot be
accommodated by these traditional models. Focusing on binary variables, we
present a model class that provides a framework for modelling marginal
independences in contingency tables. The approach taken is graphical and draws
on analogies to multivariate Gaussian models for marginal independence. For the
graphical model representation we use bi-directed graphs, which are in the
tradition of path diagrams. We show how the models can be parameterized in a
simple fashion, and how maximum likelihood estimation can be performed using a
version of the Iterated Conditional Fitting algorithm. Finally we consider
combining these models with symmetry restrictions
- …