424 research outputs found
Mean Field Theory for Sigmoid Belief Networks
We develop a mean field theory for sigmoid belief networks based on ideas
from statistical mechanics. Our mean field theory provides a tractable
approximation to the true probability distribution in these networks; it also
yields a lower bound on the likelihood of evidence. We demonstrate the utility
of this framework on a benchmark problem in statistical pattern
recognition---the classification of handwritten digits.Comment: See http://www.jair.org/ for any accompanying file
Mean Field Methods for a Special Class of Belief Networks
The chief aim of this paper is to propose mean-field approximations for a
broad class of Belief networks, of which sigmoid and noisy-or networks can be
seen as special cases. The approximations are based on a powerful mean-field
theory suggested by Plefka. We show that Saul, Jaakkola and Jordan' s approach
is the first order approximation in Plefka's approach, via a variational
derivation. The application of Plefka's theory to belief networks is not
computationally tractable. To tackle this problem we propose new approximations
based on Taylor series. Small scale experiments show that the proposed schemes
are attractive
Projecting Ising Model Parameters for Fast Mixing
Inference in general Ising models is difficult, due to high treewidth making
tree-based algorithms intractable. Moreover, when interactions are strong,
Gibbs sampling may take exponential time to converge to the stationary
distribution. We present an algorithm to project Ising model parameters onto a
parameter set that is guaranteed to be fast mixing, under several divergences.
We find that Gibbs sampling using the projected parameters is more accurate
than with the original parameters when interaction strengths are strong and
when limited time is available for sampling.Comment: Advances in Neural Information Processing Systems 201
- …