1,574 research outputs found
Projecting Ising Model Parameters for Fast Mixing
Inference in general Ising models is difficult, due to high treewidth making
tree-based algorithms intractable. Moreover, when interactions are strong,
Gibbs sampling may take exponential time to converge to the stationary
distribution. We present an algorithm to project Ising model parameters onto a
parameter set that is guaranteed to be fast mixing, under several divergences.
We find that Gibbs sampling using the projected parameters is more accurate
than with the original parameters when interaction strengths are strong and
when limited time is available for sampling.Comment: Advances in Neural Information Processing Systems 201
Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets
Inference is typically intractable in high-treewidth undirected graphical
models, making maximum likelihood learning a challenge. One way to overcome
this is to restrict parameters to a tractable set, most typically the set of
tree-structured parameters. This paper explores an alternative notion of a
tractable set, namely a set of "fast-mixing parameters" where Markov chain
Monte Carlo (MCMC) inference can be guaranteed to quickly converge to the
stationary distribution. While it is common in practice to approximate the
likelihood gradient using samples obtained from MCMC, such procedures lack
theoretical guarantees. This paper proves that for any exponential family with
bounded sufficient statistics, (not just graphical models) when parameters are
constrained to a fast-mixing set, gradient descent with gradients approximated
by sampling will approximate the maximum likelihood solution inside the set
with high-probability. When unregularized, to find a solution epsilon-accurate
in log-likelihood requires a total amount of effort cubic in 1/epsilon,
disregarding logarithmic factors. When ridge-regularized, strong convexity
allows a solution epsilon-accurate in parameter distance with effort quadratic
in 1/epsilon. Both of these provide of a fully-polynomial time randomized
approximation scheme.Comment: Advances in Neural Information Processing Systems 201
Glauber dynamics on trees:Boundary conditions and mixing time
We give the first comprehensive analysis of the effect of boundary conditions
on the mixing time of the Glauber dynamics in the so-called Bethe
approximation. Specifically, we show that spectral gap and the log-Sobolev
constant of the Glauber dynamics for the Ising model on an n-vertex regular
tree with plus-boundary are bounded below by a constant independent of n at all
temperatures and all external fields. This implies that the mixing time is
O(log n) (in contrast to the free boundary case, where it is not bounded by any
fixed polynomial at low temperatures). In addition, our methods yield simpler
proofs and stronger results for the spectral gap and log-Sobolev constant in
the regime where there are multiple phases but the mixing time is insensitive
to the boundary condition. Our techniques also apply to a much wider class of
models, including those with hard-core constraints like the antiferromagnetic
Potts model at zero temperature (proper colorings) and the hard--core lattice
gas (independent sets)
Projecting Markov Random Field Parameters for Fast Mixing
Markov chain Monte Carlo (MCMC) algorithms are simple and extremely powerful
techniques to sample from almost arbitrary distributions. The flaw in practice
is that it can take a large and/or unknown amount of time to converge to the
stationary distribution. This paper gives sufficient conditions to guarantee
that univariate Gibbs sampling on Markov Random Fields (MRFs) will be fast
mixing, in a precise sense. Further, an algorithm is given to project onto this
set of fast-mixing parameters in the Euclidean norm. Following recent work, we
give an example use of this to project in various divergence measures,
comparing univariate marginals obtained by sampling after projection to common
variational methods and Gibbs sampling on the original parameters.Comment: Neural Information Processing Systems 201
A General Algorithm for Sampling Rare Events in Non-Equilibrium and Non-Stationary Systems
Although many computational methods for rare event sampling exist, this type
of calculation is not usually practical for general nonequilibrium conditions,
with macroscopically irreversible dynamics and away from both stationary and
metastable states. A novel method for calculating the time-series of the
probability of a rare event is presented which is designed for these
conditions. The method is validated for the cases of the Glauber-Ising model
under time-varying shear flow, the Kawasaki-Ising model after a quench into the
region between nucleation dominated and spinodal decomposition dominated phase
change dynamics, and the parallel open asymmetric exclusion process (p-o ASEP).
The method requires a subdivision of the phase space of the system: it is
benchmarked and found to scale well for increasingly fine subdivisions, meaning
that it can be applied without detailed foreknowledge of the physically
important reaction pathways.Comment: 11 pages, 6 figure
- …