3 research outputs found
The Vertex Sample Complexity of Free Energy is Polynomial
We study the following question: given a massive Markov random field on
nodes, can a small sample from it provide a rough approximation to the free
energy ?
Results in graph limit literature by Borgs, Chayes, Lov\'asz, S\'os, and
Vesztergombi show that for Ising models on nodes and interactions of
strength , an approximation to can be
achieved by sampling a randomly induced model on nodes.
We show that the sampling complexity of this problem is {\em polynomial in}
. We further show a polynomial dependence on cannot be
avoided.
Our results are very general as they apply to higher order Markov random
fields. For Markov random fields of order , we obtain an algorithm that
achieves approximation using a number of samples polynomial in
and and running time that is up to
polynomial factors in and . For ferromagnetic Ising models, the
running time is polynomial in .
Our results are intimately connected to recent research on the regularity
lemma and property testing, where the interest is in finding which properties
can tested within error in time polynomial in . In
particular, our proofs build on results from a recent work by Alon, de la Vega,
Kannan and Karpinski, who also introduced the notion of polynomial vertex
sample complexity. Another critical ingredient of the proof is an effective
bound by the authors of the paper relating the variational free energy and the
free energy.Comment: arXiv admin note: text overlap with arXiv:1802.06126 Updated
bibliograph
Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective
The free energy is a key quantity of interest in Ising models, but
unfortunately, computing it in general is computationally intractable. Two
popular (variational) approximation schemes for estimating the free energy of
general Ising models (in particular, even in regimes where correlation decay
does not hold) are: (i) the mean-field approximation with roots in statistical
physics, which estimates the free energy from below, and (ii) hierarchies of
convex relaxations with roots in theoretical computer science, which estimate
the free energy from above. We show, surprisingly, that the tight regime for
both methods to compute the free energy to leading order is identical.
More precisely, we show that the mean-field approximation is within
of the free energy, where denotes the
Frobenius norm of the interaction matrix of the Ising model. This
simultaneously subsumes both the breakthrough work of Basak and Mukherjee, who
showed the tight result that the mean-field approximation is within
whenever , as well as the work of Jain, Koehler, and
Mossel, who gave the previously best known non-asymptotic bound of
. We give a simple, algorithmic
proof of this result using a convex relaxation proposed by Risteski based on
the Sherali-Adams hierarchy, automatically giving sub-exponential time
approximation schemes for the free energy in this entire regime. Our
algorithmic result is tight under Gap-ETH.
We furthermore combine our techniques with spin glass theory to prove (in a
strong sense) the optimality of correlation rounding, refuting a recent
conjecture of Allen, O'Donnell, and Zhou. Finally, we give the tight
generalization of all of these results to -MRFs, capturing as a special case
previous work on approximating MAX--CSP.Comment: This version: minor formatting changes, added grant acknowledgement
The Mean-Field Approximation: Information Inequalities, Algorithms, and Complexity
The mean field approximation to the Ising model is a canonical variational
tool that is used for analysis and inference in Ising models. We provide a
simple and optimal bound for the KL error of the mean field approximation for
Ising models on general graphs, and extend it to higher order Markov random
fields. Our bound improves on previous bounds obtained in work in the graph
limit literature by Borgs, Chayes, Lov\'asz, S\'os, and Vesztergombi and
another recent work by Basak and Mukherjee. Our bound is tight up to lower
order terms. Building on the methods used to prove the bound, along with
techniques from combinatorics and optimization, we study the algorithmic
problem of estimating the (variational) free energy for Ising models and
general Markov random fields. For a graph on vertices and interaction
matrix with Frobenius norm , we provide algorithms that
approximate the free energy within an additive error of in
time . We also show that approximation within is NP-hard for every . Finally, we provide
more efficient approximation algorithms, which find the optimal mean field
approximation, for ferromagnetic Ising models and for Ising models satisfying
Dobrushin's condition.Comment: Updated bibliograph