The Complexity of Approximating a Bethe Equilibrium

This paper resolves a common complexity issue in the Bethe approximation of statistical physics and the Belief Propagation (BP) algorithm of artificial intelligence. The Bethe approximation and the BP algorithm are heuristic methods for estimating the partition function and marginal probabilities in graphical models, respectively. The computational complexity of the Bethe approximation is decided by the number of operations required to solve a set of non-linear equations, the so-called Bethe equation. Although the BP algorithm was inspired and developed independently, Yedidia, Freeman and Weiss (2004) showed that the BP algorithm solves the Bethe equation if it converges (however, it often does not). This naturally motivates the following question to understand limitations and empirical successes of the Bethe and BP methods: is the Bethe equation computationally easy to solve? We present a message-passing algorithm solving the Bethe equation in a polynomial number of operations for general binary graphical models of n variables where the maximum degree in the underlying graph is O(log n). Our algorithm can be used as an alternative to BP fixing its convergence issue and is the first fully polynomial-time approximation scheme for the BP fixed-point computation in such a large class of graphical models, while the approximate fixed-point computation is known to be (PPAD-)hard in general. We believe that our technique is of broader interest to understand the computational complexity of the cavity method in statistical physics

Bethe free-energy approximations for disordered quantum systems

Given a locally consistent set of reduced density matrices, we construct approximate density matrices which are globally consistent with the local density matrices we started from when the trial density matrix has a tree structure. We employ the cavity method of statistical physics to find the optimal density matrix representation by slowly decreasing the temperature in an annealing algorithm, or by minimizing an approximate Bethe free energy depending on the reduced density matrices and some cavity messages originated from the Bethe approximation of the entropy. We obtain the classical Bethe expression for the entropy within a naive (mean-field) approximation of the cavity messages, which is expected to work well at high temperatures. In the next order of the approximation, we obtain another expression for the Bethe entropy depending only on the diagonal elements of the reduced density matrices. In principle, we can improve the entropy approximation by considering more accurate cavity messages in the Bethe approximation of the entropy. We compare the annealing algorithm and the naive approximation of the Bethe entropy with exact and approximate numerical simulations for small and large samples of the random transverse Ising model on random regular graphs.Comment: 23 pages, 4 figures, 4 appendice

An improved Belief Propagation algorithm finds many Bethe states in the random field Ising model on random graphs

We first present an empirical study of the Belief Propagation (BP) algorithm, when run on the random field Ising model defined on random regular graphs in the zero temperature limit. We introduce the notion of maximal solutions for the BP equations and we use them to fix a fraction of spins in their ground state configuration. At the phase transition point the fraction of unconstrained spins percolates and their number diverges with the system size. This in turn makes the associated optimization problem highly non trivial in the critical region. Using the bounds on the BP messages provided by the maximal solutions we design a new and very easy to implement BP scheme which is able to output a large number of stable fixed points. On one side this new algorithm is able to provide the minimum energy configuration with high probability in a competitive time. On the other side we found that the number of fixed points of the BP algorithm grows with the system size in the critical region. This unexpected feature poses new relevant questions on the physics of this class of models.Comment: 20 pages, 8 figure

Loop corrections in spin models through density consistency

Computing marginal distributions of discrete or semidiscrete Markov random fields (MRFs) is a fundamental, generally intractable problem with a vast number of applications in virtually all fields of science. We present a new family of computational schemes to approximately calculate the marginals of discrete MRFs. This method shares some desirable properties with belief propagation, in particular, providing exact marginals on acyclic graphs, but it differs with the latter in that it includes some loop corrections; i.e., it takes into account correlations coming from all cycles in the factor graph. It is also similar to the adaptive Thouless-Anderson-Palmer method, but it differs with the latter in that the consistency is not on the first two moments of the distribution but rather on the value of its density on a subset of values. The results on finite-dimensional Isinglike models show a significant improvement with respect to the Bethe-Peierls (tree) approximation in all cases and with respect to the plaquette cluster variational method approximation in many cases. In particular, for the critical inverse temperature $\beta_{c}$ of the homogeneous hypercubic lattice, the expansion of $\left(d\beta_{c}\right)^{-1}$ around $d=\infty$ of the proposed scheme is exact up to the $d^{-4}$ order, whereas the two latter are exact only up to the $d^{-2}$ order.Comment: 12 pages, 3 figures, 1 tabl

Diagonalizing transfer matrices and matrix product operators: a medley of exact and computational methods

Transfer matrices and matrix product operators play an ubiquitous role in the field of many body physics. This paper gives an ideosyncratic overview of applications, exact results and computational aspects of diagonalizing transfer matrices and matrix product operators. The results in this paper are a mixture of classic results, presented from the point of view of tensor networks, and of new results. Topics discussed are exact solutions of transfer matrices in equilibrium and non-equilibrium statistical physics, tensor network states, matrix product operator algebras, and numerical matrix product state methods for finding extremal eigenvectors of matrix product operators.Comment: Lecture notes from a course at Vienna Universit

Stability of self-consistent solutions for the Hubbard model at intermediate and strong coupling

We present a general framework how to investigate stability of solutions within a single self-consistent renormalization scheme being a parquet-type extension of the Baym-Kadanoff construction of conserving approximations. To obtain a consistent description of one- and two-particle quantities, needed for the stability analysis, we impose equations of motion on the one- as well on the two-particle Green functions simultaneously and introduce approximations in their input, the completely irreducible two-particle vertex. Thereby we do not loose singularities caused by multiple two-particle scatterings. We find a complete set of stability criteria and show that each instability, singularity in a two-particle function, is connected with a symmetry-breaking order parameter, either of density type or anomalous. We explicitly study the Hubbard model at intermediate coupling and demonstrate that approximations with static vertices get unstable before a long-range order or a metal-insulator transition can be reached. We use the parquet approximation and turn it to a workable scheme with dynamical vertex corrections. We derive a qualitatively new theory with two-particle self-consistence, the complexity of which is comparable with FLEX-type approximations. We show that it is the simplest consistent and stable theory being able to describe qualitatively correctly quantum critical points and the transition from weak to strong coupling in correlated electron systems.Comment: REVTeX, 26 pages, 12 PS figure