4,219 research outputs found
A very fast inference algorithm for finite-dimensional spin glasses: Belief Propagation on the dual lattice
Starting from a Cluster Variational Method, and inspired by the correctness
of the paramagnetic Ansatz (at high temperatures in general, and at any
temperature in the 2D Edwards-Anderson model) we propose a novel message
passing algorithm --- the Dual algorithm --- to estimate the marginal
probabilities of spin glasses on finite dimensional lattices. We show that in a
wide range of temperatures our algorithm compares very well with Monte Carlo
simulations, with the Double Loop algorithm and with exact calculation of the
ground state of 2D systems with bimodal and Gaussian interactions. Moreover it
is usually 100 times faster than other provably convergent methods, as the
Double Loop algorithm.Comment: 23 pages, 12 figures. v2: improved introductio
Intrinsic limitations of inverse inference in the pairwise Ising spin glass
We analyze the limits inherent to the inverse reconstruction of a pairwise
Ising spin glass based on susceptibility propagation. We establish the
conditions under which the susceptibility propagation algorithm is able to
reconstruct the characteristics of the network given first- and second-order
local observables, evaluate eventual errors due to various types of noise in
the originally observed data, and discuss the scaling of the problem with the
number of degrees of freedom
Blind Sensor Calibration using Approximate Message Passing
The ubiquity of approximately sparse data has led a variety of com- munities
to great interest in compressed sensing algorithms. Although these are very
successful and well understood for linear measurements with additive noise,
applying them on real data can be problematic if imperfect sensing devices
introduce deviations from this ideal signal ac- quisition process, caused by
sensor decalibration or failure. We propose a message passing algorithm called
calibration approximate message passing (Cal-AMP) that can treat a variety of
such sensor-induced imperfections. In addition to deriving the general form of
the algorithm, we numerically investigate two particular settings. In the
first, a fraction of the sensors is faulty, giving readings unrelated to the
signal. In the second, sensors are decalibrated and each one introduces a
different multiplicative gain to the measures. Cal-AMP shares the scalability
of approximate message passing, allowing to treat big sized instances of these
problems, and ex- perimentally exhibits a phase transition between domains of
success and failure.Comment: 27 pages, 9 figure
A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
Restricted Boltzmann machines (RBMs) are energy-based neural-networks which
are commonly used as the building blocks for deep architectures neural
architectures. In this work, we derive a deterministic framework for the
training, evaluation, and use of RBMs based upon the Thouless-Anderson-Palmer
(TAP) mean-field approximation of widely-connected systems with weak
interactions coming from spin-glass theory. While the TAP approach has been
extensively studied for fully-visible binary spin systems, our construction is
generalized to latent-variable models, as well as to arbitrarily distributed
real-valued spin systems with bounded support. In our numerical experiments, we
demonstrate the effective deterministic training of our proposed models and are
able to show interesting features of unsupervised learning which could not be
directly observed with sampling. Additionally, we demonstrate how to utilize
our TAP-based framework for leveraging trained RBMs as joint priors in
denoising problems
Inference and Optimization of Real Edges on Sparse Graphs - A Statistical Physics Perspective
Inference and optimization of real-value edge variables in sparse graphs are
studied using the Bethe approximation and replica method of statistical
physics. Equilibrium states of general energy functions involving a large set
of real edge-variables that interact at the network nodes are obtained in
various cases. When applied to the representative problem of network resource
allocation, efficient distributed algorithms are also devised. Scaling
properties with respect to the network connectivity and the resource
availability are found, and links to probabilistic Bayesian approximation
methods are established. Different cost measures are considered and algorithmic
solutions in the various cases are devised and examined numerically. Simulation
results are in full agreement with the theory.Comment: 21 pages, 10 figures, major changes: Sections IV to VII updated,
Figs. 1 to 3 replace
An improved Belief Propagation algorithm finds many Bethe states in the random field Ising model on random graphs
We first present an empirical study of the Belief Propagation (BP) algorithm,
when run on the random field Ising model defined on random regular graphs in
the zero temperature limit. We introduce the notion of maximal solutions for
the BP equations and we use them to fix a fraction of spins in their ground
state configuration. At the phase transition point the fraction of
unconstrained spins percolates and their number diverges with the system size.
This in turn makes the associated optimization problem highly non trivial in
the critical region. Using the bounds on the BP messages provided by the
maximal solutions we design a new and very easy to implement BP scheme which is
able to output a large number of stable fixed points. On one side this new
algorithm is able to provide the minimum energy configuration with high
probability in a competitive time. On the other side we found that the number
of fixed points of the BP algorithm grows with the system size in the critical
region. This unexpected feature poses new relevant questions on the physics of
this class of models.Comment: 20 pages, 8 figure
Automating embedded analysis capabilities and managing software complexity in multiphysics simulation part II: application to partial differential equations
A template-based generic programming approach was presented in a previous
paper that separates the development effort of programming a physical model
from that of computing additional quantities, such as derivatives, needed for
embedded analysis algorithms. In this paper, we describe the implementation
details for using the template-based generic programming approach for
simulation and analysis of partial differential equations (PDEs). We detail
several of the hurdles that we have encountered, and some of the software
infrastructure developed to overcome them. We end with a demonstration where we
present shape optimization and uncertainty quantification results for a 3D PDE
application
Gauge-free cluster variational method by maximal messages and moment matching
We present an implementation of the cluster variational method (CVM) as a message passing algorithm. The kind of message passing algorithm used for CVM, usually named generalized belief propagation (GBP), is a generalization of the belief propagation algorithm in the same way that CVM is a generalization of the Bethe approximation for estimating the partition function. However, the connection between fixed points of GBP and the extremal points of the CVM free energy is usually not a one-to-one correspondence because of the existence of a gauge transformation involving the GBP messages. Our contribution is twofold. First, we propose a way of defining messages (fields) in a generic CVM approximation, such that messages arrive on a given region from all its ancestors, and not only from its direct parents, as in the standard parent-to-child GBP. We call this approach maximal messages. Second, we focus on the case of binary variables, reinterpreting the messages as fields enforcing the consistency between the moments of the local (marginal) probability distributions. We provide a precise rule to enforce all consistencies, avoiding any redundancy, that would otherwise lead to a gauge transformation on the messages. This moment matching method is gauge free, i.e., it guarantees that the resulting GBP is not gauge invariant. We apply our maximal messages and moment matching GBP to obtain an analytical expression for the critical temperature of the Ising model in general dimensions at the level of plaquette CVM. The values obtained outperform Bethe estimates, and are comparable with loop corrected belief propagation equations. The method allows for a straightforward generalization to disordered systems
- …