12,825 research outputs found
Cycle-based Cluster Variational Method for Direct and Inverse Inference
We elaborate on the idea that loop corrections to belief propagation could be
dealt with in a systematic way on pairwise Markov random fields, by using the
elements of a cycle basis to define region in a generalized belief propagation
setting. The region graph is specified in such a way as to avoid dual loops as
much as possible, by discarding redundant Lagrange multipliers, in order to
facilitate the convergence, while avoiding instabilities associated to minimal
factor graph construction. We end up with a two-level algorithm, where a belief
propagation algorithm is run alternatively at the level of each cycle and at
the inter-region level. The inverse problem of finding the couplings of a
Markov random field from empirical covariances can be addressed region wise. It
turns out that this can be done efficiently in particular in the Ising context,
where fixed point equations can be derived along with a one-parameter log
likelihood function to minimize. Numerical experiments confirm the
effectiveness of these considerations both for the direct and inverse MRF
inference.Comment: 47 pages, 16 figure
Truncating the loop series expansion for Belief Propagation
Recently, M. Chertkov and V.Y. Chernyak derived an exact expression for the
partition sum (normalization constant) corresponding to a graphical model,
which is an expansion around the Belief Propagation solution. By adding
correction terms to the BP free energy, one for each "generalized loop" in the
factor graph, the exact partition sum is obtained. However, the usually
enormous number of generalized loops generally prohibits summation over all
correction terms. In this article we introduce Truncated Loop Series BP
(TLSBP), a particular way of truncating the loop series of M. Chertkov and V.Y.
Chernyak by considering generalized loops as compositions of simple loops. We
analyze the performance of TLSBP in different scenarios, including the Ising
model, regular random graphs and on Promedas, a large probabilistic medical
diagnostic system. We show that TLSBP often improves upon the accuracy of the
BP solution, at the expense of increased computation time. We also show that
the performance of TLSBP strongly depends on the degree of interaction between
the variables. For weak interactions, truncating the series leads to
significant improvements, whereas for strong interactions it can be
ineffective, even if a high number of terms is considered.Comment: 31 pages, 12 figures, submitted to Journal of Machine Learning
Researc
Factored expectation propagation for input-output FHMM models in systems biology
We consider the problem of joint modelling of metabolic signals and gene
expression in systems biology applications. We propose an approach based on
input-output factorial hidden Markov models and propose a structured
variational inference approach to infer the structure and states of the model.
We start from the classical free form structured variational mean field
approach and use a expectation propagation to approximate the expectations
needed in the variational loop. We show that this corresponds to a factored
expectation constrained approximate inference. We validate our model through
extensive simulations and demonstrate its applicability on a real world
bacterial data set
Quantifying the tension between the Higgs mass and (g-2)_mu in the CMSSM
Supersymmetry has been often invoqued as the new physics that might reconcile
the experimental muon magnetic anomaly, a_mu, with the theoretical prediction
(basing the computation of the hadronic contribution on e^+ e^- data). However,
in the context of the CMSSM, the required supersymmetric contributions (which
grow with decreasing supersymmetric masses) are in potential tension with a
possibly large Higgs mass (which requires large stop masses). In the limit of
very large m_h supersymmetry gets decoupled, and the CMSSM must show the same
discrepancy as the SM with a_mu . But it is much less clear for which size of
m_h does the tension start to be unbearable. In this paper, we quantify this
tension with the help of Bayesian techniques. We find that for m_h > 125 GeV
the maximum level of discrepancy given current data (~ 3.3 sigma) is already
achieved. Requiring less than 3 sigma discrepancy, implies m_h < 120 GeV. For a
larger Higgs mass we should give up either the CMSSM model or the computation
of a_mu based on e^+ e^-; or accept living with such inconsistency
Loop corrections in spin models through density consistency
Computing marginal distributions of discrete or semidiscrete Markov random
fields (MRFs) is a fundamental, generally intractable problem with a vast
number of applications in virtually all fields of science. We present a new
family of computational schemes to approximately calculate the marginals of
discrete MRFs. This method shares some desirable properties with belief
propagation, in particular, providing exact marginals on acyclic graphs, but it
differs with the latter in that it includes some loop corrections; i.e., it
takes into account correlations coming from all cycles in the factor graph. It
is also similar to the adaptive Thouless-Anderson-Palmer method, but it differs
with the latter in that the consistency is not on the first two moments of the
distribution but rather on the value of its density on a subset of values. The
results on finite-dimensional Isinglike models show a significant improvement
with respect to the Bethe-Peierls (tree) approximation in all cases and with
respect to the plaquette cluster variational method approximation in many
cases. In particular, for the critical inverse temperature of the
homogeneous hypercubic lattice, the expansion of
around of the proposed scheme is exact up to the order,
whereas the two latter are exact only up to the order.Comment: 12 pages, 3 figures, 1 tabl
Loop series for discrete statistical models on graphs
In this paper we present derivation details, logic, and motivation for the
loop calculus introduced in \cite{06CCa}. Generating functions for three
inter-related discrete statistical models are each expressed in terms of a
finite series. The first term in the series corresponds to the Bethe-Peierls
(Belief Propagation)-BP contribution, the other terms are labeled by loops on
the factor graph. All loop contributions are simple rational functions of spin
correlation functions calculated within the BP approach. We discuss two
alternative derivations of the loop series. One approach implements a set of
local auxiliary integrations over continuous fields with the BP contribution
corresponding to an integrand saddle-point value. The integrals are replaced by
sums in the complimentary approach, briefly explained in \cite{06CCa}. A local
gauge symmetry transformation that clarifies an important invariant feature of
the BP solution, is revealed in both approaches. The partition function remains
invariant while individual terms change under the gauge transformation. The
requirement for all individual terms to be non-zero only for closed loops in
the factor graph (as opposed to paths with loose ends) is equivalent to fixing
the first term in the series to be exactly equal to the BP contribution.
Further applications of the loop calculus to problems in statistical physics,
computer and information sciences are discussed.Comment: 20 pages, 3 figure
- …