11,196 research outputs found
Maximum A Posteriori Inference in Sum-Product Networks
Sum-product networks (SPNs) are a class of probabilistic graphical models
that allow tractable marginal inference. However, the maximum a posteriori
(MAP) inference in SPNs is NP-hard. We investigate MAP inference in SPNs from
both theoretical and algorithmic perspectives. For the theoretical part, we
reduce general MAP inference to its special case without evidence and hidden
variables; we also show that it is NP-hard to approximate the MAP problem to
for fixed , where is the input size.
For the algorithmic part, we first present an exact MAP solver that runs
reasonably fast and could handle SPNs with up to 1k variables and 150k arcs in
our experiments. We then present a new approximate MAP solver with a good
balance between speed and accuracy, and our comprehensive experiments on
real-world datasets show that it has better overall performance than existing
approximate solvers
Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks
We discuss the computational complexity of approximating maximum a posteriori
inference in sum-product networks. We first show NP-hardness in trees of height
two by a reduction from maximum independent set; this implies
non-approximability within a sublinear factor. We show that this is a tight
bound, as we can find an approximation within a linear factor in networks of
height two. We then show that, in trees of height three, it is NP-hard to
approximate the problem within a factor for any sublinear function
of the size of the input . Again, this bound is tight, as we prove that
the usual max-product algorithm finds (in any network) approximations within
factor for some constant . Last, we present a simple
algorithm, and show that it provably produces solutions at least as good as,
and potentially much better than, the max-product algorithm. We empirically
analyze the proposed algorithm against max-product using synthetic and
realistic networks.Comment: 18 page
Parametric Inference for Biological Sequence Analysis
One of the major successes in computational biology has been the unification,
using the graphical model formalism, of a multitude of algorithms for
annotating and comparing biological sequences. Graphical models that have been
applied towards these problems include hidden Markov models for annotation,
tree models for phylogenetics, and pair hidden Markov models for alignment. A
single algorithm, the sum-product algorithm, solves many of the inference
problems associated with different statistical models. This paper introduces
the \emph{polytope propagation algorithm} for computing the Newton polytope of
an observation from a graphical model. This algorithm is a geometric version of
the sum-product algorithm and is used to analyze the parametric behavior of
maximum a posteriori inference calculations for graphical models.Comment: 15 pages, 4 figures. See also companion paper "Tropical Geometry of
Statistical Models" (q-bio.QM/0311009
Maximum Weight Matching via Max-Product Belief Propagation
Max-product "belief propagation" is an iterative, local, message-passing
algorithm for finding the maximum a posteriori (MAP) assignment of a discrete
probability distribution specified by a graphical model. Despite the
spectacular success of the algorithm in many application areas such as
iterative decoding, computer vision and combinatorial optimization which
involve graphs with many cycles, theoretical results about both correctness and
convergence of the algorithm are known in few cases (Weiss-Freeman Wainwright,
Yeddidia-Weiss-Freeman, Richardson-Urbanke}.
In this paper we consider the problem of finding the Maximum Weight Matching
(MWM) in a weighted complete bipartite graph. We define a probability
distribution on the bipartite graph whose MAP assignment corresponds to the
MWM. We use the max-product algorithm for finding the MAP of this distribution
or equivalently, the MWM on the bipartite graph. Even though the underlying
bipartite graph has many short cycles, we find that surprisingly, the
max-product algorithm always converges to the correct MAP assignment as long as
the MAP assignment is unique. We provide a bound on the number of iterations
required by the algorithm and evaluate the computational cost of the algorithm.
We find that for a graph of size , the computational cost of the algorithm
scales as , which is the same as the computational cost of the best
known algorithm. Finally, we establish the precise relation between the
max-product algorithm and the celebrated {\em auction} algorithm proposed by
Bertsekas. This suggests possible connections between dual algorithm and
max-product algorithm for discrete optimization problems.Comment: In the proceedings of the 2005 IEEE International Symposium on
Information Theor
Neighbourhood-consensus message passing and its potentials in image processing applications
In this paper, a novel algorithm for inference in Markov Random Fields (MRFs) is presented. Its goal is to find approximate maximum a posteriori estimates in a simple manner by combining neighbourhood influence of iterated conditional modes (ICM) and message passing of loopy belief propagation (LBP). We call the proposed method neighbourhood-consensus message passing because a single joint message is sent from the specified neighbourhood to the central node. The message, as a function of beliefs, represents the agreement of all nodes within the neighbourhood regarding the labels of the central node. This way we are able to overcome the disadvantages of reference algorithms, ICM and LBP. On one hand, more information is propagated in comparison with ICM, while on the other hand, the huge amount of pairwise interactions is avoided in comparison with LBP by working with neighbourhoods. The idea is related to the previously developed iterated conditional expectations algorithm. Here we revisit it and redefine it in a message passing framework in a more general form. The results on three different benchmarks demonstrate that the proposed technique can perform well both for binary and multi-label MRFs without any limitations on the model definition. Furthermore, it manifests improved performance over related techniques either in terms of quality and/or speed
- …