662 research outputs found
Unsupervised learning of human motion
An unsupervised learning algorithm that can obtain a probabilistic model of an object composed of a collection of parts (a moving human body in our examples) automatically from unlabeled training data is presented. The training data include both useful "foreground" features as well as features that arise from irrelevant background clutter - the correspondence between parts and detected features is unknown. The joint probability density function of the parts is represented by a mixture of decomposable triangulated graphs which allow for fast detection. To learn the model structure as well as model parameters, an EM-like algorithm is developed where the labeling of the data (part assignments) is treated as hidden variables. The unsupervised learning technique is not limited to decomposable triangulated graphs. The efficiency and effectiveness of our algorithm is demonstrated by applying it to generate models of human motion automatically from unlabeled image sequences, and testing the learned models on a variety of sequences
Vertex Sparsifiers: New Results from Old Techniques
Given a capacitated graph and a set of terminals ,
how should we produce a graph only on the terminals so that every
(multicommodity) flow between the terminals in could be supported in
with low congestion, and vice versa? (Such a graph is called a
flow-sparsifier for .) What if we want to be a "simple" graph? What if
we allow to be a convex combination of simple graphs?
Improving on results of Moitra [FOCS 2009] and Leighton and Moitra [STOC
2010], we give efficient algorithms for constructing: (a) a flow-sparsifier
that maintains congestion up to a factor of , where , (b) a convex combination of trees over the terminals that maintains
congestion up to a factor of , and (c) for a planar graph , a
convex combination of planar graphs that maintains congestion up to a constant
factor. This requires us to give a new algorithm for the 0-extension problem,
the first one in which the preimages of each terminal are connected in .
Moreover, this result extends to minor-closed families of graphs.
Our improved bounds immediately imply improved approximation guarantees for
several terminal-based cut and ordering problems.Comment: An extended abstract appears in the 13th International Workshop on
Approximation Algorithms for Combinatorial Optimization Problems (APPROX),
2010. Final version to appear in SIAM J. Computin
Probabilistic Integral Circuits
Continuous latent variables (LVs) are a key ingredient of many generative
models, as they allow modelling expressive mixtures with an uncountable number
of components. In contrast, probabilistic circuits (PCs) are hierarchical
discrete mixtures represented as computational graphs composed of input, sum
and product units. Unlike continuous LV models, PCs provide tractable inference
but are limited to discrete LVs with categorical (i.e. unordered) states. We
bridge these model classes by introducing probabilistic integral circuits
(PICs), a new language of computational graphs that extends PCs with integral
units representing continuous LVs. In the first place, PICs are symbolic
computational graphs and are fully tractable in simple cases where analytical
integration is possible. In practice, we parameterise PICs with light-weight
neural nets delivering an intractable hierarchical continuous mixture that can
be approximated arbitrarily well with large PCs using numerical quadrature. On
several distribution estimation benchmarks, we show that such PIC-approximating
PCs systematically outperform PCs commonly learned via expectation-maximization
or SGD
Approximation Complexity of Maximum A Posteriori Inference in Sum-Product Networks
We discuss the computational complexity of approximating maximum a posteriori
inference in sum-product networks. We first show NP-hardness in trees of height
two by a reduction from maximum independent set; this implies
non-approximability within a sublinear factor. We show that this is a tight
bound, as we can find an approximation within a linear factor in networks of
height two. We then show that, in trees of height three, it is NP-hard to
approximate the problem within a factor for any sublinear function
of the size of the input . Again, this bound is tight, as we prove that
the usual max-product algorithm finds (in any network) approximations within
factor for some constant . Last, we present a simple
algorithm, and show that it provably produces solutions at least as good as,
and potentially much better than, the max-product algorithm. We empirically
analyze the proposed algorithm against max-product using synthetic and
realistic networks.Comment: 18 page
Inference via low-dimensional couplings
We investigate the low-dimensional structure of deterministic transformations
between random variables, i.e., transport maps between probability measures. In
the context of statistics and machine learning, these transformations can be
used to couple a tractable "reference" measure (e.g., a standard Gaussian) with
a target measure of interest. Direct simulation from the desired measure can
then be achieved by pushing forward reference samples through the map. Yet
characterizing such a map---e.g., representing and evaluating it---grows
challenging in high dimensions. The central contribution of this paper is to
establish a link between the Markov properties of the target measure and the
existence of low-dimensional couplings, induced by transport maps that are
sparse and/or decomposable. Our analysis not only facilitates the construction
of transformations in high-dimensional settings, but also suggests new
inference methodologies for continuous non-Gaussian graphical models. For
instance, in the context of nonlinear state-space models, we describe new
variational algorithms for filtering, smoothing, and sequential parameter
inference. These algorithms can be understood as the natural
generalization---to the non-Gaussian case---of the square-root
Rauch-Tung-Striebel Gaussian smoother.Comment: 78 pages, 25 figure
Efficient approximation of probability distributions with k-order decomposable models
During the last decades several learning algorithms have been proposed to learn probability distributions based on decomposable models. Some of these algorithms can be used to search for a maximum likelihood decomposable model with a given maximum clique size, k. Unfortunately, the problem of learning a maximum likelihood decomposable model given a maximum clique size is NP-hard for k > 2. In this work, we propose the fractal tree family of algorithms which approximates this problem with a computational complexity of O(k 2 · n 2 · N ) in the worst case, where n is the number of implied random variables and N is the size of the training set. The fractal tree algorithms construct a sequence of maximal i-order decomposable graphs, for i = 2, ..., k, in k − 1 steps. At each step, the algorithms follow a divide-and-conquer strategy that decomposes the problem into a set of separator problems. Each separator problem is efficiently solved using the generalized Chow-Liu algorithm. Fractal trees can be considered a natural extension of the Chow-Liu algorithm, from k = 2 to arbitrary values of k, and they have shown a competitive behaviour to deal with the maximum likelihood problem. Due to their competitive behavior, their low computational complexity and their modularity, which allow them to implement different parallelization strategies, the proposed procedures are especially advisable for modelling high dimensional domains.Saiotek and IT609-13 programs (Basque Government)
TIN2013-41272-P (Spanish Ministry of Science and Innovation)
COMBIOMED network in computational bio-medicine (Carlos III Health Institute
Efficient approximation of probability distributions with k-order decomposable models
During the last decades several learning algorithms have been proposed to learn probability distributions based on decomposable models. Some of these algorithms can be used to search for a maximum likelihood decomposable model with a given maximum clique size, k. Unfortunately, the problem of learning a maximum likelihood decomposable model given a maximum clique size is NP-hard for . In this work, we propose the fractal tree family of algorithms which approximates this problem with a computational complexity of in the worst case, where is the number of implied random variables and N is the size of the training set. The fractal tree algorithms construct a sequence of maximal -order decomposable graphs, for in steps. At each step, the algorithms follow a divide-and-conquer strategy that decomposes the problem into a set of separate problems. Each separate problem is efficiently solved using the generalized Chow-Liu algorithm. Fractal trees can be considered a natural extension of the Chow-Liu algorithm, from to arbitrary values of , and they have shown a competitive behavior to deal with the maximum likelihood problem. Due to their competitive behavior, their low computational complexity and their modularity, which allow them to implement different parallelization strategies, the proposed procedures are especially advisable for modeling high dimensional domains
- …