88,734 research outputs found
The Libra Toolkit for Probabilistic Models
The Libra Toolkit is a collection of algorithms for learning and inference
with discrete probabilistic models, including Bayesian networks, Markov
networks, dependency networks, and sum-product networks. Compared to other
toolkits, Libra places a greater emphasis on learning the structure of
tractable models in which exact inference is efficient. It also includes a
variety of algorithms for learning graphical models in which inference is
potentially intractable, and for performing exact and approximate inference.
Libra is released under a 2-clause BSD license to encourage broad use in
academia and industry
Unbiased Learning of Deep Generative Models with Structured Discrete Representations
By composing graphical models with deep learning architectures, we learn
generative models with the strengths of both frameworks. The structured
variational autoencoder (SVAE) inherits structure and interpretability from
graphical models, and flexible likelihoods for high-dimensional data from deep
learning, but poses substantial optimization challenges. We propose novel
algorithms for learning SVAEs, and are the first to demonstrate the SVAE's
ability to handle multimodal uncertainty when data is missing by incorporating
discrete latent variables. Our memory-efficient implicit differentiation scheme
makes the SVAE tractable to learn via gradient descent, while demonstrating
robustness to incomplete optimization. To more rapidly learn accurate graphical
model parameters, we derive a method for computing natural gradients without
manual derivations, which avoids biases found in prior work. These optimization
innovations enable the first comparisons of the SVAE to state-of-the-art time
series models, where the SVAE performs competitively while learning
interpretable and structured discrete data representations.Comment: 38 pages, 7 figure
Structure learning of antiferromagnetic Ising models
In this paper we investigate the computational complexity of learning the
graph structure underlying a discrete undirected graphical model from i.i.d.
samples. We first observe that the notoriously difficult problem of learning
parities with noise can be captured as a special case of learning graphical
models. This leads to an unconditional computational lower bound of for learning general graphical models on nodes of maximum degree
, for the class of so-called statistical algorithms recently introduced by
Feldman et al (2013). The lower bound suggests that the runtime
required to exhaustively search over neighborhoods cannot be significantly
improved without restricting the class of models.
Aside from structural assumptions on the graph such as it being a tree,
hypertree, tree-like, etc., many recent papers on structure learning assume
that the model has the correlation decay property. Indeed, focusing on
ferromagnetic Ising models, Bento and Montanari (2009) showed that all known
low-complexity algorithms fail to learn simple graphs when the interaction
strength exceeds a number related to the correlation decay threshold. Our
second set of results gives a class of repelling (antiferromagnetic) models
that have the opposite behavior: very strong interaction allows efficient
learning in time . We provide an algorithm whose performance
interpolates between and depending on the strength of the
repulsion.Comment: 15 pages. NIPS 201
Maximum Persistency via Iterative Relaxed Inference with Graphical Models
We consider the NP-hard problem of MAP-inference for undirected discrete
graphical models. We propose a polynomial time and practically efficient
algorithm for finding a part of its optimal solution. Specifically, our
algorithm marks some labels of the considered graphical model either as (i)
optimal, meaning that they belong to all optimal solutions of the inference
problem; (ii) non-optimal if they provably do not belong to any solution. With
access to an exact solver of a linear programming relaxation to the
MAP-inference problem, our algorithm marks the maximal possible (in a specified
sense) number of labels. We also present a version of the algorithm, which has
access to a suboptimal dual solver only and still can ensure the
(non-)optimality for the marked labels, although the overall number of the
marked labels may decrease. We propose an efficient implementation, which runs
in time comparable to a single run of a suboptimal dual solver. Our method is
well-scalable and shows state-of-the-art results on computational benchmarks
from machine learning and computer vision.Comment: Reworked version, submitted to PAM
Spectral Methods for Learning Multivariate Latent Tree Structure
This work considers the problem of learning the structure of multivariate
linear tree models, which include a variety of directed tree graphical models
with continuous, discrete, and mixed latent variables such as linear-Gaussian
models, hidden Markov models, Gaussian mixture models, and Markov evolutionary
trees. The setting is one where we only have samples from certain observed
variables in the tree, and our goal is to estimate the tree structure (i.e.,
the graph of how the underlying hidden variables are connected to each other
and to the observed variables). We propose the Spectral Recursive Grouping
algorithm, an efficient and simple bottom-up procedure for recovering the tree
structure from independent samples of the observed variables. Our finite sample
size bounds for exact recovery of the tree structure reveal certain natural
dependencies on underlying statistical and structural properties of the
underlying joint distribution. Furthermore, our sample complexity guarantees
have no explicit dependence on the dimensionality of the observed variables,
making the algorithm applicable to many high-dimensional settings. At the heart
of our algorithm is a spectral quartet test for determining the relative
topology of a quartet of variables from second-order statistics
- …