143,918 research outputs found
Operations for Learning with Graphical Models
This paper is a multidisciplinary review of empirical, statistical learning
from a graphical model perspective. Well-known examples of graphical models
include Bayesian networks, directed graphs representing a Markov chain, and
undirected networks representing a Markov field. These graphical models are
extended to model data analysis and empirical learning using the notation of
plates. Graphical operations for simplifying and manipulating a problem are
provided including decomposition, differentiation, and the manipulation of
probability models from the exponential family. Two standard algorithm schemas
for learning are reviewed in a graphical framework: Gibbs sampling and the
expectation maximization algorithm. Using these operations and schemas, some
popular algorithms can be synthesized from their graphical specification. This
includes versions of linear regression, techniques for feed-forward networks,
and learning Gaussian and discrete Bayesian networks from data. The paper
concludes by sketching some implications for data analysis and summarizing how
some popular algorithms fall within the framework presented. The main original
contributions here are the decomposition techniques and the demonstration that
graphical models provide a framework for understanding and developing complex
learning algorithms.Comment: See http://www.jair.org/ for any accompanying file
Neural Graphical Models
Probabilistic Graphical Models are often used to understand dynamics of a
system. They can model relationships between features (nodes) and the
underlying distribution. Theoretically these models can represent very complex
dependency functions, but in practice often simplifying assumptions are made
due to computational limitations associated with graph operations. In this work
we introduce Neural Graphical Models (NGMs) which attempt to represent complex
feature dependencies with reasonable computational costs. Given a graph of
feature relationships and corresponding samples, we capture the dependency
structure between the features along with their complex function
representations by using a neural network as a multi-task learning framework.
We provide efficient learning, inference and sampling algorithms. NGMs can fit
generic graph structures including directed, undirected and mixed-edge graphs
as well as support mixed input data types. We present empirical studies that
show NGMs' capability to represent Gaussian graphical models, perform inference
analysis of a lung cancer data and extract insights from a real world infant
mortality data provided by Centers for Disease Control and Prevention
Exploration of the search space of Gaussian graphical models for paired data
We consider the problem of learning a Gaussian graphical model in the case
where the observations come from two dependent groups sharing the same
variables. We focus on a family of coloured Gaussian graphical models
specifically suited for the paired data problem. Commonly, graphical models are
ordered by the submodel relationship so that the search space is a lattice,
called the model inclusion lattice. We introduce a novel order between models,
named the twin order. We show that, embedded with this order, the model space
is a lattice that, unlike the model inclusion lattice, is distributive.
Furthermore, we provide the relevant rules for the computation of the
neighbours of a model. The latter are more efficient than the same operations
in the model inclusion lattice, and are then exploited to achieve a more
efficient exploration of the search space. These results can be applied to
improve the efficiency of both greedy and Bayesian model search procedures.
Here we implement a stepwise backward elimination procedure and evaluate its
performance by means of simulations. Finally, the procedure is applied to learn
a brain network from fMRI data where the two groups correspond to the left and
right hemispheres, respectively
Learning Latent Tree Graphical Models
We study the problem of learning a latent tree graphical model where samples
are available only from a subset of variables. We propose two consistent and
computationally efficient algorithms for learning minimal latent trees, that
is, trees without any redundant hidden nodes. Unlike many existing methods, the
observed nodes (or variables) are not constrained to be leaf nodes. Our first
algorithm, recursive grouping, builds the latent tree recursively by
identifying sibling groups using so-called information distances. One of the
main contributions of this work is our second algorithm, which we refer to as
CLGrouping. CLGrouping starts with a pre-processing procedure in which a tree
over the observed variables is constructed. This global step groups the
observed nodes that are likely to be close to each other in the true latent
tree, thereby guiding subsequent recursive grouping (or equivalent procedures)
on much smaller subsets of variables. This results in more accurate and
efficient learning of latent trees. We also present regularized versions of our
algorithms that learn latent tree approximations of arbitrary distributions. We
compare the proposed algorithms to other methods by performing extensive
numerical experiments on various latent tree graphical models such as hidden
Markov models and star graphs. In addition, we demonstrate the applicability of
our methods on real-world datasets by modeling the dependency structure of
monthly stock returns in the S&P index and of the words in the 20 newsgroups
dataset
- …