1,474 research outputs found
Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses
We investigate the relationship between the structure of a discrete graphical
model and the support of the inverse of a generalized covariance matrix. We
show that for certain graph structures, the support of the inverse covariance
matrix of indicator variables on the vertices of a graph reflects the
conditional independence structure of the graph. Our work extends results that
have previously been established only in the context of multivariate Gaussian
graphical models, thereby addressing an open question about the significance of
the inverse covariance matrix of a non-Gaussian distribution. The proof
exploits a combination of ideas from the geometry of exponential families,
junction tree theory and convex analysis. These population-level results have
various consequences for graph selection methods, both known and novel,
including a novel method for structure estimation for missing or corrupted
observations. We provide nonasymptotic guarantees for such methods and
illustrate the sharpness of these predictions via simulations.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1162 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
High-dimensional learning of linear causal networks via inverse covariance estimation
We establish a new framework for statistical estimation of directed acyclic
graphs (DAGs) when data are generated from a linear, possibly non-Gaussian
structural equation model. Our framework consists of two parts: (1) inferring
the moralized graph from the support of the inverse covariance matrix; and (2)
selecting the best-scoring graph amongst DAGs that are consistent with the
moralized graph. We show that when the error variances are known or estimated
to close enough precision, the true DAG is the unique minimizer of the score
computed using the reweighted squared l_2-loss. Our population-level results
have implications for the identifiability of linear SEMs when the error
covariances are specified up to a constant multiple. On the statistical side,
we establish rigorous conditions for high-dimensional consistency of our
two-part algorithm, defined in terms of a "gap" between the true DAG and the
next best candidate. Finally, we demonstrate that dynamic programming may be
used to select the optimal DAG in linear time when the treewidth of the
moralized graph is bounded.Comment: 41 pages, 7 figure
Active Learning for Undirected Graphical Model Selection
This paper studies graphical model selection, i.e., the problem of estimating
a graph of statistical relationships among a collection of random variables.
Conventional graphical model selection algorithms are passive, i.e., they
require all the measurements to have been collected before processing begins.
We propose an active learning algorithm that uses junction tree representations
to adapt future measurements based on the information gathered from prior
measurements. We prove that, under certain conditions, our active learning
algorithm requires fewer scalar measurements than any passive algorithm to
reliably estimate a graph. A range of numerical results validate our theory and
demonstrates the benefits of active learning.Comment: AISTATS 201
Graphical Modeling for High Dimensional Data
With advances in science and information technologies, many scientific fields are able to meet the challenges of managing and analyzing high-dimensional data. A so-called large p small n problem arises when the number of experimental units, n, is equal to or smaller than the number of features, p. A methodology based on probability and graph theory, termed graphical models, is applied to study the structure and inference of such high-dimensional data
Probabilistic Independence Networks for Hidden Markov Probability Models
Graphical techniques for modeling the dependencies of randomvariables have been explored in a variety of different areas includingstatistics, statistical physics, artificial intelligence, speech recognition, image processing, and genetics.Formalisms for manipulating these models have been developedrelatively independently in these research communities. In this paper weexplore hidden Markov models (HMMs) and related structures within the general framework of probabilistic independencenetworks (PINs). The paper contains a self-contained review of the basic principles of PINs.It is shown that the well-known forward-backward (F-B) and Viterbialgorithms for HMMs are special cases of more general inference algorithms forarbitrary PINs. Furthermore, the existence of inference and estimationalgorithms for more general graphical models provides a set of analysistools for HMM practitioners who wish to explore a richer class of HMMstructures.Examples of relatively complex models to handle sensorfusion and coarticulationin speech recognitionare introduced and treated within the graphical model framework toillustrate the advantages of the general approach
- …