35,904 research outputs found
On Graphical Models via Univariate Exponential Family Distributions
Undirected graphical models, or Markov networks, are a popular class of
statistical models, used in a wide variety of applications. Popular instances
of this class include Gaussian graphical models and Ising models. In many
settings, however, it might not be clear which subclass of graphical models to
use, particularly for non-Gaussian and non-categorical data. In this paper, we
consider a general sub-class of graphical models where the node-wise
conditional distributions arise from exponential families. This allows us to
derive multivariate graphical model distributions from univariate exponential
family distributions, such as the Poisson, negative binomial, and exponential
distributions. Our key contributions include a class of M-estimators to fit
these graphical model distributions; and rigorous statistical analysis showing
that these M-estimators recover the true graphical model structure exactly,
with high probability. We provide examples of genomic and proteomic networks
learned via instances of our class of graphical models derived from Poisson and
exponential distributions.Comment: Journal of Machine Learning Researc
Total positivity in exponential families with application to binary variables
We study exponential families of distributions that are multivariate totally
positive of order 2 (MTP2), show that these are convex exponential families,
and derive conditions for existence of the MLE. Quadratic exponential familes
of MTP2 distributions contain attractive Gaussian graphical models and
ferromagnetic Ising models as special examples. We show that these are defined
by intersecting the space of canonical parameters with a polyhedral cone whose
faces correspond to conditional independence relations. Hence MTP2 serves as an
implicit regularizer for quadratic exponential families and leads to sparsity
in the estimated graphical model. We prove that the maximum likelihood
estimator (MLE) in an MTP2 binary exponential family exists if and only if both
of the sign patterns and are represented in the sample for
every pair of variables; in particular, this implies that the MLE may exist
with observations, in stark contrast to unrestricted binary exponential
families where observations are required. Finally, we provide a novel and
globally convergent algorithm for computing the MLE for MTP2 Ising models
similar to iterative proportional scaling and apply it to the analysis of data
from two psychological disorders
Linear Estimating Equations for Exponential Families with Application to Gaussian Linear Concentration Models
In many families of distributions, maximum likelihood estimation is
intractable because the normalization constant for the density which enters
into the likelihood function is not easily available. The score matching
estimator of Hyv\"arinen (2005) provides an alternative where this
normalization constant is not required. The corresponding estimating equations
become linear for an exponential family. The score matching estimator is shown
to be consistent and asymptotically normally distributed for such models,
although not necessarily efficient. Gaussian linear concentration models are
examples of such families. For linear concentration models that are also linear
in the covariance we show that the score matching estimator is identical to the
maximum likelihood estimator, hence in such cases it is also efficient.
Gaussian graphical models and graphical models with symmetries form
particularly interesting subclasses of linear concentration models and we
investigate the potential use of the score matching estimator for this case
Algebraic Geometry of Quantum Graphical Models
Algebro-geometric methods have proven to be very successful in the study of
graphical models in statistics. In this paper we introduce the foundations to
carry out a similar study of their quantum counterparts. These quantum
graphical models are families of quantum states satisfying certain locality or
correlation conditions encoded by a graph. We lay out several ways to associate
an algebraic variety to a quantum graphical model. The classical graphical
models can be recovered from most of these varieties by restricting to quantum
states represented by diagonal matrices. We study fundamental properties of
these varieties and provide algorithms to compute their defining equations.
Moreover, we study quantum information projections to quantum exponential
families defined by graphs and prove a quantum analogue of Birch's Theorem.Comment: 20 pages, comments welcome
Stratified Gaussian graphical models
Gaussian graphical models represent the backbone of the statistical toolbox for analyzing continuous multivariate systems. However, due to the intrinsic properties of the multivariate normal distribution, use of this model family may hide certain forms of context-specific independence that are natural to consider from an applied perspective. Such independencies have been earlier introduced to generalize discrete graphical models and Bayesian networks into more flexible model families. Here, we adapt the idea of context-specific independence to Gaussian graphical models by introducing a stratification of the Euclidean space such that a conditional independence may hold in certain segments but be absent elsewhere. It is shown that the stratified models define a curved exponential family, which retains considerable tractability for parameter estimation and model selection.Peer reviewe
Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses
We investigate the relationship between the structure of a discrete graphical
model and the support of the inverse of a generalized covariance matrix. We
show that for certain graph structures, the support of the inverse covariance
matrix of indicator variables on the vertices of a graph reflects the
conditional independence structure of the graph. Our work extends results that
have previously been established only in the context of multivariate Gaussian
graphical models, thereby addressing an open question about the significance of
the inverse covariance matrix of a non-Gaussian distribution. The proof
exploits a combination of ideas from the geometry of exponential families,
junction tree theory and convex analysis. These population-level results have
various consequences for graph selection methods, both known and novel,
including a novel method for structure estimation for missing or corrupted
observations. We provide nonasymptotic guarantees for such methods and
illustrate the sharpness of these predictions via simulations.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1162 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
- …