140,718 research outputs found
Graphs for margins of Bayesian networks
Directed acyclic graph (DAG) models, also called Bayesian networks, impose
conditional independence constraints on a multivariate probability
distribution, and are widely used in probabilistic reasoning, machine learning
and causal inference. If latent variables are included in such a model, then
the set of possible marginal distributions over the remaining (observed)
variables is generally complex, and not represented by any DAG. Larger classes
of mixed graphical models, which use multiple edge types, have been introduced
to overcome this; however, these classes do not represent all the models which
can arise as margins of DAGs. In this paper we show that this is because
ordinary mixed graphs are fundamentally insufficiently rich to capture the
variety of marginal models.
We introduce a new class of hyper-graphs, called mDAGs, and a latent
projection operation to obtain an mDAG from the margin of a DAG. We show that
each distinct marginal of a DAG model is represented by at least one mDAG, and
provide graphical results towards characterizing when two such marginal models
are the same. Finally we show that mDAGs correctly capture the marginal
structure of causally-interpreted DAGs under interventions on the observed
variables
Neural Graphical Models
Probabilistic Graphical Models are often used to understand dynamics of a
system. They can model relationships between features (nodes) and the
underlying distribution. Theoretically these models can represent very complex
dependency functions, but in practice often simplifying assumptions are made
due to computational limitations associated with graph operations. In this work
we introduce Neural Graphical Models (NGMs) which attempt to represent complex
feature dependencies with reasonable computational costs. Given a graph of
feature relationships and corresponding samples, we capture the dependency
structure between the features along with their complex function
representations by using a neural network as a multi-task learning framework.
We provide efficient learning, inference and sampling algorithms. NGMs can fit
generic graph structures including directed, undirected and mixed-edge graphs
as well as support mixed input data types. We present empirical studies that
show NGMs' capability to represent Gaussian graphical models, perform inference
analysis of a lung cancer data and extract insights from a real world infant
mortality data provided by Centers for Disease Control and Prevention
Learning mixed graphical models with separate sparsity parameters and stability-based model selection
Background: Mixed graphical models (MGMs) are graphical models learned over a combination of continuous and discrete variables. Mixed variable types are common in biomedical datasets. MGMs consist of a parameterized joint probability density, which implies a network structure over these heterogeneous variables. The network structure reveals direct associations between the variables and the joint probability density allows one to ask arbitrary probabilistic questions on the data. This information can be used for feature selection, classification and other important tasks. Results: We studied the properties of MGM learning and applications of MGMs to high-dimensional data (biological and simulated). Our results show that MGMs reliably uncover the underlying graph structure, and when used for classification, their performance is comparable to popular discriminative methods (lasso regression and support vector machines). We also show that imposing separate sparsity penalties for edges connecting different types of variables significantly improves edge recovery performance. To choose these sparsity parameters, we propose a new efficient model selection method, named Stable Edge-specific Penalty Selection (StEPS). StEPS is an expansion of an earlier method, StARS, to mixed variable types. In terms of edge recovery, StEPS selected MGMs outperform those models selected using standard techniques, including AIC, BIC and cross-validation. In addition, we use a heuristic search that is linear in size of the sparsity value search space as opposed to the cubic grid search required by other model selection methods. We applied our method to clinical and mRNA expression data from the Lung Genomics Research Consortium (LGRC) and the learned MGM correctly recovered connections between the diagnosis of obstructive or interstitial lung disease, two diagnostic breathing tests, and cigarette smoking history. Our model also suggested biologically relevant mRNA markers that are linked to these three clinical variables. Conclusions: MGMs are able to accurately recover dependencies between sets of continuous and discrete variables in both simulated and biomedical datasets. Separation of sparsity penalties by edge type is essential for accurate network edge recovery. Furthermore, our stability based method for model selection determines sparsity parameters faster and more accurately (in terms of edge recovery) than other model selection methods. With the ongoing availability of comprehensive clinical and biomedical datasets, MGMs are expected to become a valuable tool for investigating disease mechanisms and answering an array of critical healthcare questions
- …