1,817 research outputs found
Learning Bounded Treewidth Bayesian Networks with Thousands of Variables
We present a method for learning treewidth-bounded Bayesian networks from
data sets containing thousands of variables. Bounding the treewidth of a
Bayesian greatly reduces the complexity of inferences. Yet, being a global
property of the graph, it considerably increases the difficulty of the learning
process. We propose a novel algorithm for this task, able to scale to large
domains and large treewidths. Our novel approach consistently outperforms the
state of the art on data sets with up to ten thousand variables
On the Relationship between Sum-Product Networks and Bayesian Networks
In this paper, we establish some theoretical connections between Sum-Product
Networks (SPNs) and Bayesian Networks (BNs). We prove that every SPN can be
converted into a BN in linear time and space in terms of the network size. The
key insight is to use Algebraic Decision Diagrams (ADDs) to compactly represent
the local conditional probability distributions at each node in the resulting
BN by exploiting context-specific independence (CSI). The generated BN has a
simple directed bipartite graphical structure. We show that by applying the
Variable Elimination algorithm (VE) to the generated BN with ADD
representations, we can recover the original SPN where the SPN can be viewed as
a history record or caching of the VE inference process. To help state the
proof clearly, we introduce the notion of {\em normal} SPN and present a
theoretical analysis of the consistency and decomposability properties. We
conclude the paper with some discussion of the implications of the proof and
establish a connection between the depth of an SPN and a lower bound of the
tree-width of its corresponding BN.Comment: Full version of the same paper to appear at ICML-201
Efficient Algorithms for Bayesian Network Parameter Learning from Incomplete Data
We propose an efficient family of algorithms to learn the parameters of a
Bayesian network from incomplete data. In contrast to textbook approaches such
as EM and the gradient method, our approach is non-iterative, yields closed
form parameter estimates, and eliminates the need for inference in a Bayesian
network. Our approach provides consistent parameter estimates for missing data
problems that are MCAR, MAR, and in some cases, MNAR. Empirically, our approach
is orders of magnitude faster than EM (as our approach requires no inference).
Given sufficient data, we learn parameters that can be orders of magnitude more
accurate
On the Complexity of Counterfactual Reasoning
We study the computational complexity of counterfactual reasoning in relation
to the complexity of associational and interventional reasoning on structural
causal models (SCMs). We show that counterfactual reasoning is no harder than
associational or interventional reasoning on fully specified SCMs in the
context of two computational frameworks. The first framework is based on the
notion of treewidth and includes the classical variable elimination and
jointree algorithms. The second framework is based on the more recent and
refined notion of causal treewidth which is directed towards models with
functional dependencies such as SCMs. Our results are constructive and based on
bounding the (causal) treewidth of twin networks -- used in standard
counterfactual reasoning that contemplates two worlds, real and imaginary -- to
the (causal) treewidth of the underlying SCM structure. In particular, we show
that the latter (causal) treewidth is no more than twice the former plus one.
Hence, if associational or interventional reasoning is tractable on a fully
specified SCM then counterfactual reasoning is tractable too. We extend our
results to general counterfactual reasoning that requires contemplating more
than two worlds and discuss applications of our results to counterfactual
reasoning with a partially specified SCM that is coupled with data. We finally
present empirical results that measure the gap between the complexities of
counterfactual reasoning and associational/interventional reasoning on random
SCMs.Comment: An earlier version of this paper appeared in NeurIPS 2022 workshop,
"A causal view on dynamical systems.
Exact Inference Techniques for the Analysis of Bayesian Attack Graphs
Attack graphs are a powerful tool for security risk assessment by analysing
network vulnerabilities and the paths attackers can use to compromise network
resources. The uncertainty about the attacker's behaviour makes Bayesian
networks suitable to model attack graphs to perform static and dynamic
analysis. Previous approaches have focused on the formalization of attack
graphs into a Bayesian model rather than proposing mechanisms for their
analysis. In this paper we propose to use efficient algorithms to make exact
inference in Bayesian attack graphs, enabling the static and dynamic network
risk assessments. To support the validity of our approach we have performed an
extensive experimental evaluation on synthetic Bayesian attack graphs with
different topologies, showing the computational advantages in terms of time and
memory use of the proposed techniques when compared to existing approaches.Comment: 14 pages, 15 figure
Parameter Learning of Logic Programs for Symbolic-Statistical Modeling
We propose a logical/mathematical framework for statistical parameter
learning of parameterized logic programs, i.e. definite clause programs
containing probabilistic facts with a parameterized distribution. It extends
the traditional least Herbrand model semantics in logic programming to
distribution semantics, possible world semantics with a probability
distribution which is unconditionally applicable to arbitrary logic programs
including ones for HMMs, PCFGs and Bayesian networks. We also propose a new EM
algorithm, the graphical EM algorithm, that runs for a class of parameterized
logic programs representing sequential decision processes where each decision
is exclusive and independent. It runs on a new data structure called support
graphs describing the logical relationship between observations and their
explanations, and learns parameters by computing inside and outside probability
generalized for logic programs. The complexity analysis shows that when
combined with OLDT search for all explanations for observations, the graphical
EM algorithm, despite its generality, has the same time complexity as existing
EM algorithms, i.e. the Baum-Welch algorithm for HMMs, the Inside-Outside
algorithm for PCFGs, and the one for singly connected Bayesian networks that
have been developed independently in each research field. Learning experiments
with PCFGs using two corpora of moderate size indicate that the graphical EM
algorithm can significantly outperform the Inside-Outside algorithm
- …