88 research outputs found
Justifying additive-noise-model based causal discovery via algorithmic information theory
A recent method for causal discovery is in many cases able to infer whether X
causes Y or Y causes X for just two observed variables X and Y. It is based on
the observation that there exist (non-Gaussian) joint distributions P(X,Y) for
which Y may be written as a function of X up to an additive noise term that is
independent of X and no such model exists from Y to X. Whenever this is the
case, one prefers the causal model X--> Y.
Here we justify this method by showing that the causal hypothesis Y--> X is
unlikely because it requires a specific tuning between P(Y) and P(X|Y) to
generate a distribution that admits an additive noise model from X to Y. To
quantify the amount of tuning required we derive lower bounds on the
algorithmic information shared by P(Y) and P(X|Y). This way, our justification
is consistent with recent approaches for using algorithmic information theory
for causal reasoning. We extend this principle to the case where P(X,Y) almost
admits an additive noise model.
Our results suggest that the above conclusion is more reliable if the
complexity of P(Y) is high.Comment: 17 pages, 1 Figur
Information-Theoretic Causal Discovery
It is well-known that correlation does not equal causation, but how can we infer causal relations from data? Causal discovery tries to answer precisely this question by rigorously analyzing under which assumptions it is feasible to infer causal networks from passively collected, so-called observational data. Particularly, causal discovery aims to infer a directed graph among a set of observed random variables under assumptions which are as realistic as possible. A key assumption in causal discovery is faithfulness. That is, we assume that separations in the true graph imply independencies in the distribution and vice versa. If faithfulness holds and we have access to a perfect independence oracle, traditional causal discovery approaches can infer the Markov equivalence class of the true causal graph---i.e., infer the correct undirected network and even some of the edge directions. In a real-world setting, faithfulness may be violated, however, and neither do we have access to such an independence oracle. Beyond that, we are interested in inferring the complete DAG structure and not just the Markov equivalence class. To circumvent or at least alleviate these limitations, we take an information-theoretic approach. In the first part of this thesis, we consider violations of faithfulness that can be induced by exclusive or relations or cancelling paths, and develop a weaker faithfulness assumption, called 2-adjacency faithfulness, to detect some of these mechanisms. Further, we analyze under which conditions it is possible to infer the correct DAG structure even if such violations occur. In the second part, we focus on independence testing via conditional mutual information (CMI). CMI is an information-theoretic measure of dependence based on Shannon entropy. We first suggest estimating CMI for discrete variables via normalized maximum likelihood instead of the plug-in maximum likelihood estimator that tends to overestimate dependencies. On top of that, we show that CMI can be consistently estimated for discrete-continuous mixture random variables by simply discretizing the continuous parts of each variable. Last, we consider the problem of distinguishing the two Markov equivalent graphs X to Y and Y to X, which is a necessary step towards discovering all edge directions. To solve this problem, it is inevitable to make assumptions about the generating mechanism. We build upon the idea which states that the cause is algorithmically independent of its mechanism. We propose two methods to approximate this postulate via the Minimum Description Length (MDL) principle: one for univariate numeric data and one for multivariate mixed-type data. Finally, we combine insights from our MDL-based approach and regression-based methods with strong guarantees and show we can identify cause and effect via L0-regularized regression
Bayes Nets and Rationality
Bayes nets are a powerful tool for researchers in statistics and artificial intelligence. This chapter demonstrates that they are also of much use for philosophers and psychologists interested in (Bayesian) rationality. To do so, we outline the general methodology of Bayes nets modeling in rationality research and illustrate it with several examples from the philosophy and psychology of reasoning and argumentation. Along the way, we discuss the normative foundations of Bayes nets modeling and address some of the methodological problems it raises
Bayes Nets and Rationality
Bayes nets are a powerful tool for researchers in statistics and artificial intelligence. This chapter demonstrates that they are also of much use for philosophers and psychologists interested in (Bayesian) rationality. To do so, we outline the general methodology of Bayes nets modeling in rationality research and illustrate it with several examples from the philosophy and psychology of reasoning and argumentation. Along the way, we discuss the normative foundations of Bayes nets modeling and address some of the methodological problems it raises
Structural Agnostic Modeling: Adversarial Learning of Causal Graphs
A new causal discovery method, Structural Agnostic Modeling (SAM), is
presented in this paper. Leveraging both conditional independencies and
distributional asymmetries in the data, SAM aims at recovering full causal
models from continuous observational data along a multivariate non-parametric
setting. The approach is based on a game between players estimating each
variable distribution conditionally to the others as a neural net, and an
adversary aimed at discriminating the overall joint conditional distribution,
and that of the original data. An original learning criterion combining
distribution estimation, sparsity and acyclicity constraints is used to enforce
the end-to-end optimization of the graph structure and parameters through
stochastic gradient descent. Besides the theoretical analysis of the approach
in the large sample limit, SAM is extensively experimentally validated on
synthetic and real data
Integrity Constraints Revisited: From Exact to Approximate Implication
Integrity constraints such as functional dependencies (FD), and multi-valued
dependencies (MVD) are fundamental in database schema design. Likewise,
probabilistic conditional independences (CI) are crucial for reasoning about
multivariate probability distributions. The implication problem studies whether
a set of constraints (antecedents) implies another constraint (consequent), and
has been investigated in both the database and the AI literature, under the
assumption that all constraints hold exactly. However, many applications today
consider constraints that hold only approximately. In this paper we define an
approximate implication as a linear inequality between the degree of
satisfaction of the antecedents and consequent, and we study the relaxation
problem: when does an exact implication relax to an approximate implication? We
use information theory to define the degree of satisfaction, and prove several
results. First, we show that any implication from a set of data dependencies
(MVDs+FDs) can be relaxed to a simple linear inequality with a factor at most
quadratic in the number of variables; when the consequent is an FD, the factor
can be reduced to 1. Second, we prove that there exists an implication between
CIs that does not admit any relaxation; however, we prove that every
implication between CIs relaxes "in the limit". Finally, we show that the
implication problem for differential constraints in market basket analysis also
admits a relaxation with a factor equal to 1. Our results recover, and
sometimes extend, several previously known results about the implication
problem: implication of MVDs can be checked by considering only 2-tuple
relations, and the implication of differential constraints for frequent item
sets can be checked by considering only databases containing a single
transaction
Integrity Constraints Revisited: From Exact to Approximate Implication
Integrity constraints such as functional dependencies (FD), and multi-valued dependencies (MVD) are fundamental in database schema design. Likewise, probabilistic conditional independences (CI) are crucial for reasoning about multivariate probability distributions. The implication problem studies whether a set of constraints (antecedents) implies another constraint (consequent), and has been investigated in both the database and the AI literature, under the assumption that all constraints hold exactly. However, many applications today consider constraints that hold only approximately. In this paper we define an approximate implication as a linear inequality between the degree of satisfaction of the antecedents and consequent, and we study the relaxation problem: when does an exact implication relax to an approximate implication? We use information theory to define the degree of satisfaction, and prove several results. First, we show that any implication from a set of data dependencies (MVDs+FDs) can be relaxed to a simple linear inequality with a factor at most quadratic in the number of variables; when the consequent is an FD, the factor can be reduced to 1. Second, we prove that there exists an implication between CIs that does not admit any relaxation; however, we prove that every implication between CIs relaxes "in the limit". Finally, we show that the implication problem for differential constraints in market basket analysis also admits a relaxation with a factor equal to 1. Our results recover, and sometimes extend, several previously known results about the implication problem: implication of MVDs can be checked by considering only 2-tuple relations, and the implication of differential constraints for frequent item sets can be checked by considering only databases containing a single transaction
Approximate Implication for Probabilistic Graphical Models
The graphical structure of Probabilistic Graphical Models (PGMs) represents
the conditional independence (CI) relations that hold in the modeled
distribution. Every separator in the graph represents a conditional
independence relation in the distribution, making them the vehicle through
which new conditional independencies are inferred and verified. The notion of
separation in graphs depends on whether the graph is directed (i.e., a Bayesian
Network), or undirected (i.e., a Markov Network).
The premise of all current systems-of-inference for deriving CIs in PGMs, is
that the set of CIs used for the construction of the PGM hold exactly. In
practice, algorithms for extracting the structure of PGMs from data discover
approximate CIs that do not hold exactly in the distribution. In this paper, we
ask how the error in this set propagates to the inferred CIs read off the
graphical structure. More precisely, what guarantee can we provide on the
inferred CI when the set of CIs that entailed it hold only approximately? It
has recently been shown that in the general case, no such guarantee can be
provided.
In this work, we prove new negative and positive results concerning this
problem. We prove that separators in undirected PGMs do not necessarily
represent approximate CIs. That is, no guarantee can be provided for CIs
inferred from the structure of undirected graphs. We prove that such a
guarantee exists for the set of CIs inferred in directed graphical models,
making the -separation algorithm a sound and complete system for inferring
approximate CIs. We also establish improved approximation guarantees for
independence relations derived from marginal and saturated CIs.Comment: arXiv admin note: substantial text overlap with arXiv:2105.1446
Distributional Equivalence and Structure Learning for Bow-free Acyclic Path Diagrams
We consider the problem of structure learning for bow-free acyclic path
diagrams (BAPs). BAPs can be viewed as a generalization of linear Gaussian DAG
models that allow for certain hidden variables. We present a first method for
this problem using a greedy score-based search algorithm. We also prove some
necessary and some sufficient conditions for distributional equivalence of BAPs
which are used in an algorithmic ap- proach to compute (nearly) equivalent
model structures. This allows us to infer lower bounds of causal effects. We
also present applications to real and simulated datasets using our publicly
available R-package
- …