Disintegration and Bayesian Inversion via String Diagrams

The notions of disintegration and Bayesian inversion are fundamental in conditional probability theory. They produce channels, as conditional probabilities, from a joint state, or from an already given channel (in opposite direction). These notions exist in the literature, in concrete situations, but are presented here in abstract graphical formulations. The resulting abstract descriptions are used for proving basic results in conditional probability theory. The existence of disintegration and Bayesian inversion is discussed for discrete probability, and also for measure-theoretic probability --- via standard Borel spaces and via likelihoods. Finally, the usefulness of disintegration and Bayesian inversion is illustrated in several examples.Comment: Accepted for publication in Mathematical Structures in Computer Scienc

Searching for the Causal Structure of a Vector Autoregression

Vector autoregressions (VARs) are economically interpretable only when identified by being transformed into a structural form (the SVAR) in which the contemporaneous variables stand in a well-defined causal order. These identifying transformations are not unique. It is widely believed that practitioners must choose among them using a priori theory or other criteria not rooted in the data under analysis. We show how to apply graph-theoretic methods of searching for causal structure based on relations of conditional independence to select among the possible causal orders – or at least to reduce the admissible causal orders to a narrow equivalence class. The graph-theoretic approaches were developed by computer scientists and philosophers (Pearl, Glymour, Spirtes among others) and applied to cross-sectional data. We provide an accessible introduction to this work. Then building on the work of Swanson and Granger (1997), we show how to apply it to searching for the causal order of an SVAR. We present simulation results to show how the efficacy of the search method algorithm varies with signal strength for realistic sample lengths. Our findings suggest that graph-theoretic methods may prove to be a useful tool in the analysis of SVARs.search, causality, structural vector autoregression, graph theory, common cause, causal Markov condition, Wold causal order, identification; PC algorithm

Picturing classical and quantum Bayesian inference

We introduce a graphical framework for Bayesian inference that is sufficiently general to accommodate not just the standard case but also recent proposals for a theory of quantum Bayesian inference wherein one considers density operators rather than probability distributions as representative of degrees of belief. The diagrammatic framework is stated in the graphical language of symmetric monoidal categories and of compact structures and Frobenius structures therein, in which Bayesian inversion boils down to transposition with respect to an appropriate compact structure. We characterize classical Bayesian inference in terms of a graphical property and demonstrate that our approach eliminates some purely conventional elements that appear in common representations thereof, such as whether degrees of belief are represented by probabilities or entropic quantities. We also introduce a quantum-like calculus wherein the Frobenius structure is noncommutative and show that it can accommodate Leifer's calculus of `conditional density operators'. The notion of conditional independence is also generalized to our graphical setting and we make some preliminary connections to the theory of Bayesian networks. Finally, we demonstrate how to construct a graphical Bayesian calculus within any dagger compact category.Comment: 38 pages, lots of picture

Structural Agnostic Modeling: Adversarial Learning of Causal Graphs

A new causal discovery method, Structural Agnostic Modeling (SAM), is presented in this paper. Leveraging both conditional independencies and distributional asymmetries in the data, SAM aims at recovering full causal models from continuous observational data along a multivariate non-parametric setting. The approach is based on a game between $d$ players estimating each variable distribution conditionally to the others as a neural net, and an adversary aimed at discriminating the overall joint conditional distribution, and that of the original data. An original learning criterion combining distribution estimation, sparsity and acyclicity constraints is used to enforce the end-to-end optimization of the graph structure and parameters through stochastic gradient descent. Besides the theoretical analysis of the approach in the large sample limit, SAM is extensively experimentally validated on synthetic and real data