Exchangeable Variable Models

A sequence of random variables is exchangeable if its joint distribution is invariant under variable permutations. We introduce exchangeable variable models (EVMs) as a novel class of probabilistic models whose basic building blocks are partially exchangeable sequences, a generalization of exchangeable sequences. We prove that a family of tractable EVMs is optimal under zero-one loss for a large class of functions, including parity and threshold functions, and strictly subsumes existing tractable independence-based model families. Extensive experiments show that EVMs outperform state of the art classifiers such as SVMs and probabilistic models which are solely based on independence assumptions.Comment: ICML 201

How to be an imprecise impermissivist

Rational credence should be coherent in the sense that your attitudes should not leave you open to a sure loss. Rational credence should be such that you can learn when confronted with relevant evidence. Rational credence should not be sensitive to irrelevant differences in the presentation of the epistemic situation. We explore the extent to which orthodox probabilistic approaches to rational credence can satisfy these three desiderata and find them wanting. We demonstrate that an imprecise probability approach does better. Along the way we shall demonstrate that the problem of “belief inertia” is not an issue for a large class of IP credences, and provide a solution to van Fraassen’s box factory puzzle

A Generalization of Hierarchical Exchangeability on Trees to Directed Acyclic Graphs

Motivated by the problem of designing inference-friendly Bayesian nonparametric models in probabilistic programming languages, we introduce a general class of partially exchangeable random arrays which generalizes the notion of hierarchical exchangeability introduced in Austin and Panchenko (2014). We say that our partially exchangeable arrays are DAG-exchangeable since their partially exchangeable structure is governed by a collection of Directed Acyclic Graphs. More specifically, such a random array is indexed by $\mathbb{N}^{|V|}$ for some DAG $G=(V,E)$, and its exchangeability structure is governed by the edge set $E$. We prove a representation theorem for such arrays which generalizes the Aldous-Hoover and Austin-Panchenko representation theorems.Comment: 35 pages, 10 figures. Accepted version before re-formattin

Probabilistic logic as a unified framework for inference

I argue that a probabilistic logical language incorporates all the features of deductive, inductive, and abductive inference with the exception of how to generate hypotheses ex nihilo. In the context of abduction, it leads to the Bayes theorem for confirming hypotheses, and naturally captures the theoretical virtue of quantitative parsimony. I address common criticisms against this approach, including how to assign probabilities to sentences, the problem of the catch-all hypothesis, and the problem of auxiliary hypotheses. Finally, I make a tentative argument that mathematical deduction fits in the same probabilistic framework as a deterministic limiting case