Local Exchangeability

Exchangeability---in which the distribution of an infinite sequence is invariant to reorderings of its elements---implies the existence of a simple conditional independence structure that may be leveraged in the design of probabilistic models, efficient inference algorithms, and randomization-based testing procedures. In practice, however, this assumption is too strong an idealization; the distribution typically fails to be exactly invariant to permutations and de Finetti's representation theory does not apply. Thus there is the need for a distributional assumption that is both weak enough to hold in practice, and strong enough to guarantee a useful underlying representation. We introduce a relaxed notion of local exchangeability---where swapping data associated with nearby covariates causes a bounded change in the distribution. We prove that locally exchangeable processes correspond to independent observations from an underlying measure-valued stochastic process. We thereby show that de Finetti's theorem is robust to perturbation and provide further justification for the Bayesian modelling approach. Using this probabilistic result, we develop three novel statistical procedures for (1) estimating the underlying process via local empirical measures, (2) testing via local randomization, and (3) estimating the canonical premetric of local exchangeability. These three procedures extend the applicability of previous exchangeability-based methods without sacrificing rigorous statistical guarantees. The paper concludes with examples of popular statistical models that exhibit local exchangeability

Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference

Exchangeability is a central notion in statistics and probability theory. The assumption that an infinite sequence of data points is exchangeable is at the core of Bayesian statistics. However, finite exchangeability as a statistical property that renders probabilistic inference tractable is less well-understood. We develop a theory of finite exchangeability and its relation to tractable probabilistic inference. The theory is complementary to that of independence and conditional independence. We show that tractable inference in probabilistic models with high treewidth and millions of variables can be understood using the notion of finite (partial) exchangeability. We also show that existing lifted inference algorithms implicitly utilize a combination of conditional independence and partial exchangeability.Comment: In Proceedings of the 28th AAAI Conference on Artificial Intelligenc

On a representation theorem for finitely exchangeable random vectors

A random vector $X=(X_1,\ldots,X_n)$ with the $X_i$ taking values in an arbitrary measurable space $(S, \mathscr{S})$ is exchangeable if its law is the same as that of $(X_{\sigma(1)}, \ldots, X_{\sigma(n)})$ for any permutation $\sigma$. We give an alternative and shorter proof of the representation result (Jaynes \cite{Jay86} and Kerns and Sz\'ekely \cite{KS06}) stating that the law of $X$ is a mixture of product probability measures with respect to a signed mixing measure. The result is "finitistic" in nature meaning that it is a matter of linear algebra for finite $S$. The passing from finite $S$ to an arbitrary one may pose some measure-theoretic difficulties which are avoided by our proof. The mixing signed measure is not unique (examples are given), but we pay more attention to the one constructed in the proof ("canonical mixing measure") by pointing out some of its characteristics. The mixing measure is, in general, defined on the space of probability measures on $S$, but for $S=\mathbb{R}$, one can choose a mixing measure on $\mathbb{R}^n$.Comment: We here give an alternative proof of the measurability of the random signed-measure underlying the construction. We also add an independent proof of the main algebraic fact used in the paper. Title update