Entropic Inequalities and Marginal Problems

A marginal problem asks whether a given family of marginal distributions for some set of random variables arises from some joint distribution of these variables. Here we point out that the existence of such a joint distribution imposes non-trivial conditions already on the level of Shannon entropies of the given marginals. These entropic inequalities are necessary (but not sufficient) criteria for the existence of a joint distribution. For every marginal problem, a list of such Shannon-type entropic inequalities can be calculated by Fourier-Motzkin elimination, and we offer a software interface to a Fourier-Motzkin solver for doing so. For the case that the hypergraph of given marginals is a cycle graph, we provide a complete analytic solution to the problem of classifying all relevant entropic inequalities, and use this result to bound the decay of correlations in stochastic processes. Furthermore, we show that Shannon-type inequalities for differential entropies are not relevant for continuous-variable marginal problems; non-Shannon-type inequalities are, both in the discrete and in the continuous case. In contrast to other approaches, our general framework easily adapts to situations where one has additional (conditional) independence requirements on the joint distribution, as in the case of graphical models. We end with a list of open problems. A complementary article discusses applications to quantum nonlocality and contextuality.Comment: 26 pages, 3 figure

About the analogy between optimal transport and minimal entropy

We describe some analogy between optimal transport and the Schr\"odinger problem where the transport cost is replaced by an entropic cost with a reference path measure. A dual Kantorovich type formulation and a Benamou-Brenier type representation formula of the entropic cost are derived, as well as contraction inequalities with respect to the entropic cost. This analogy is also illustrated with some numerical examples where the reference path measure is given by the Brownian or the Ornstein-Uhlenbeck process. Our point of view is measure theoretical and the relative entropy with respect to path measures plays a prominent role

An entropic approach to local realism and noncontextuality

For any Bell locality scenario (or Kochen-Specker noncontextuality scenario), the joint Shannon entropies of local (or noncontextual) models define a convex cone for which the non-trivial facets are tight entropic Bell (or contextuality) inequalities. In this paper we explore this entropic approach and derive tight entropic inequalities for various scenarios. One advantage of entropic inequalities is that they easily adapt to situations like bilocality scenarios, which have additional independence requirements that are non-linear on the level of probabilities, but linear on the level of entropies. Another advantage is that, despite the nonlinearity, taking detection inefficiencies into account turns out to be very simple. When joint measurements are conducted by a single detector only, the detector efficiency for witnessing quantum contextuality can be arbitrarily low.Comment: 12 pages, 8 figures, minor mistakes correcte

Entropic Bell inequalities

We derive entropic Bell inequalities from considering entropy Venn diagrams. These entropic inequalities, akin to the Braunstein-Caves inequalities, are violated for a quantum-mechanical Einstein-Podolsky-Rosen pair, which implies that the conditional entropies of Bell variables must be negative in this case. This suggests that the satisfaction of entropic Bell inequalities is equivalent to the non-negativity of conditional entropies as a necessary condition for separability

The Inflation Technique for Causal Inference with Latent Variables

The problem of causal inference is to determine if a given probability distribution on observed variables is compatible with some causal structure. The difficult case is when the causal structure includes latent variables. We here introduce the $\textit{inflation technique}$ for tackling this problem. An inflation of a causal structure is a new causal structure that can contain multiple copies of each of the original variables, but where the ancestry of each copy mirrors that of the original. To every distribution of the observed variables that is compatible with the original causal structure, we assign a family of marginal distributions on certain subsets of the copies that are compatible with the inflated causal structure. It follows that compatibility constraints for the inflation can be translated into compatibility constraints for the original causal structure. Even if the constraints at the level of inflation are weak, such as observable statistical independences implied by disjoint causal ancestry, the translated constraints can be strong. We apply this method to derive new inequalities whose violation by a distribution witnesses that distribution's incompatibility with the causal structure (of which Bell inequalities and Pearl's instrumental inequality are prominent examples). We describe an algorithm for deriving all such inequalities for the original causal structure that follow from ancestral independences in the inflation. For three observed binary variables with pairwise common causes, it yields inequalities that are stronger in at least some aspects than those obtainable by existing methods. We also describe an algorithm that derives a weaker set of inequalities but is more efficient. Finally, we discuss which inflations are such that the inequalities one obtains from them remain valid even for quantum (and post-quantum) generalizations of the notion of a causal model.Comment: Minor final corrections, updated to match the published version as closely as possibl