Properties of Classical and Quantum Jensen-Shannon Divergence

Jensen-Shannon divergence (JD) is a symmetrized and smoothed version of the most important divergence measure of information theory, Kullback divergence. As opposed to Kullback divergence it determines in a very direct way a metric; indeed, it is the square of a metric. We consider a family of divergence measures (JD_alpha for alpha>0), the Jensen divergences of order alpha, which generalize JD as JD_1=JD. Using a result of Schoenberg, we prove that JD_alpha is the square of a metric for alpha lies in the interval (0,2], and that the resulting metric space of probability distributions can be isometrically embedded in a real Hilbert space. Quantum Jensen-Shannon divergence (QJD) is a symmetrized and smoothed version of quantum relative entropy and can be extended to a family of quantum Jensen divergences of order alpha (QJD_alpha). We strengthen results by Lamberti et al. by proving that for qubits and pure states, QJD_alpha^1/2 is a metric space which can be isometrically embedded in a real Hilbert space when alpha lies in the interval (0,2]. In analogy with Burbea and Rao's generalization of JD, we also define general QJD by associating a Jensen-type quantity to any weighted family of states. Appropriate interpretations of quantities introduced are discussed and bounds are derived in terms of the total variation and trace distance.Comment: 13 pages, LaTeX, expanded contents, added references and corrected typo

Data Set Models and Exponential Families in Statistical Physics and Beyond

The exponential family of models is defined in a general setting, not relying on probability theory. Some results of information geometry are shown to remain valid. Exponential families both of classical and of quantum mechanical statistical physics fit into the new formalism. Other less obvious applications are predicted. For instance, quantum states can be modeled as points in a classical phase space and the resulting model belongs to the exponential family

Continuity of the Maximum-Entropy Inference

We study the inverse problem of inferring the state of a finite-level quantum system from expected values of a fixed set of observables, by maximizing a continuous ranking function. We have proved earlier that the maximum-entropy inference can be a discontinuous map from the convex set of expected values to the convex set of states because the image contains states of reduced support, while this map restricts to a smooth parametrization of a Gibbsian family of fully supported states. Here we prove for arbitrary ranking functions that the inference is continuous up to boundary points. This follows from a continuity condition in terms of the openness of the restricted linear map from states to their expected values. The openness condition shows also that ranking functions with a discontinuous inference are typical. Moreover it shows that the inference is continuous in the restriction to any polytope which implies that a discontinuity belongs to the quantum domain of non-commutative observables and that a geodesic closure of a Gibbsian family equals the set of maximum-entropy states. We discuss eight descriptions of the set of maximum-entropy states with proofs of accuracy and an analysis of deviations.Comment: 34 pages, 1 figur

Basic concepts, identities and inequalities – the Toolkit of Information Theory

entrop

Maximum entropy fundamentals

entrop