Entropy, Optimization and Counting

In this paper we study the problem of computing max-entropy distributions over a discrete set of objects subject to observed marginals. Interest in such distributions arises due to their applicability in areas such as statistical physics, economics, biology, information theory, machine learning, combinatorics and, more recently, approximation algorithms. A key difficulty in computing max-entropy distributions has been to show that they have polynomially-sized descriptions. We show that such descriptions exist under general conditions. Subsequently, we show how algorithms for (approximately) counting the underlying discrete set can be translated into efficient algorithms to (approximately) compute max-entropy distributions. In the reverse direction, we show how access to algorithms that compute max-entropy distributions can be used to count, which establishes an equivalence between counting and computing max-entropy distributions

Computation of Microcanonical Entropy at Fixed Magnetization Without Direct Counting

We discuss a method to compute the microcanonical entropy at fixed magnetization without direct counting. Our approach is based on the evaluation of a saddle-point leading to an optimization problem. The method is applied to a benchmark Ising model with simultaneous presence of mean-field and nearest-neighbour interactions for which direct counting is indeed possible, thus allowing a comparison. Moreover, we apply the method to an Ising model with mean-field, nearest-neighbour and next-nearest-neighbour interactions, for which direct counting is not straightforward. For this model, we compare the solution obtained by our method with the one obtained from the formula for the entropy in terms of all correlation functions. This example shows that for general couplings our method is much more convenient than direct counting methods to compute the microcanonical entropy at fixed magnetization

Entropy of unimodular Lattice Triangulations

Triangulations are important objects of study in combinatorics, finite element simulations and quantum gravity, where its entropy is crucial for many physical properties. Due to their inherent complex topological structure even the number of possible triangulations is unknown for large systems. We present a novel algorithm for an approximate enumeration which is based on calculations of the density of states using the Wang-Landau flat histogram sampling. For triangulations on two-dimensional integer lattices we achive excellent agreement with known exact numbers of small triangulations as well as an improvement of analytical calculated asymptotics. The entropy density is $C=2.196(3)$ consistent with rigorous upper and lower bounds. The presented numerical scheme can easily be applied to other counting and optimization problems.Comment: 6 pages, 7 figure

Guessing probability distributions from small samples

We propose a new method for the calculation of the statistical properties, as e.g. the entropy, of unknown generators of symbolic sequences. The probability distribution $p(k)$ of the elements $k$ of a population can be approximated by the frequencies $f(k)$ of a sample provided the sample is long enough so that each element $k$ occurs many times. Our method yields an approximation if this precondition does not hold. For a given $f(k)$ we recalculate the Zipf--ordered probability distribution by optimization of the parameters of a guessed distribution. We demonstrate that our method yields reliable results.Comment: 10 pages, uuencoded compressed PostScrip

Estimating the Spectrum in Computed Tomography Via Kullback–Leibler Divergence Constrained Optimization

Purpose We study the problem of spectrum estimation from transmission data of a known phantom. The goal is to reconstruct an x‐ray spectrum that can accurately model the x‐ray transmission curves and reflects a realistic shape of the typical energy spectra of the CT system. Methods Spectrum estimation is posed as an optimization problem with x‐ray spectrum as unknown variables, and a Kullback–Leibler (KL)‐divergence constraint is employed to incorporate prior knowledge of the spectrum and enhance numerical stability of the estimation process. The formulated constrained optimization problem is convex and can be solved efficiently by use of the exponentiated‐gradient (EG) algorithm. We demonstrate the effectiveness of the proposed approach on the simulated and experimental data. The comparison to the expectation–maximization (EM) method is also discussed. Results In simulations, the proposed algorithm is seen to yield x‐ray spectra that closely match the ground truth and represent the attenuation process of x‐ray photons in materials, both included and not included in the estimation process. In experiments, the calculated transmission curve is in good agreement with the measured transmission curve, and the estimated spectra exhibits physically realistic looking shapes. The results further show the comparable performance between the proposed optimization‐based approach and EM. Conclusions Our formulation of a constrained optimization provides an interpretable and flexible framework for spectrum estimation. Moreover, a KL‐divergence constraint can include a prior spectrum and appears to capture important features of x‐ray spectrum, allowing accurate and robust estimation of x‐ray spectrum in CT imaging

Measurement-induced disturbances and nonclassical correlations of Gaussian states

We study quantum correlations beyond entanglement in two-mode Gaussian states of continuous variable systems, by means of the measurement-induced disturbance (MID) and its ameliorated version (AMID). In analogy with the recent studies of the Gaussian quantum discord, we define a Gaussian AMID by constraining the optimization to all bi-local Gaussian positive operator valued measurements. We solve the optimization explicitly for relevant families of states, including squeezed thermal states. Remarkably, we find that there is a finite subset of two-mode Gaussian states, comprising pure states, where non-Gaussian measurements such as photon counting are globally optimal for the AMID and realize a strictly smaller state disturbance compared to the best Gaussian measurements. However, for the majority of two--mode Gaussian states the unoptimized MID provides a loose overestimation of the actual content of quantum correlations, as evidenced by its comparison with Gaussian discord. This feature displays strong similarity with the case of two qubits. Upper and lower bounds for the Gaussian AMID at fixed Gaussian discord are identified. We further present a comparison between Gaussian AMID and Gaussian entanglement of formation, and classify families of two-mode states in terms of their Gaussian AMID, Gaussian discord, and Gaussian entanglement of formation. Our findings provide a further confirmation of the genuinely quantum nature of general Gaussian states, yet they reveal that non-Gaussian measurements can play a crucial role for the optimized extraction and potential exploitation of classical and nonclassical correlations in Gaussian states.Comment: 16 pages, 5 figures; new results added; to appear in Phys. Rev.