At Every Corner: Determining Corner Points of Two-User Gaussian Interference Channels

The corner points of the capacity region of the two-user Gaussian interference channel under strong or weak interference are determined using the notions of almost Gaussian random vectors, almost lossless addition of random vectors, and almost linearly dependent random vectors. In particular, the "missing" corner point problem is solved in a manner that differs from previous works in that it avoids the use of integration over a continuum of SNR values or of Monge-Kantorovitch transportation problems

A Lower Bound on the Entropy Rate for a Large Class of Stationary Processes and its Relation to the Hyperplane Conjecture

We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the differential entropy rate of such a process and the differential entropy rate of a Gaussian process with the same autocovariance function is bounded. This result is based on a recent result on bounding the Kullback-Leibler divergence by the Wasserstein distance given by Polyanskiy and Wu. Moreover, it is related to the famous hyperplane conjecture, also known as slicing problem, in convex geometry originally stated by J. Bourgain. Based on an entropic formulation of the hyperplane conjecture given by Bobkov and Madiman we discuss the relation of our result to the hyperplane conjecture.Comment: presented at the 2016 IEEE Information Theory Workshop (ITW), Cambridge, U

Maximal power output of a stochastic thermodynamic engine

Classical thermodynamics aimed to quantify the efficiency of thermodynamic engines, by bounding the maximal amount of mechanical energy produced, compared to the amount of heat required. While this was accomplished early on, by Carnot and Clausius, the more practical problem to quantify limits of power that can be delivered, remained elusive due to the fact that quasistatic processes require infinitely slow cycling, resulting in a vanishing power output. Recent insights, drawn from stochastic models, appear to bridge the gap between theory and practice in that they lead to physically meaningful expressions for the dissipation cost in operating a thermodynamic engine over a finite time window. Indeed, the problem to optimize power can be expressed as a stochastic control problem. Building on this framework of stochastic thermodynamics we derive bounds on the maximal power that can be drawn by cycling an overdamped ensemble of particles via a time-varying potential while alternating contact with heat baths of different temperature (Tc cold, and Th hot). Specifically, assuming a suitable bound M on the spatial gradient of the controlling potential, we show that the maximal achievable power is bounded by [Formula presented]. Moreover, we show that this bound can be reached to within a factor of [Formula presented] by operating the cyclic thermodynamic process with a quadratic potential

The Quantum Wasserstein Distance of Order 1

We propose a generalization of the Wasserstein distance of order 1 to the quantum states of $n$ qudits. The proposal recovers the Hamming distance for the vectors of the canonical basis, and more generally the classical Wasserstein distance for quantum states diagonal in the canonical basis. The proposed distance is invariant with respect to permutations of the qudits and unitary operations acting on one qudit and is additive with respect to the tensor product. Our main result is a continuity bound for the von Neumann entropy with respect to the proposed distance, which significantly strengthens the best continuity bound with respect to the trace distance. We also propose a generalization of the Lipschitz constant to quantum observables. The notion of quantum Lipschitz constant allows us to compute the proposed distance with a semidefinite program. We prove a quantum version of Marton's transportation inequality and a quantum Gaussian concentration inequality for the spectrum of quantum Lipschitz Moreover, we derive bounds on the contraction coefficients of shallow quantum circuits and of the tensor product of one-qudit quantum channels with respect to the proposed distance. We discuss other possible applications in quantum machine learning, quantum Shannon theory, and quantum many-body systems

Convergence of Smoothed Empirical Measures with Applications to Entropy Estimation

This paper studies convergence of empirical measures smoothed by a Gaussian kernel. Specifically, consider approximating $P\ast\mathcal{N}_\sigma$, for $\mathcal{N}_\sigma\triangleq\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$, by $\hat{P}_n\ast\mathcal{N}_\sigma$, where $\hat{P}_n$ is the empirical measure, under different statistical distances. The convergence is examined in terms of the Wasserstein distance, total variation (TV), Kullback-Leibler (KL) divergence, and $\chi^2$-divergence. We show that the approximation error under the TV distance and 1-Wasserstein distance ($\mathsf{W}_1$) converges at rate $e^{O(d)}n^{-\frac{1}{2}}$ in remarkable contrast to a typical $n^{-\frac{1}{d}}$ rate for unsmoothed $\mathsf{W}_1$ (and $d\ge 3$). For the KL divergence, squared 2-Wasserstein distance ($\mathsf{W}_2^2$), and $\chi^2$-divergence, the convergence rate is $e^{O(d)}n^{-1}$, but only if $P$ achieves finite input-output $\chi^2$ mutual information across the additive white Gaussian noise channel. If the latter condition is not met, the rate changes to $\omega(n^{-1})$ for the KL divergence and $\mathsf{W}_2^2$, while the $\chi^2$-divergence becomes infinite - a curious dichotomy. As a main application we consider estimating the differential entropy $h(P\ast\mathcal{N}_\sigma)$ in the high-dimensional regime. The distribution $P$ is unknown but $n$ i.i.d samples from it are available. We first show that any good estimator of $h(P\ast\mathcal{N}_\sigma)$ must have sample complexity that is exponential in $d$. Using the empirical approximation results we then show that the absolute-error risk of the plug-in estimator converges at the parametric rate $e^{O(d)}n^{-\frac{1}{2}}$, thus establishing the minimax rate-optimality of the plug-in. Numerical results that demonstrate a significant empirical superiority of the plug-in approach to general-purpose differential entropy estimators are provided.Comment: arXiv admin note: substantial text overlap with arXiv:1810.1158

On the High-SNR Capacity of the Gaussian Interference Channel and New Capacity Bounds

The best outer bound on the capacity region of the two-user Gaussian Interference Channel (GIC) is known to be the intersection of regions of various bounds including genie-aided outer bounds, in which a genie provides noisy input signals to the intended receiver. The Han and Kobayashi (HK) scheme provides the best known inner bound. The rate difference between the best known lower and upper bounds on the sum capacity remains as large as 1 bit per channel use especially around $g^2=P^{-1/3}$, where $P$ is the symmetric power constraint and $g$ is the symmetric real cross-channel coefficient. In this paper, we pay attention to the \emph{moderate interference regime} where $g^2\in (\max(0.086, P^{-1/3}),1)$. We propose a new upper-bounding technique that utilizes noisy observation of interfering signals as genie signals and applies time sharing to the genie signals at the receivers. A conditional version of the worst additive noise lemma is also introduced to derive new capacity bounds. The resulting upper (outer) bounds on the sum capacity (capacity region) are shown to be tighter than the existing bounds in a certain range of the moderate interference regime. Using the new upper bounds and the HK lower bound, we show that $R_\text{sym}^*=\frac{1}{2}\log \big(|g|P+|g|^{-1}(P+1)\big)$ characterizes the capacity of the symmetric real GIC to within $0.104$ bit per channel use in the moderate interference regime at any signal-to-noise ratio (SNR). We further establish a high-SNR characterization of the symmetric real GIC, where the proposed upper bound is at most $0.1$ bit far from a certain HK achievable scheme with Gaussian signaling and time sharing for $g^2\in (0,1]$. In particular, $R_\text{sym}^*$ is achievable at high SNR by the proposed HK scheme and turns out to be the high-SNR capacity at least at $g^2=0.25, 0.5$.Comment: Submitted to IEEE Transactions on Information Theory on June 2015, revised on November 2016, and accepted for publication on Feb. 28, 201