31 research outputs found
Yet Another Proof of the Entropy Power Inequality
Yet another simple proof of the entropy power inequality is given, which
avoids both the integration over a path of Gaussian perturbation and the use of
Young's inequality with sharp constant or R\'enyi entropies. The proof is based
on a simple change of variables, is formally identical in one and several
dimensions, and easily settles the equality case
Entropy-constrained scalar quantization with a lossy-compressed bit
We consider the compression of a continuous real-valued source X using scalar quantizers and average squared error distortion D. Using lossless compression of the quantizer's output, Gish and Pierce showed that uniform quantizing yields the smallest output entropy in the limit D -> 0, resulting in a rate penalty of 0.255 bits/sample above the Shannon Lower Bound (SLB). We present a scalar quantization scheme named lossy-bit entropy-constrained scalar quantization (Lb-ECSQ) that is able to reduce the D -> 0 gap to SLB to 0.251 bits/sample by combining both lossless and binary lossy compression of the quantizer's output. We also study the low-resolution regime and show that Lb-ECSQ significantly outperforms ECSQ in the case of 1-bit quantization.The authors wish to thank Tobias Koch and Gonzalo Vázquez Vilar for fruitful discussions and helpful comments to the manuscript. This work has been supported in part by the European Union 7th Framework Programme through the Marie Curie Initial Training Network “Machine Learning for Personalized Medicine” MLPM2012, Grant No. 316861, by the Spanish Ministry of Economy and Competitiveness and Ministry of Education under grants TEC2016-78434-C3-3-R (MINECO/FEDER, EU) and IJCI-2014-19150, and by Comunidad de Madrid (project ’CASI-CAM-CM’, id. S2013/ICE-2845).Publicad