On Convergence Properties of Shannon Entropy

Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential entropies. A general result for the desired differential entropy convergence is provided, taking into account both compactly and uncompactly supported densities. Convergence of differential entropy is also characterized in terms of the Kullback-Liebler discriminant for densities with fairly general supports, and it is shown that convergence in variation of probability measures guarantees such convergence under an appropriate boundedness condition on the densities involved. Results for the discrete setting are also provided, allowing for infinitely supported probability measures, by taking advantage of the equivalence between weak convergence and convergence in variation in this setting.Comment: Submitted to IEEE Transactions on Information Theor

Capacity Sensitivity in Additive Non-Gaussian Noise Channels

International audienceIn this paper, a new framework based on the notion of \textit{capacity sensitivity} is introduced to study the capacity of continuous memoryless point-to-point channels. The capacity sensitivity reflects how the capacity changes with small perturbations in any of the parameters describing the channel, even when the capacity is not available in closed-form. This includes perturbations of the cost constraints on the input distribution as well as on the channel distribution. The framework is based on continuity of the capacity, which is shown for a class of perturbations in the cost constraint and the channel distribution. The continuity then forms the foundation for obtaining bounds on the capacity sensitivity. As an illustration, the capacity sensitivity bound is applied to obtain scaling laws when the support of additive α-stable noise is truncated

Sensibilité de capacité dans les canaux continus

In this research report, a new framework based on the notion of capacity sensitivity is introduced to study the capacity of continuous memoryless point-to-point channels. The capacity sensitivity reflects how the capacity changes with small perturbations in any of the parameters describing the channel, even when the capacity is not available in closed-form. This includes perturbations of the cost constraints on the input distribution as well as on the channel distribution. The framework is based on continuity of the capacity, which is shown for a class of perturbations in the cost constraint and the channel distribution. The continuity then forms the foundation for obtaining bounds on the capacity sensitivity. As an illustration, the capacity sensitivity bound is applied to obtain scaling laws when the support of additive $\alpha$-stable noise is truncated.Dans ce rapport de recherche, un nouveau cadre basé sur la notion de la sensibilité de capacité est présenté afin d'étudier la capacité des canaux point-à-point continus sans mémoire. La sensibilité de capacité reflète comment la capacité varie en fonctions des petites perturbations de l'un des paramètres décrivant le canal, même si l'expression explicite de la capacité n'est pas connue. Cela inclut les perturbations des contraintes de coût sur la distribution en entrée du canal ainsi que sur la distribution du canal. Ce cadre est basé sur la continuité de la capacité, qui est démontrée pour une classe de perturbations des contraintes de coût et la distribution du canal. La continuité forme ainsi la base pour obtenir les bornes sur la sensibilité de la capacité. Pour illustrer tout ça, la borne sur sensibilité de la capacité est appliquée afin d'obtenir des lois de mise à l'échelle quand le support du bruit additif $\alpha$-stable est tronqué