Large Deviations Behavior of the Logarithmic Error Probability of Random Codes

This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which coincide in some cases. In addition, the probability of randomly drawing a codebook whose error exponent is larger than the TRC exponent is shown to be double–exponentially small; upper and lower bounds to the double–exponential exponent are given. The results suggest that codebooks whose error exponent is larger than the error exponent of the TRC are extremely rare. The key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables

Geometry of Information Integration

Information geometry is used to quantify the amount of information integration within multiple terminals of a causal dynamical system. Integrated information quantifies how much information is lost when a system is split into parts and information transmission between the parts is removed. Multiple measures have been proposed as a measure of integrated information. Here, we analyze four of the previously proposed measures and elucidate their relations from a viewpoint of information geometry. Two of them use dually flat manifolds and the other two use curved manifolds to define a split model. We show that there are hierarchical structures among the measures. We provide explicit expressions of these measures

Subset sum phase transitions and data compression

We propose a rigorous analysis approach for the subset sum problem in the context of lossless data compression, where the phase transition of the subset sum problem is directly related to the passage between ambiguous and non-ambiguous decompression, for a compression scheme that is based on specifying the sequence composition. The proposed analysis lends itself to straightforward extensions in several directions of interest, including non-binary alphabets, incorporation of side information at the decoder (Slepian-Wolf coding), and coding schemes based on multiple subset sums. It is also demonstrated that the proposed technique can be used to analyze the critical behavior in a more involved situation where the sequence composition is not specified by the encoder.Comment: 14 pages, submitted to the Journal of Statistical Mechanics: Theory and Experimen

Expurgated random-coding ensembles: Exponents, refinements, and connections

This paper studies expurgated random-coding bounds and exponents with a given (possibly suboptimal) decoding rule. Variations of Gallager’s analysis are presented, yielding new asymptotic and non-asymptotic bounds on the error probability for an arbitrary codeword distribution. A simple non-asymptotic bound is shown to attain an exponent which coincides with that of Csiszár and Körner for discrete alphabets, while also remaining valid for continuous alphabets. The method of type class enumeration is studied for both discrete and continuous alphabets, and it is shown that this approach yields improved exponents for some codeword distributions. A refined analysis of expurgated i.i.d. random prefactor, thus improving on Gallager’s O(1) prefactor. coding is given which yields an exponent with a O ( 1 √n I

Bose--Einstein Condensation in the Large Deviations Regime with Applications to Information System Models

We study the large deviations behavior of systems that admit a certain form of a product distribution, which is frequently encountered both in Physics and in various information system models. First, to fix ideas, we demonstrate a simple calculation of the large deviations rate function for a single constraint (event). Under certain conditions, the behavior of this function is shown to exhibit an analogue of Bose--Einstein condensation (BEC). More interestingly, we also study the large deviations rate function associated with two constraints (and the extension to any number of constraints is conceptually straightforward). The phase diagram of this rate function is shown to exhibit as many as seven phases, and it suggests a two--dimensional generalization of the notion of BEC (or more generally, a multi--dimensional BEC). While the results are illustrated for a simple model, the underlying principles are actually rather general. We also discuss several applications and implications pertaining to information system models

Quantum Hypothesis Testing and Non-Equilibrium Statistical Mechanics

We extend the mathematical theory of quantum hypothesis testing to the general $W^*$-algebraic setting and explore its relation with recent developments in non-equilibrium quantum statistical mechanics. In particular, we relate the large deviation principle for the full counting statistics of entropy flow to quantum hypothesis testing of the arrow of time.Comment: 60 page

Non differentiable large-deviation functionals in boundary-driven diffusive systems

We study the probability of arbitrary density profiles in conserving diffusive fields which are driven by the boundaries. We demonstrate the existence of singularities in the large-deviation functional, the direct analog of the free-energy in non-equilibrium systems. These singularities are unique to non-equilibrium systems and are a direct consequence of the breaking of time-reversal symmetry. This is demonstrated in an exactly-solvable model and also in numerical simulations on a boundary-driven Ising model. We argue that this singular behavior is expected to occur in models where the compressibility has a deep enough minimum. The mechanism is explained using a simple model.Comment: 5 pages, 3 figure