A Packing Lemma for Polar Codes

A packing lemma is proved using a setting where the channel is a binary-input discrete memoryless channel $(\mathcal{X},w(y|x),\mathcal{Y})$, the code is selected at random subject to parity-check constraints, and the decoder is a joint typicality decoder. The ensemble is characterized by (i) a pair of fixed parameters $(H,q)$ where $H$ is a parity-check matrix and $q$ is a channel input distribution and (ii) a random parameter $S$ representing the desired parity values. For a code of length $n$, the constraint is sampled from $p_S(s) = \sum_{x^n\in {\mathcal{X}}^n} \phi(s,x^n)q^n(x^n)$ where $\phi(s,x^n)$ is the indicator function of event $\{s = x^n H^T\}$ and $q^n(x^n) = \prod_{i=1}^nq(x_i)$. Given $S=s$, the codewords are chosen conditionally independently from $p_{X^n|S}(x^n|s) \propto \phi(s,x^n) q^n(x^n)$. It is shown that the probability of error for this ensemble decreases exponentially in $n$ provided the rate $R$ is kept bounded away from $I(X;Y)-\frac{1}{n}I(S;Y^n)$ with $(X,Y)\sim q(x)w(y|x)$ and $(S,Y^n)\sim p_S(s)\sum_{x^n} p_{X^n|S}(x^n|s) \prod_{i=1}^{n} w(y_i|x_i)$. In the special case where $H$ is the parity-check matrix of a standard polar code, it is shown that the rate penalty $\frac{1}{n}I(S;Y^n)$ vanishes as $n$ increases. The paper also discusses the relation between ordinary polar codes and random codes based on polar parity-check matrices.Comment: 5 pages. To be presented at 2015 IEEE International Symposium on Information Theory, June 14-19, 2015, Hong Kong. Minor corrections to v

On privacy amplification, lossy compression, and their duality to channel coding

We examine the task of privacy amplification from information-theoretic and coding-theoretic points of view. In the former, we give a one-shot characterization of the optimal rate of privacy amplification against classical adversaries in terms of the optimal type-II error in asymmetric hypothesis testing. This formulation can be easily computed to give finite-blocklength bounds and turns out to be equivalent to smooth min-entropy bounds by Renner and Wolf [Asiacrypt 2005] and Watanabe and Hayashi [ISIT 2013], as well as a bound in terms of the $E_\gamma$ divergence by Yang, Schaefer, and Poor [arXiv:1706.03866 [cs.IT]]. In the latter, we show that protocols for privacy amplification based on linear codes can be easily repurposed for channel simulation. Combined with known relations between channel simulation and lossy source coding, this implies that privacy amplification can be understood as a basic primitive for both channel simulation and lossy compression. Applied to symmetric channels or lossy compression settings, our construction leads to proto- cols of optimal rate in the asymptotic i.i.d. limit. Finally, appealing to the notion of channel duality recently detailed by us in [IEEE Trans. Info. Theory 64, 577 (2018)], we show that linear error-correcting codes for symmetric channels with quantum output can be transformed into linear lossy source coding schemes for classical variables arising from the dual channel. This explains a "curious duality" in these problems for the (self-dual) erasure channel observed by Martinian and Yedidia [Allerton 2003; arXiv:cs/0408008] and partly anticipates recent results on optimal lossy compression by polar and low-density generator matrix codes.Comment: v3: updated to include equivalence of the converse bound with smooth entropy formulations. v2: updated to include comparison with the one-shot bounds of arXiv:1706.03866. v1: 11 pages, 4 figure

Sphere packing bounds in the Grassmann and Stiefel manifolds

Applying the Riemann geometric machinery of volume estimates in terms of curvature, bounds for the minimal distance of packings/codes in the Grassmann and Stiefel manifolds will be derived and analyzed. In the context of space-time block codes this leads to a monotonically increasing minimal distance lower bound as a function of the block length. This advocates large block lengths for the code design.Comment: Replaced with final version, 11 page

Asymmetric Lee Distance Codes for DNA-Based Storage

We consider a new family of codes, termed asymmetric Lee distance codes, that arise in the design and implementation of DNA-based storage systems and systems with parallel string transmission protocols. The codewords are defined over a quaternary alphabet, although the results carry over to other alphabet sizes; furthermore, symbol confusability is dictated by their underlying binary representation. Our contributions are two-fold. First, we demonstrate that the new distance represents a linear combination of the Lee and Hamming distance and derive upper bounds on the size of the codes under this metric based on linear programming techniques. Second, we propose a number of code constructions which imply lower bounds

Sphere packing bounds via spherical codes

The sphere packing problem asks for the greatest density of a packing of congruent balls in Euclidean space. The current best upper bound in all sufficiently high dimensions is due to Kabatiansky and Levenshtein in 1978. We revisit their argument and improve their bound by a constant factor using a simple geometric argument, and we extend the argument to packings in hyperbolic space, for which it gives an exponential improvement over the previously known bounds. Additionally, we show that the Cohn-Elkies linear programming bound is always at least as strong as the Kabatiansky-Levenshtein bound; this result is analogous to Rodemich's theorem in coding theory. Finally, we develop hyperbolic linear programming bounds and prove the analogue of Rodemich's theorem there as well.Comment: 30 pages, 2 figure

Polar Coding for Secret-Key Generation

Practical implementations of secret-key generation are often based on sequential strategies, which handle reliability and secrecy in two successive steps, called reconciliation and privacy amplification. In this paper, we propose an alternative approach based on polar codes that jointly deals with reliability and secrecy. Specifically, we propose secret-key capacity-achieving polar coding schemes for the following models: (i) the degraded binary memoryless source (DBMS) model with rate-unlimited public communication, (ii) the DBMS model with one-way rate-limited public communication, (iii) the 1-to-m broadcast model and (iv) the Markov tree model with uniform marginals. For models (i) and (ii) our coding schemes remain valid for non-degraded sources, although they may not achieve the secret-key capacity. For models (i), (ii) and (iii), our schemes rely on pre-shared secret seed of negligible rate; however, we provide special cases of these models for which no seed is required. Finally, we show an application of our results to secrecy and privacy for biometric systems. We thus provide the first examples of low-complexity secret-key capacity-achieving schemes that are able to handle vector quantization for model (ii), or multiterminal communication for models (iii) and (iv).Comment: 26 pages, 9 figures, accepted to IEEE Transactions on Information Theory; parts of the results were presented at the 2013 IEEE Information Theory Worksho