Design and Analysis of Nonbinary LDPC Codes for Arbitrary Discrete-Memoryless Channels

We present an analysis, under iterative decoding, of coset LDPC codes over GF(q), designed for use over arbitrary discrete-memoryless channels (particularly nonbinary and asymmetric channels). We use a random-coset analysis to produce an effect that is similar to output-symmetry with binary channels. We show that the random selection of the nonzero elements of the GF(q) parity-check matrix induces a permutation-invariance property on the densities of the decoder messages, which simplifies their analysis and approximation. We generalize several properties, including symmetry and stability from the analysis of binary LDPC codes. We show that under a Gaussian approximation, the entire q-1 dimensional distribution of the vector messages is described by a single scalar parameter (like the distributions of binary LDPC messages). We apply this property to develop EXIT charts for our codes. We use appropriately designed signal constellations to obtain substantial shaping gains. Simulation results indicate that our codes outperform multilevel codes at short block lengths. We also present simulation results for the AWGN channel, including results within 0.56 dB of the unconstrained Shannon limit (i.e. not restricted to any signal constellation) at a spectral efficiency of 6 bits/s/Hz.Comment: To appear, IEEE Transactions on Information Theory, (submitted October 2004, revised and accepted for publication, November 2005). The material in this paper was presented in part at the 41st Allerton Conference on Communications, Control and Computing, October 2003 and at the 2005 IEEE International Symposium on Information Theor

Decoding communities in networks

According to a recent information-theoretical proposal, the problem of defining and identifying communities in networks can be interpreted as a classical communication task over a noisy channel: memberships of nodes are information bits erased by the channel, edges and non-edges in the network are parity bits introduced by the encoder but degraded through the channel, and a community identification algorithm is a decoder. The interpretation is perfectly equivalent to the one at the basis of well-known statistical inference algorithms for community detection. The only difference in the interpretation is that a noisy channel replaces a stochastic network model. However, the different perspective gives the opportunity to take advantage of the rich set of tools of coding theory to generate novel insights on the problem of community detection. In this paper, we illustrate two main applications of standard coding-theoretical methods to community detection. First, we leverage a state-of-the-art decoding technique to generate a family of quasi-optimal community detection algorithms. Second and more important, we show that the Shannon's noisy-channel coding theorem can be invoked to establish a lower bound, here named as decodability bound, for the maximum amount of noise tolerable by an ideal decoder to achieve perfect detection of communities. When computed for well-established synthetic benchmarks, the decodability bound explains accurately the performance achieved by the best community detection algorithms existing on the market, telling us that only little room for their improvement is still potentially left.Comment: 9 pages, 5 figures + Appendi

Parameters of Combinatorial Neural Codes

Motivated by recent developments in the mathematical theory of neural codes, we study the structure of error-correcting codes for the binary asymmetric channel. These are also known as combinatorial neural codes and can be seen as the discrete version of neural receptive field codes. We introduce two notions of discrepancy between binary vectors, which are not metric functions in general but nonetheless capture the mathematics of the binary asymmetric channel. In turn, these lead to two new fundamental parameters of combinatorial neural codes, both of which measure the probability that the maximum likelihood decoder fails. We then derive various bounds for the cardinality and weight distribution of a combinatorial neural code in terms of these new parameters, giving examples of codes meeting the bounds with equality

Error Correcting Coding for a Non-symmetric Ternary Channel

Ternary channels can be used to model the behavior of some memory devices, where information is stored in three different levels. In this paper, error correcting coding for a ternary channel where some of the error transitions are not allowed, is considered. The resulting channel is non-symmetric, therefore classical linear codes are not optimal for this channel. We define the maximum-likelihood (ML) decoding rule for ternary codes over this channel and show that it is complex to compute, since it depends on the channel error probability. A simpler alternative decoding rule which depends only on code properties, called \da-decoding, is then proposed. It is shown that \da-decoding and ML decoding are equivalent, i.e., \da-decoding is optimal, under certain conditions. Assuming \da-decoding, we characterize the error correcting capabilities of ternary codes over the non-symmetric ternary channel. We also derive an upper bound and a constructive lower bound on the size of codes, given the code length and the minimum distance. The results arising from the constructive lower bound are then compared, for short sizes, to optimal codes (in terms of code size) found by a clique-based search. It is shown that the proposed construction method gives good codes, and that in some cases the codes are optimal.Comment: Submitted to IEEE Transactions on Information Theory. Part of this work was presented at the Information Theory and Applications Workshop 200

Construction of lattices for communications and security

In this thesis, we propose a new class of lattices based on polar codes, namely polar lattices. Polar lattices enjoy explicit construction and provable goodness for the additive white Gaussian noise (AWGN) channel, \textit{i.e.}, they are \emph{AWGN-good} lattices, in the sense that the error probability (for infinite lattice coding) vanishes for any fixed volume-to-noise ratio (VNR) greater than $2\pi e$. Our construction is based on the multilevel approach of Forney \textit{et al.}, where on each level we construct a capacity-achieving polar code. We show the component polar codes are naturally nested, thereby fulfilling the requirement of the multilevel lattice construction. We present a more precise analysis of the VNR of the resultant lattice, which is upper-bounded in terms of the flatness factor and the capacity losses of the component codes. The proposed polar lattices are efficiently decodable by using multi-stage decoding. Design examples are presented to demonstrate the superior performance of polar lattices. However, there is no infinite lattice coding in the practical applications. We need to apply the power constraint on the polar lattices which generates the polar lattice codes. We prove polar lattice codes can achieve the capacity \frac{1}{2}\log(1+\SNR) of the power-constrained AWGN channel with a novel shaping scheme. The main idea is that by implementing the lattice Gaussian distribution over the AWGN-good polar lattices, the maximum error-free transmission rate of the resultant coding scheme can be arbitrarily close to the capacity \frac{1}{2}\log(1+\SNR). The shaping technique is based on discrete lattice Gaussian distribution, which leads to a binary asymmetric channel at each level for the multilevel lattice codes. Then it is straightforward to employ multilevel asymmetric polar codes which is a combination of polar lossless source coding and polar channel coding. The construction of polar codes for an asymmetric channel can be converted to that for a related symmetric channel, and it turns out that this symmetric channel is equivalent to an minimum mean-square error (MMSE) scaled $\Lambda/\Lambda'$ channel in lattice coding in terms of polarization, which eventually simplifies our coding design. Finally, we investigate the application of polar lattices in physical layer security. Polar lattice codes are proved to be able to achieve the strong secrecy capacity of the Mod-$\Lambda$ AWGN wiretap channel. The Mod-$\Lambda$ assumption was due to the fact that a practical shaping scheme aiming to achieve the optimum shaping gain was missing. In this thesis, we use our shaping scheme and extend polar lattice coding to the Gaussian wiretap channel. By employing the polar coding technique for asymmetric channels, we manage to construct an AWGN-good lattice and a secrecy-good lattice with optimal shaping simultaneously. Then we prove the resultant wiretap coding scheme can achieve the strong secrecy capacity for the Gaussian wiretap channel.Open Acces

Information-Theoretic Foundations of Mismatched Decoding

Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical considerations such as channel uncertainty and implementation constraints rule out the use of an optimal decoder. The mismatched decoding problem addresses such scenarios by considering the case that the decoder cannot be optimized, but is instead fixed as part of the problem statement. This problem is not only of direct interest in its own right, but also has close connections with other long-standing theoretical problems in information theory. In this monograph, we survey both classical literature and recent developments on the mismatched decoding problem, with an emphasis on achievable random-coding rates for memoryless channels. We present two widely-considered achievable rates known as the generalized mutual information (GMI) and the LM rate, and overview their derivations and properties. In addition, we survey several improved rates via multi-user coding techniques, as well as recent developments and challenges in establishing upper bounds on the mismatch capacity, and an analogous mismatched encoding problem in rate-distortion theory. Throughout the monograph, we highlight a variety of applications and connections with other prominent information theory problems.Comment: Published in Foundations and Trends in Communications and Information Theory (Volume 17, Issue 2-3