Linear-Codes-Based Lossless Joint Source-Channel Coding for Multiple-Access Channels

A general lossless joint source-channel coding (JSCC) scheme based on linear codes and random interleavers for multiple-access channels (MACs) is presented and then analyzed in this paper. By the information-spectrum approach and the code-spectrum approach, it is shown that a linear code with a good joint spectrum can be used to establish limit-approaching lossless JSCC schemes for correlated general sources and general MACs, where the joint spectrum is a generalization of the input-output weight distribution. Some properties of linear codes with good joint spectra are investigated. A formula on the "distance" property of linear codes with good joint spectra is derived, based on which, it is further proved that, the rate of any systematic codes with good joint spectra cannot be larger than the reciprocal of the corresponding alphabet cardinality, and any sparse generator matrices cannot yield linear codes with good joint spectra. The problem of designing arbitrary rate coding schemes is also discussed. A novel idea called "generalized puncturing" is proposed, which makes it possible that one good low-rate linear code is enough for the design of coding schemes with multiple rates. Finally, various coding problems of MACs are reviewed in a unified framework established by the code-spectrum approach, under which, criteria and candidates of good linear codes in terms of spectrum requirements for such problems are clearly presented.Comment: 18 pages, 3 figure

On the Performance of Lossless Joint Source-Channel Coding Based on Linear Codes

A general lossless joint source-channel coding scheme based on linear codes is proposed and then analyzed in this paper. It is shown that a linear code with good joint spectrum can be used to establish limit-approaching joint source-channel coding schemes for arbitrary sources and channels, where the joint spectrum of the code is a generalization of the input-output weight distribution.Comment: To appear in Proc. 2006 IEEE Information Theory Workshop, October 22-26, 2006, Chengdu, China. (5 pages, 2 figures

Network vector quantization

We present an algorithm for designing locally optimal vector quantizers for general networks. We discuss the algorithm's implementation and compare the performance of the resulting "network vector quantizers" to traditional vector quantizers (VQs) and to rate-distortion (R-D) bounds where available. While some special cases of network codes (e.g., multiresolution (MR) and multiple description (MD) codes) have been studied in the literature, we here present a unifying approach that both includes these existing solutions as special cases and provides solutions to previously unsolved examples

Low-Complexity Approaches to Slepian–Wolf Near-Lossless Distributed Data Compression

This paper discusses the Slepian–Wolf problem of distributed near-lossless compression of correlated sources. We introduce practical new tools for communicating at all rates in the achievable region. The technique employs a simple “source-splitting” strategy that does not require common sources of randomness at the encoders and decoders. This approach allows for pipelined encoding and decoding so that the system operates with the complexity of a single user encoder and decoder. Moreover, when this splitting approach is used in conjunction with iterative decoding methods, it produces a significant simplification of the decoding process. We demonstrate this approach for synthetically generated data. Finally, we consider the Slepian–Wolf problem when linear codes are used as syndrome-formers and consider a linear programming relaxation to maximum-likelihood (ML) sequence decoding. We note that the fractional vertices of the relaxed polytope compete with the optimal solution in a manner analogous to that observed when the “min-sum” iterative decoding algorithm is applied. This relaxation exhibits the ML-certificate property: if an integral solution is found, it is the ML solution. For symmetric binary joint distributions, we show that selecting easily constructable “expander”-style low-density parity check codes (LDPCs) as syndrome-formers admits a positive error exponent and therefore provably good performance

Lecture Notes on Network Information Theory

These lecture notes have been converted to a book titled Network Information Theory published recently by Cambridge University Press. This book provides a significantly expanded exposition of the material in the lecture notes as well as problems and bibliographic notes at the end of each chapter. The authors are currently preparing a set of slides based on the book that will be posted in the second half of 2012. More information about the book can be found at http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom