On the Reliability Function of Variable-Rate Slepian-Wolf Coding

The reliability function of variable-rate Slepian-Wolf coding is linked to the reliability function of channel coding with constant composition codes, through which computable lower and upper bounds are derived. The bounds coincide at rates close to the Slepian-Wolf limit, yielding a complete characterization of the reliability function in that rate regime. It is shown that variable-rate Slepian-Wolf codes can significantly outperform fixed-rate Slepian-Wolf codes in terms of rate-error tradeoff. The reliability function of variable-rate Slepian-Wolf coding with rate below the Slepian-Wolf limit is determined. In sharp contrast with fixed-rate Slepian-Wolf codes for which the correct decoding probability decays to zero exponentially fast if the rate is below the Slepian-Wolf limit, the correct decoding probability of variable-rate Slepian-Wolf codes can be bounded away from zero.Comment: This is an old manuscript written in 2007-2008 based on our 2007 Allerton conference paper with the same titl

Improved Source Coding Exponents via Witsenhausen's Rate

We provide a novel upper-bound on Witsenhausen's rate, the rate required in the zero-error analogue of the Slepian-Wolf problem; our bound is given in terms of a new information-theoretic functional defined on a certain graph. We then use the functional to give a single letter lower-bound on the error exponent for the Slepian-Wolf problem under the vanishing error probability criterion, where the decoder has full (i.e. unencoded) side information. Our exponent stems from our new encoding scheme which makes use of source distribution only through the positions of the zeros in the `channel' matrix connecting the source with the side information, and in this sense is `semi-universal'. We demonstrate that our error exponent can beat the `expurgated' source-coding exponent of Csisz\'{a}r and K\"{o}rner, achievability of which requires the use of a non-universal maximum-likelihood decoder. An extension of our scheme to the lossy case (i.e. Wyner-Ziv) is given. For the case when the side information is a deterministic function of the source, the exponent of our improved scheme agrees with the sphere-packing bound exactly (thus determining the reliability function). An application of our functional to zero-error channel capacity is also given.Comment: 24 pages, 4 figures. Submitted to IEEE Trans. Info. Theory (Jan 2010

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

On Binary Distributed Hypothesis Testing

We consider the problem of distributed binary hypothesis testing of two sequences that are generated by an i.i.d. doubly-binary symmetric source. Each sequence is observed by a different terminal. The two hypotheses correspond to different levels of correlation between the two source components, i.e., the crossover probability between the two. The terminals communicate with a decision function via rate-limited noiseless links. We analyze the tradeoff between the exponential decay of the two error probabilities associated with the hypothesis test and the communication rates. We first consider the side-information setting where one encoder is allowed to send the full sequence. For this setting, previous work exploits the fact that a decoding error of the source does not necessarily lead to an erroneous decision upon the hypothesis. We provide improved achievability results by carrying out a tighter analysis of the effect of binning error; the results are also more complete as they cover the full exponent tradeoff and all possible correlations. We then turn to the setting of symmetric rates for which we utilize Korner-Marton coding to generalize the results, with little degradation with respect to the performance with a one-sided constraint (side-information setting)

On the Key Generation Rate of Physically Unclonable Functions

In this paper, an algebraic binning based coding scheme and its associated achievable rate for key generation using physically unclonable functions (PUFs) is determined. This achievable rate is shown to be optimal under the generated-secret (GS) model for PUFs. Furthermore, a polar code based polynomial-time encoding and decoding scheme that achieves this rate is also presented

Wireless Information-Theoretic Security - Part II: Practical Implementation

In Part I of this two-part paper on confidential communication over wireless channels, we studied the fundamental security limits of quasi-static fading channels from the point of view of outage secrecy capacity with perfect and imperfect channel state information. In Part II, we develop a practical secret key agreement protocol for Gaussian and quasi-static fading wiretap channels. The protocol uses a four-step procedure to secure communications: establish common randomness via an opportunistic transmission, perform message reconciliation, establish a common key via privacy amplification, and use of the key. We introduce a new reconciliation procedure that uses multilevel coding and optimized low density parity check codes which in some cases comes close to achieving the secrecy capacity limits established in Part I. Finally, we develop new metrics for assessing average secure key generation rates and show that our protocol is effective in secure key renewal.Comment: 25 pages, 11 figures, submitted to Special Issue of IEEE Trans. on Info. Theory on Information Theoretic Securit

Simulation of a Channel with Another Channel

In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provide infeasibility and achievability results for simulation of a binary channel from another binary channel in the case of no shared randomness. To do this, we use properties of R\'enyi capacity of a given order. We also introduce a notion of "channel diameter" which is shown to be additive and satisfy a data processing inequality.Comment: 31 pages, 10 figures, and some parts of this work were published at ITW 201

An Information-Spectrum Approach to Weak Variable-Length Source Coding with Side-Information

This paper studies variable-length (VL) source coding of general sources with side-information. Novel one-shot coding theorems for coding with common side-information available at the encoder and the decoder and Slepian- Wolf (SW) coding (i.e., with side-information only at the decoder) are given, and then, are applied to asymptotic analyses of these coding problems. Especially, a general formula for the infimum of the coding rate asymptotically achievable by weak VL-SW coding (i.e., VL-SW coding with vanishing error probability) is derived. Further, the general formula is applied to investigating weak VL-SW coding of mixed sources. Our results derive and extend several known results on SW coding and weak VL coding, e.g., the optimal achievable rate of VL-SW coding for mixture of i.i.d. sources is given for countably infinite alphabet case with mild condition. In addition, the usefulness of the encoder side-information is investigated. Our result shows that if the encoder side-information is useless in weak VL coding then it is also useless even in the case where the error probability may be positive asymptotically.Comment: 54 pages, 2 figur

Secrecy Amplification for Distributed Encrypted Sources with Correlated Keys using Affine Encoders

This paper proposed the application of post-encryption-compression (PEC) to strengthen the secrecy in the case of distributed encryption where the encryption keys are correlated to each other. We derive the universal code construction for the compression and the rate region where codes with achievability and secrecy are obtainable. Our main technique is to use affine encoders which are constructed from certain linear encoders to encode the ciphertexts before sending them to public communication channels. We show that if the rates of linear codes are within a certain rate region:(1) information leakage on the original sources from the encoded ciphertexts without the keys is negligible, while (2) one who has legitimate keys is able to retrieve the original source data with negligible error probability.Comment: An extended abstract is submitted to ISIT 2018. This is the full version. arXiv admin note: text overlap with arXiv:1801.0256

Key agreement over a 3-receiver broadcast channel

In this paper, we consider the problem of secret key agreement in state-dependent 3-receiver broadcast channels. In the proposed model, there are two legitimate receivers, an eavesdropper and a transmitter where the channel state information is non-causally available at the transmitter. We consider two setups. In the first setup, the transmitter tries to agree on a common key with the legitimate receivers while keeping it concealed from the eavesdropper. Simultaneously, the transmitter agrees on a private key with each of the legitimate receivers that needs to be kept secret from the other legitimate receiver and the eavesdropper. For this setup, we derive inner and outer bounds on the secret key capacity region. In the second setup, we assume that a backward public channel is available among the receivers and the transmitter. Each legitimate receiver wishes to share a private key with the transmitter. For this setup, an inner bound on the private key capacity region is found. Furthermore, the capacity region of the secret key in the state-dependent wiretap channel can be deduced from our inner and outer bounds.Comment: Accepted in IWCIT 201