Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding

This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two coding schemes: variable-length posterior matching and MaxEJS coding scheme which is based on a greedy maximization of the EJS divergence. As an asymptotic corollary of the main results, this paper also provides a rate-reliability test. Variable-length coding schemes that satisfy the condition(s) of the test for parameters $R$ and $E$, are guaranteed to achieve rate $R$ and error exponent $E$. The results are specialized for posterior matching and MaxEJS to obtain deterministic one-phase coding schemes achieving capacity and optimal error exponent. For the special case of symmetric binary-input channels, simpler deterministic schemes of optimal performance are proposed and analyzed.Comment: 17 pages (two-column), 4 figures, to appear in IEEE Transactions on Information Theor

Information Rates of ASK-Based Molecular Communication in Fluid Media

This paper studies the capacity of molecular communications in fluid media, where the information is encoded in the number of transmitted molecules in a time-slot (amplitude shift keying). The propagation of molecules is governed by random Brownian motion and the communication is in general subject to inter-symbol interference (ISI). We first consider the case where ISI is negligible and analyze the capacity and the capacity per unit cost of the resulting discrete memoryless molecular channel and the effect of possible practical constraints, such as limitations on peak and/or average number of transmitted molecules per transmission. In the case with a constrained peak molecular emission, we show that as the time-slot duration increases, the input distribution achieving the capacity per channel use transitions from binary inputs to a discrete uniform distribution. In this paper, we also analyze the impact of ISI. Crucially, we account for the correlation that ISI induces between channel output symbols. We derive an upper bound and two lower bounds on the capacity in this setting. Using the input distribution obtained by an extended Blahut-Arimoto algorithm, we maximize the lower bounds. Our results show that, over a wide range of parameter values, the bounds are close.Comment: 31 pages, 8 figures, Accepted for publication on IEEE Transactions on Molecular, Biological, and Multi-Scale Communication

Models and information-theoretic bounds for nanopore sequencing

Nanopore sequencing is an emerging new technology for sequencing DNA, which can read long fragments of DNA (~50,000 bases) in contrast to most current short-read sequencing technologies which can only read hundreds of bases. While nanopore sequencers can acquire long reads, the high error rates (20%-30%) pose a technical challenge. In a nanopore sequencer, a DNA is migrated through a nanopore and current variations are measured. The DNA sequence is inferred from this observed current pattern using an algorithm called a base-caller. In this paper, we propose a mathematical model for the "channel" from the input DNA sequence to the observed current, and calculate bounds on the information extraction capacity of the nanopore sequencer. This model incorporates impairments like (non-linear) inter-symbol interference, deletions, as well as random response. These information bounds have two-fold application: (1) The decoding rate with a uniform input distribution can be used to calculate the average size of the plausible list of DNA sequences given an observed current trace. This bound can be used to benchmark existing base-calling algorithms, as well as serving a performance objective to design better nanopores. (2) When the nanopore sequencer is used as a reader in a DNA storage system, the storage capacity is quantified by our bounds

A New Upperbound for the Oblivious Transfer Capacity of Discrete Memoryless Channels

We derive a new upper bound on the string oblivious transfer capacity of discrete memoryless channels. The main tool we use is the tension region of a pair of random variables introduced in Prabhakaran and Prabhakaran (2014) where it was used to derive upper bounds on rates of secure sampling in the source model. In this paper, we consider secure computation of string oblivious transfer in the channel model. Our bound is based on a monotonicity property of the tension region in the channel model. We show that our bound strictly improves upon the upper bound of Ahlswede and Csisz\'ar (2013).Comment: 7 pages, 3 figures, extended version of submission to IEEE Information Theory Workshop, 201

A General Formula for the Mismatch Capacity

The fundamental limits of channels with mismatched decoding are addressed. A general formula is established for the mismatch capacity of a general channel, defined as a sequence of conditional distributions with a general decoding metrics sequence. We deduce an identity between the Verd\'{u}-Han general channel capacity formula, and the mismatch capacity formula applied to Maximum Likelihood decoding metric. Further, several upper bounds on the capacity are provided, and a simpler expression for a lower bound is derived for the case of a non-negative decoding metric. The general formula is specialized to the case of finite input and output alphabet channels with a type-dependent metric. The closely related problem of threshold mismatched decoding is also studied, and a general expression for the threshold mismatch capacity is obtained. As an example of threshold mismatch capacity, we state a general expression for the erasures-only capacity of the finite input and output alphabet channel. We observe that for every channel there exists a (matched) threshold decoder which is capacity achieving. Additionally, necessary and sufficient conditions are stated for a channel to have a strong converse. Csisz\'{a}r and Narayan's conjecture is proved for bounded metrics, providing a positive answer to the open problem introduced in [1], i.e., that the "product-space" improvement of the lower random coding bound, $C_q^{(\infty)}(W)$, is indeed the mismatch capacity of the discrete memoryless channel $W$. We conclude by presenting an identity between the threshold capacity and $C_q^{(\infty)}(W)$ in the DMC case

A Tight Upper Bound for the Third-Order Asymptotics for Most Discrete Memoryless Channels

This paper shows that the logarithm of the epsilon-error capacity (average error probability) for n uses of a discrete memoryless channel is upper bounded by the normal approximation plus a third-order term that does not exceed 1/2 log n + O(1) if the epsilon-dispersion of the channel is positive. This matches a lower bound by Y. Polyanskiy (2010) for discrete memoryless channels with positive reverse dispersion. If the epsilon-dispersion vanishes, the logarithm of the epsilon-error capacity is upper bounded by the n times the capacity plus a constant term except for a small class of DMCs and epsilon >= 1/2.Comment: published versio