Optimal Feedback Communication via Posterior Matching

In this paper we introduce a fundamental principle for optimal communication over general memoryless channels in the presence of noiseless feedback, termed posterior matching. Using this principle, we devise a (simple, sequential) generic feedback transmission scheme suitable for a large class of memoryless channels and input distributions, achieving any rate below the corresponding mutual information. This provides a unified framework for optimal feedback communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that the Horstein scheme indeed attains the BSC capacity, settling a longstanding conjecture. We further provide closed form expressions for the error probability of the scheme over a range of rates, and derive the achievable rates in a mismatch setting where the scheme is designed according to the wrong channel model. Several illustrative examples of the posterior matching scheme for specific channels are given, and the corresponding error probability expressions are evaluated. The proof techniques employed utilize novel relations between information rates and contraction properties of iterated function systems.Comment: IEEE Transactions on Information Theor

Information capacity in the weak-signal approximation

We derive an approximate expression for mutual information in a broad class of discrete-time stationary channels with continuous input, under the constraint of vanishing input amplitude or power. The approximation describes the input by its covariance matrix, while the channel properties are described by the Fisher information matrix. This separation of input and channel properties allows us to analyze the optimality conditions in a convenient way. We show that input correlations in memoryless channels do not affect channel capacity since their effect decreases fast with vanishing input amplitude or power. On the other hand, for channels with memory, properly matching the input covariances to the dependence structure of the noise may lead to almost noiseless information transfer, even for intermediate values of the noise correlations. Since many model systems described in mathematical neuroscience and biophysics operate in the high noise regime and weak-signal conditions, we believe, that the described results are of potential interest also to researchers in these areas.Comment: 11 pages, 4 figures; accepted for publication in Physical Review

Capacity of Molecular Channels with Imperfect Particle-Intensity Modulation and Detection

This work introduces the particle-intensity channel (PIC) as a model for molecular communication systems and characterizes the properties of the optimal input distribution and the capacity limits for this system. In the PIC, the transmitter encodes information, in symbols of a given duration, based on the number of particles released, and the receiver detects and decodes the message based on the number of particles detected during the symbol interval. In this channel, the transmitter may be unable to control precisely the number of particles released, and the receiver may not detect all the particles that arrive. We demonstrate that the optimal input distribution for this channel always has mass points at zero and the maximum number of particles that can be released. We then consider diffusive particle transport, derive the capacity expression when the input distribution is binary, and show conditions under which the binary input is capacity-achieving. In particular, we demonstrate that when the transmitter cannot generate particles at a high rate, the optimal input distribution is binary.Comment: Accepted at IEEE International Symposium on Information Theory (ISIT