Algorithmic Aspects of Optimal Channel Coding

A central question in information theory is to determine the maximum success probability that can be achieved in sending a fixed number of messages over a noisy channel. This was first studied in the pioneering work of Shannon who established a simple expression characterizing this quantity in the limit of multiple independent uses of the channel. Here we consider the general setting with only one use of the channel. We observe that the maximum success probability can be expressed as the maximum value of a submodular function. Using this connection, we establish the following results: 1. There is a simple greedy polynomial-time algorithm that computes a code achieving a (1-1/e)-approximation of the maximum success probability. Moreover, for this problem it is NP-hard to obtain an approximation ratio strictly better than (1-1/e). 2. Shared quantum entanglement between the sender and the receiver can increase the success probability by a factor of at most 1/(1-1/e). In addition, this factor is tight if one allows an arbitrary non-signaling box between the sender and the receiver. 3. We give tight bounds on the one-shot performance of the meta-converse of Polyanskiy-Poor-Verdu.Comment: v2: 16 pages. Added alternate proof of main result with random codin

Network Information Flow with Correlated Sources

In this paper, we consider a network communications problem in which multiple correlated sources must be delivered to a single data collector node, over a network of noisy independent point-to-point channels. We prove that perfect reconstruction of all the sources at the sink is possible if and only if, for all partitions of the network nodes into two subsets S and S^c such that the sink is always in S^c, we have that H(U_S|U_{S^c}) < \sum_{i\in S,j\in S^c} C_{ij}. Our main finding is that in this setup a general source/channel separation theorem holds, and that Shannon information behaves as a classical network flow, identical in nature to the flow of water in pipes. At first glance, it might seem surprising that separation holds in a fairly general network situation like the one we study. A closer look, however, reveals that the reason for this is that our model allows only for independent point-to-point channels between pairs of nodes, and not multiple-access and/or broadcast channels, for which separation is well known not to hold. This ``information as flow'' view provides an algorithmic interpretation for our results, among which perhaps the most important one is the optimality of implementing codes using a layered protocol stack.Comment: Final version, to appear in the IEEE Transactions on Information Theory -- contains (very) minor changes based on the last round of review

Power and Channel Allocation for Non-orthogonal Multiple Access in 5G Systems: Tractability and Computation

Network capacity calls for significant increase for 5G cellular systems. A promising multi-user access scheme, non-orthogonal multiple access (NOMA) with successive interference cancellation (SIC), is currently under consideration. In NOMA, spectrum efficiency is improved by allowing more than one user to simultaneously access the same frequency-time resource and separating multi-user signals by SIC at the receiver. These render resource allocation and optimization in NOMA different from orthogonal multiple access in 4G. In this paper, we provide theoretical insights and algorithmic solutions to jointly optimize power and channel allocation in NOMA. For utility maximization, we mathematically formulate NOMA resource allocation problems. We characterize and analyze the problems' tractability under a range of constraints and utility functions. For tractable cases, we provide polynomial-time solutions for global optimality. For intractable cases, we prove the NP-hardness and propose an algorithmic framework combining Lagrangian duality and dynamic programming (LDDP) to deliver near-optimal solutions. To gauge the performance of the obtained solutions, we also provide optimality bounds on the global optimum. Numerical results demonstrate that the proposed algorithmic solution can significantly improve the system performance in both throughput and fairness over orthogonal multiple access as well as over a previous NOMA resource allocation scheme.Comment: IEEE Transactions on Wireless Communications, revisio

Short Block-length Codes for Ultra-Reliable Low-Latency Communications

This paper reviews the state of the art channel coding techniques for ultra-reliable low latency communication (URLLC). The stringent requirements of URLLC services, such as ultra-high reliability and low latency, have made it the most challenging feature of the fifth generation (5G) mobile systems. The problem is even more challenging for the services beyond the 5G promise, such as tele-surgery and factory automation, which require latencies less than 1ms and failure rate as low as $10^{-9}$. The very low latency requirements of URLLC do not allow traditional approaches such as re-transmission to be used to increase the reliability. On the other hand, to guarantee the delay requirements, the block length needs to be small, so conventional channel codes, originally designed and optimised for moderate-to-long block-lengths, show notable deficiencies for short blocks. This paper provides an overview on channel coding techniques for short block lengths and compares them in terms of performance and complexity. Several important research directions are identified and discussed in more detail with several possible solutions.Comment: Accepted for publication in IEEE Communications Magazin

Feedback Allocation For OFDMA Systems With Slow Frequency-domain Scheduling

We study the problem of allocating limited feedback resources across multiple users in an orthogonal-frequency-division-multiple-access downlink system with slow frequency-domain scheduling. Many flavors of slow frequency-domain scheduling (e.g., persistent scheduling, semi-persistent scheduling), that adapt user-sub-band assignments on a slower time-scale, are being considered in standards such as 3GPP Long-Term Evolution. In this paper, we develop a feedback allocation algorithm that operates in conjunction with any arbitrary slow frequency-domain scheduler with the goal of improving the throughput of the system. Given a user-sub-band assignment chosen by the scheduler, the feedback allocation algorithm involves solving a weighted sum-rate maximization at each (slow) scheduling instant. We first develop an optimal dynamic-programming-based algorithm to solve the feedback allocation problem with pseudo-polynomial complexity in the number of users and in the total feedback bit budget. We then propose two approximation algorithms with complexity further reduced, for scenarios where the problem exhibits additional structure.Comment: Accepted to IEEE Transactions on Signal Processin

Shannon Information and Kolmogorov Complexity

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories: Shannon entropy versus Kolmogorov complexity, the relation of both to universal coding, Shannon mutual information versus Kolmogorov (`algorithmic') mutual information, probabilistic sufficient statistic versus algorithmic sufficient statistic (related to lossy compression in the Shannon theory versus meaningful information in the Kolmogorov theory), and rate distortion theory versus Kolmogorov's structure function. Part of the material has appeared in print before, scattered through various publications, but this is the first comprehensive systematic comparison. The last mentioned relations are new.Comment: Survey, LaTeX 54 pages, 3 figures, Submitted to IEEE Trans Information Theor