Refined Strong Converse for the Constant Composition Codes

A strong converse bound for constant composition codes of the form $P_{e}^{(n)} \geq 1- A n^{-0.5(1-E_{sc}'(R,W,p))} e^{-n E_{sc}(R,W,p)}$ is established using the Berry-Esseen theorem through the concepts of Augustin information and Augustin mean, where $A$ is a constant determined by the channel $W$, the composition $p$, and the rate $R$, i.e., $A$ does not depend on the block length $n$.Comment: 7 page

On the Reliability Function of Distributed Hypothesis Testing Under Optimal Detection

The distributed hypothesis testing problem with full side-information is studied. The trade-off (reliability function) between the two types of error exponents under limited rate is studied in the following way. First, the problem is reduced to the problem of determining the reliability function of channel codes designed for detection (in analogy to a similar result which connects the reliability function of distributed lossless compression and ordinary channel codes). Second, a single-letter random-coding bound based on a hierarchical ensemble, as well as a single-letter expurgated bound, are derived for the reliability of channel-detection codes. Both bounds are derived for a system which employs the optimal detection rule. We conjecture that the resulting random-coding bound is ensemble-tight, and consequently optimal within the class of quantization-and-binning schemes

Chip and Signature Interleaving in DS CDMA Systems

Siirretty Doriast

Capacity of interference networks : achievable regions and outer bounds

textIn an interference network, multiple transmitters communicate with multiple receivers using the same communication channel. The capacity region of an interference network is defined as the set of data rates that can be simultaneously achieved by the users of the network. One of the most important example of an interference network is the wireless network, where the communication channel is the wireless channel. Wireless interference networks are known to be interference limited rather than noise limited since the interference power level at the receivers (caused by other user's transmissions) is much higher than the noise power level. Most wireless communication systems deployed today employ transmission strategies where the interfering signals are treated in the same manner as thermal noise. Such strategies are known to be suboptimal (in terms of achieving higher data rates), because the interfering signals generated by other transmitters have a structure to them that is very different from that of random thermal noise. Hence, there is a need to design transmission strategies that exploit this structure of the interfering signals to achieve higher data rates. However, determining optimal strategies for mitigating interference has been a long standing open problem. In fact, even for the simplest interference network with just two users, the capacity region is unknown. In this dissertation, we will investigate the capacity region of several models of interference channels. We will derive limits on achievable data rates and design effective transmission strategies that come close to achieving the limits. We will investigate two kinds of networks - "small" (usually characterized by two transmitters and two receivers) and "large" where the number of users is large.Electrical and Computer Engineerin

Information Theory and Machine Learning

The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems

Practical interference management strategies in Gaussian networks

Increasing demand for bandwidth intensive activities on high-penetration wireless hand-held personal devices, combined with their processing power and advanced radio features, has necessitated a new look at the problems of resource provisioning and distributed management of coexistence in wireless networks. Information theory, as the science of studying the ultimate limits of communication e ciency, plays an important role in outlining guiding principles in the design and analysis of such communication schemes. Network information theory, the branch of information theory that investigates problems of multiuser and distributed nature in information transmission is ideally poised to answer questions about the design and analysis of multiuser communication systems. In the past few years, there have been major advances in network information theory, in particular in the generalized degrees of freedom framework for asymptotic analysis and interference alignment which have led to constant gap to capacity results for Gaussian interference channels. Unfortunately, practical adoption of these results has been slowed by their reliance on unrealistic assumptions like perfect channel state information at the transmitter and intricate constructions based on alignment over transcendental dimensions of real numbers. It is therefore necessary to devise transmission methods and coexistence schemes that fall under the umbrella of existing interference management and cognitive radio toolbox and deliver close to optimal performance. In this thesis we work on the theme of designing and characterizing the performance of conceptually simple transmission schemes that are robust and achieve performance that is close to optimal. In particular, our work is broadly divided into two parts. In the rst part, looking at cognitive radio networks, we seek to relax the assumption of non-causal knowledge of primary user's message at the secondary user's transmitter. We study a cognitive channel model based on Gaussian interference channel that does not assume anything about users other than primary user's priority over secondary user in reaching its desired quality of service. We characterize this quality of service requirement as a minimum rate that the primary user should be able to achieve. Studying the achievable performance of simple encoding and decoding schemes in this scenario, we propose a few di erent simple encoding schemes and explore di erent decoder designs. We show that surprisingly, all these schemes achieve the same rate region. Next, we study the problem of rate maximization faced by the secondary user subject to primary's QoS constraint. We show that this problem is not convex or smooth in general. We then use the symmetry properties of the problem to reduce its solution to a feasibly implementable line search. We also provide numerical results to demonstrate the performance of the scheme. Continuing on the theme of simple yet well-performing schemes for wireless networks, in the second part of the thesis, we direct our attention from two-user cognitive networks to the problem of smart interference management in large wireless networks. Here, we study the problem of interference-aware wireless link scheduling. Link scheduling is the problem of allocating a set of transmission requests into as small a set of time slots as possible such that all transmissions satisfy some condition of feasibility. The feasibility criterion has traditionally been lack of pair of links that interfere too much. This makes the problem amenable to solution using graph theoretical tools. Inspired by the recent results that the simple approach of treating interference as noise achieves maximal Generalized Degrees of Freedom (which is a measure that roughly captures how many equivalent single-user channels are contained in a given multi-user channel) and the generalization that it can attain rates within a constant gap of the capacity for a large class of Gaussian interference networks, we study the problem of scheduling links under a set Signal to Interference plus Noise Ratio (SINR) constraint. We show that for nodes distributed in a metric space and obeying path loss channel model, a re ned framework based on combining geometric and graph theoretic results can be devised to analyze the problem of nding the feasible sets of transmissions for a given level of desired SINR. We use this general framework to give a link scheduling algorithm that is provably within a logarithmic factor of the best possible schedule. Numerical simulations con rm that this approach outperforms other recently proposed SINR-based approaches. Finally, we conclude by identifying open problems and possible directions for extending these results

Expander Graphs and Coding Theory

Expander graphs are highly connected sparse graphs which lie at the interface of many different fields of study. For example, they play important roles in prime sieves, cryptography, compressive sensing, metric embedding, and coding theory to name a few. This thesis focuses on the connections between sparse graphs and coding theory. It is a major challenge to explicitly construct sparse graphs with good expansion properties, for example Ramanujan graphs. Nevertheless, explicit constructions do exist, and in this thesis, we survey many of these constructions up to this point including a new construction which slightly improves on an earlier edge expansion bound. The edge expansion of a graph is crucial in applications, and it is well-known that computing the edge expansion of an arbitrary graph is NP-hard. We present a simple algo-rithm for approximating the edge expansion of a graph using linear programming techniques. While Andersen and Lang (2008) proved similar results, our analysis attacks the problem from a different vantage point and was discovered independently. The main contribution in the thesis is a new result in fast decoding for expander codes. Current algorithms in the literature can decode a constant fraction of errors in linear time but require that the underlying graphs have vertex expansion at least 1/2. We present a fast decoding algorithm that can decode a constant fraction of errors in linear time given any vertex expansion (even if it is much smaller than 1/2) by using a stronger local code, and the fraction of errors corrected almost doubles that of Viderman (2013)