On the Reliability Function of Distributed Hypothesis Testing Under Optimal Detection

The distributed hypothesis testing problem with full side-information is studied. The trade-off (reliability function) between the two types of error exponents under limited rate is studied in the following way. First, the problem is reduced to the problem of determining the reliability function of channel codes designed for detection (in analogy to a similar result which connects the reliability function of distributed lossless compression and ordinary channel codes). Second, a single-letter random-coding bound based on a hierarchical ensemble, as well as a single-letter expurgated bound, are derived for the reliability of channel-detection codes. Both bounds are derived for a system which employs the optimal detection rule. We conjecture that the resulting random-coding bound is ensemble-tight, and consequently optimal within the class of quantization-and-binning schemes

Throughput Maximization in Cognitive Radio Under Peak Interference Constraints With Limited Feedback

A spectrum-sharing scenario in a cognitive radio (CR) network where a secondary user (SU) shares a narrowband channel with N primary users (PUs) is considered. We investigate the SU ergodic capacity maximization problem under an average transmit power constraint on the SU and N individual peak interference power constraints at each primary-user receiver (PU-Rx) with various forms of imperfect channel-state information (CSI) available at the secondary-user transmitter (SU-Tx). For easy exposition, we first look at the case when the SU-Tx can obtain perfect knowledge of the CSI from the SU-Tx to the secondary-user receiver link, which is denoted as g 1 , but can only access quantized CSI of the SU-Tx to PU-Rx links, which is denoted as g oi , i = 1,..., N, through a limited-feed back link of B = log 2 L b. For this scenario, a locally optimum quantized power allocation (codebook) is obtained with quantized g 0i , i = 1,..., N information by using the Karush-Kuhn-Tucker (KKT) necessary optimality conditions to numerically solve the nonconvex SU capacity maximization problem. We derive asymptotic approximations for the SU ergodic capacity performance for the case when the number of feedback bits grows large (B → ∞) and/or there is a large number of PUs (N → ∞) that operate. For the interference-limited regime, where the average transmit power constraint is inactive, an alternative locally optimum scheme, called the quantized-rate allocation strategy, based on the quantized-ratio g 1 /max i g oi information, is proposed. Subsequently, we relax the strong assumption of full-CSI knowledge of g 1 at the SU-Tx to imperfect g 1 knowledge that is also available at the SU-Tx. Depending on the way the SU-Tx obtains the g 1 information, the following two different suboptimal quantized power codebooks are derived for the SU ergodic capacity maximization problem: 1) the power codebook with noisy g 1 estimates and quantized g oi , i = 1,..., N knowledge and 2) another power codebook with both quantized g 1 and g oi , i = 1,... , N information. We emphasize the fact that, although the proposed algorithms result in locally optimum or strictly suboptimal solutions, numerical results demonstrate that they are extremely efficient. The efficacy of the proposed asymptotic approximations is also illustrated through numerical simulation results

Zero-delay source-channel coding

In this thesis, we investigate the zero-delay transmission of source samples over three different types of communication channel models. First, we consider the zero-delay transmission of a Gaussian source sample over an additive white Gaussian noise (AWGN) channel in the presence of an additive white Gaussian (AWG) interference, which is fully known by the transmitter. We propose three parameterized linear and non-linear transmission schemes for this scenario, and compare the corresponding mean square error (MSE) performances with that of a numerically optimized encoder, obtained using the necessary optimality conditions. Next, we consider the zero-delay transmission of a Gaussian source sample over an AWGN channel with a one-bit analog-to-digital (ADC) front end. We study this problem under two different performance criteria, namely the MSE distortion and the distortion outage probability (DOP), and obtain the optimal encoder and the decoder for both criteria. As generalizations of this scenario, we consider the performance with a K-level ADC front end as well as with multiple one-bit ADC front ends. We derive necessary conditions for the optimal encoder and decoder, which are then used to obtain numerically optimized encoder and decoder mappings. Finally, we consider the transmission of a Gaussian source sample over an AWGN channel with a one-bit ADC front end in the presence of correlated side information at the receiver. Again, we derive the necessary optimality conditions, and using these conditions obtain numerically optimized encoder and decoder mappings. We also consider the scenario in which the side information is available also at the encoder, and obtain the optimal encoder and decoder mappings. The performance of the latter scenario serves as a lower bound on the performance of the case in which the side information is available only at the decoder.Open Acces

Hardware-Conscious Wireless Communication System Design

The work at hand is a selection of topics in efficient wireless communication system design, with topics logically divided into two groups.One group can be described as hardware designs conscious of their possibilities and limitations. In other words, it is about hardware that chooses its configuration and properties depending on the performance that needs to be delivered and the influence of external factors, with the goal of keeping the energy consumption as low as possible. Design parameters that trade off power with complexity are identified for analog, mixed signal and digital circuits, and implications of these tradeoffs are analyzed in detail. An analog front end and an LDPC channel decoder that adapt their parameters to the environment (e.g. fluctuating power level due to fading) are proposed, and it is analyzed how much power/energy these environment-adaptive structures save compared to non-adaptive designs made for the worst-case scenario. Additionally, the impact of ADC bit resolution on the energy efficiency of a massive MIMO system is examined in detail, with the goal of finding bit resolutions that maximize the energy efficiency under various system setups.In another group of themes, one can recognize systems where the system architect was conscious of fundamental limitations stemming from hardware.Put in another way, in these designs there is no attempt of tweaking or tuning the hardware. On the contrary, system design is performed so as to work around an existing and unchangeable hardware limitation. As a workaround for the problematic centralized topology, a massive MIMO base station based on the daisy chain topology is proposed and a method for signal processing tailored to the daisy chain setup is designed. In another example, a large group of cooperating relays is split into several smaller groups, each cooperatively performing relaying independently of the others. As cooperation consumes resources (such as bandwidth), splitting the system into smaller, independent cooperative parts helps save resources and is again an example of a workaround for an inherent limitation.From the analyses performed in this thesis, promising observations about hardware consciousness can be made. Adapting the structure of a hardware block to the environment can bring massive savings in energy, and simple workarounds prove to perform almost as good as the inherently limited designs, but with the limitation being successfully bypassed. As a general observation, it can be concluded that hardware consciousness pays off

Belief Refinement Approaches to Communication and Inference Problems

This dissertation considers a problem where a single agent or a group of agents aim to estimate/learn unknown (possibly time-varying) parameters of interest despite making noisy observations. The agents take a Bayesian-like approach by maintaining a posterior probability distribution or “belief" over a parameter space conditioned on past observations. The agents aim to iteratively refine their belief over the parameter space as new information is acquired from their private observations or through collaboration with other agents. In particular, the agents aim to ensure that sufficient belief is assigned in neighborhoods centered around the true parameter with high probability or “reliability". In the context of communication problems considered in this dissertation, the agents may be active, i.e., agents may additionally take actions which provide new observations. Furthermore, agents may employ an adaptive strategy, i.e., using their past actions and the resulting observations, agents can adaptively choose actions to control the concentration of the belief. When the agents are active, we propose and analyze adaptive belief refinement approaches to obtain belief concentration on the unknown parameter with high reliability. In a different context, namely that of decentralized inference, we consider passive agents. Here, agents face an additional challenge due to the statistical insufficiency of their private observations to learn the unknown parameter. While individual agents’ observations are not informative enough, we assume that the agents’ observations are collectively informative to learn the unknown parameter. Here, we propose and analyze decentralized belief refining strategies to collaboratively obtain belief concentration on the unknown parameter. In the first part of this dissertation, we consider active strategies that are extensions of the posterior matching strategy (PM) introduced by Horstein, which is a generalization of the well-known binary search algorithm. We propose and analyze PM based strategies in the context of modern communication systems, namely the problem of establishing initial access in mm-Wave communication and spectrum sensing for Cognitive Radio. We propose and analyze channel coding strategies for real-time streaming and control applications. The second part of the dissertation investigates the belief refinement approaches for decentralized learning. In particular, it focusing on developing and analyzing a decentralized learning rule for statistical hypothesis testing and its application to decentralized machine learning

Information Theory and Machine Learning

The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems

Optical Communication Through the Turbulent Atmosphere with Transmitter and Receiver Diversity, Wavefront Control, and Coherent Detection

Thesis Supervisor: Vincent W. S. Chan Title: Joan and Irwin M. Jacobs Professor of Electrical Engineering and Computer ScienceFree space optical communication through the atmosphere has the potential to provide secure, low-cost, rapidly deployable, dynamic, data transmission at very high rates. However, the deleterious e ects of turbulence can severely limit the utility of such a system, causing outages of up to 100 ms. For this thesis, we investigate an architecture that uses multiple transmitters and multiple coherent receivers to overcome these turbulence-induced outages. By controlling the amplitude and phase of the optical eld at each transmitter, based on turbulence state information fed back from the receiver, we show that the system performance is greatly increased by exploiting the instantaneous structure of the turbulence. This architecture provides a robust highcapacity free-space optical communication link over multiple spectral bands, from visible to infrared. We aim to answer questions germane to the design and implementation of the diversity optical communication architecture in a turbulent environment. We analyze several di erent optical eld spatial modulation techniques, each of which is based on a di erent assumption about the quality of turbulence state information at the transmitter. For example, we explore a diversity optical system with perfect turbulence state information at the transmitter and receiver that allocates transmit power into the spatial modes with the smallest propagation losses in order to decrease bit errors and mitigate turbulence-induced outages. Another example of a diversity optical system that we examine is a diversity optical system with only a subset of the turbulence state information: this system could allocate all power to the transmitter with the smallest attenuation. We characterize the system performance for the various spatial modulation techniques in terms of average bit error rate (BER), outage probability, and power gain due to diversity. We rst characterize the performance of these techniques in the idealized case, where the instantaneous channel state is perfectly known at both the receiver and transmitter. The time evolution of the atmosphere, as wind moves tur- 3 bules across the propagation path, can limit the ability to have perfect turbulence state knowledge at the transmitter and, thus can limit any improvement realized by optical eld spatial modulation techniques. The improvement is especially limited if the latency is large or the feedback rate is short compared to the time it takes for turbules to move across the link. As a result, we make successive generalizations, until we describe the optimal system design and communication techniques for sparse aperture systems for the most general realistic case, one with inhomogeneous turbulence and imperfect (delayed, noisy, and distorted) knowledge of the atmospheric state

Information reconciliation methods in secret key distribution

We consider in this thesis the problem of information reconciliation in the context of secret key distillation between two legitimate parties. In some scenarios of interest this problem can be advantageously solved with low density parity check (LDPC) codes optimized for the binary symmetric channel. In particular, we demonstrate that our method leads to a significant efficiency improvement, with respect to earlier interactive reconciliation methods. We propose a protocol based on LDPC codes that can be adapted to changes in the communication channel extending the original source. The efficiency of our protocol is only limited by the quality of the code and, while transmitting more information than needed to reconcile Alice’s and Bob’s sequences, it does not reveal any more information on the original source than an ad-hoc code would have revealed.---ABSTRACT---En esta tesis estudiamos el problema de la reconciliación de información en el contexto de la destilación de secreto entre dos partes. En algunos escenarios de interés, códigos de baja densidad de ecuaciones de paridad (LDPC) adaptados al canal binario simétrico ofrecen una buena solución al problema estudiado. Demostramos que nuestro método mejora significativamente la eficiencia de la reconciliación. Proponemos un protocolo basado en códigos LDPC que puede ser adaptado a cambios en el canal de comunicaciones mediante una extensión de la fuente original. La eficiencia de nuestro protocolo está limitada exclusivamente por el código utilizado y no revela información adicional sobre la fuente original que la que un código con la tasa de información adaptada habría revelado