On Continuous-Time Gaussian Channels

A continuous-time white Gaussian channel can be formulated using a white Gaussian noise, and a conventional way for examining such a channel is the sampling approach based on the Shannon-Nyquist sampling theorem, where the original continuous-time channel is converted to an equivalent discrete-time channel, to which a great variety of established tools and methodology can be applied. However, one of the key issues of this scheme is that continuous-time feedback and memory cannot be incorporated into the channel model. It turns out that this issue can be circumvented by considering the Brownian motion formulation of a continuous-time white Gaussian channel. Nevertheless, as opposed to the white Gaussian noise formulation, a link that establishes the information-theoretic connection between a continuous-time channel under the Brownian motion formulation and its discrete-time counterparts has long been missing. This paper is to fill this gap by establishing causality-preserving connections between continuous-time Gaussian feedback/memory channels and their associated discrete-time versions in the forms of sampling and approximation theorems, which we believe will play important roles in the long run for further developing continuous-time information theory. As an immediate application of the approximation theorem, we propose the so-called approximation approach to examine continuous-time white Gaussian channels in the point-to-point or multi-user setting. It turns out that the approximation approach, complemented by relevant tools from stochastic calculus, can enhance our understanding of continuous-time Gaussian channels in terms of giving alternative and strengthened interpretation to some long-held folklore, recovering "long known" results from new perspectives, and rigorously establishing new results predicted by the intuition that the approximation approach carries

Capacity Bounds For Multi-User Channels With Feedback, Relaying and Cooperation

Recent developments in communications are driven by the goal of achieving high data rates for wireless communication devices. To achieve this goal, several new phenomena need to be investigated from an information theoretic perspective. In this dissertation, we focus on three of these phenomena: feedback, relaying and cooperation. We study these phenomena for various multi-user channels from an information theoretic point of view. One of the aims of this dissertation is to study the performance limits of simple wireless networks, for various forms of feedback and cooperation. Consider an uplink communication system, where several users wish to transmit independent data to a base-station. If the base-station can send feedback to the users, one can expect to achieve higher data-rates since feedback can enable cooperation among the users. Another way to improve data-rates is to make use of the broadcast nature of the wireless medium, where the users can overhear each other's transmitted signals. This particular phenomenon has garnered much attention lately, where users can help in increasing each other's data-rates by utilizing the overheard information. This overheard information can be interpreted as a generalized form of feedback. To take these several models of feedback and cooperation into account, we study the two-user multiple access channel and the two-user interference channel with generalized feedback. For all these models, we derive new outer bounds on their capacity regions. We specialize these results for noiseless feedback, additive noisy feedback and user-cooperation models and show strict improvements over the previously known bounds. Next, we study state-dependent channels with rate-limited state information to the receiver or to the transmitter. This state-dependent channel models a practical situation of fading, where the fade information is partially available to the receiver or to the transmitter. We derive new bounds on the capacity of such channels and obtain capacity results for a special sub-class of such channels. We study the effect of relaying by considering the parallel relay network, also known as the diamond channel. The parallel relay network considered in this dissertation comprises of a cascade of a general broadcast channel to the relays and an orthogonal multiple access channel from the relays to the receiver. We characterize the capacity of the diamond channel, when the broadcast channel is deterministic. We also study the diamond channel with partially separated relays, and obtain capacity results when the broadcast channel is either semi-deterministic or physically degraded. Our results also demonstrate that feedback to the relays can strictly increase the capacity of the diamond channel. In several sensor network applications, distributed lossless compression of sources is of considerable interest. The presence of adversarial nodes makes it important to design compression schemes which serve the dual purpose of reliable source transmission to legitimate nodes while minimizing the information leakage to the adversarial nodes. Taking this constraint into account, we consider information theoretic secrecy, where our aim is to limit the information leakage to the eavesdropper. For this purpose, we study a secure source coding problem with coded side information from a helper to the legitimate user. We derive the rate-equivocation region for this problem. We show that the helper node serves the dual purpose of reducing the source transmission rate and increasing the uncertainty at the adversarial node. Next, we considered two different secure source coding models and provide the corresponding rate-equivocation regions

MODEL UPDATING AND STRUCTURAL HEALTH MONITORING OF HORIZONTAL AXIS WIND TURBINES VIA ADVANCED SPINNING FINITE ELEMENTS AND STOCHASTIC SUBSPACE IDENTIFICATION METHODS

Wind energy has been one of the most growing sectors of the nation’s renewable energy portfolio for the past decade, and the same tendency is being projected for the upcoming years given the aggressive governmental policies for the reduction of fossil fuel dependency. Great technological expectation and outstanding commercial penetration has shown the so called Horizontal Axis Wind Turbines (HAWT) technologies. Given its great acceptance, size evolution of wind turbines over time has increased exponentially. However, safety and economical concerns have emerged as a result of the newly design tendencies for massive scale wind turbine structures presenting high slenderness ratios and complex shapes, typically located in remote areas (e.g. offshore wind farms). In this regard, safety operation requires not only having first-hand information regarding actual structural dynamic conditions under aerodynamic action, but also a deep understanding of the environmental factors in which these multibody rotating structures operate. Given the cyclo-stochastic patterns of the wind loading exerting pressure on a HAWT, a probabilistic framework is appropriate to characterize the risk of failure in terms of resistance and serviceability conditions, at any given time. Furthermore, sources of uncertainty such as material imperfections, buffeting and flutter, aeroelastic damping, gyroscopic effects, turbulence, among others, have pleaded for the use of a more sophisticated mathematical framework that could properly handle all these sources of indetermination. The attainable modeling complexity that arises as a result of these characterizations demands a data-driven experimental validation methodology to calibrate and corroborate the model. For this aim, System Identification (SI) techniques offer a spectrum of well-established numerical methods appropriated for stationary, deterministic, and data-driven numerical schemes, capable of predicting actual dynamic states (eigenrealizations) of traditional time-invariant dynamic systems. As a consequence, it is proposed a modified data-driven SI metric based on the so called Subspace Realization Theory, now adapted for stochastic non-stationary and timevarying systems, as is the case of HAWT’s complex aerodynamics. Simultaneously, this investigation explores the characterization of the turbine loading and response envelopes for critical failure modes of the structural components the wind turbine is made of. In the long run, both aerodynamic framework (theoretical model) and system identification (experimental model) will be merged in a numerical engine formulated as a search algorithm for model updating, also known as Adaptive Simulated Annealing (ASA) process. This iterative engine is based on a set of function minimizations computed by a metric called Modal Assurance Criterion (MAC). In summary, the Thesis is composed of four major parts: (1) development of an analytical aerodynamic framework that predicts interacted wind-structure stochastic loads on wind turbine components; (2) development of a novel tapered-swept-corved Spinning Finite Element (SFE) that includes dampedgyroscopic effects and axial-flexural-torsional coupling; (3) a novel data-driven structural health monitoring (SHM) algorithm via stochastic subspace identification methods; and (4) a numerical search (optimization) engine based on ASA and MAC capable of updating the SFE aerodynamic model

Lecture Notes on Network Information Theory

These lecture notes have been converted to a book titled Network Information Theory published recently by Cambridge University Press. This book provides a significantly expanded exposition of the material in the lecture notes as well as problems and bibliographic notes at the end of each chapter. The authors are currently preparing a set of slides based on the book that will be posted in the second half of 2012. More information about the book can be found at http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/

Distributed secrecy for information theoretic sensor network models

This dissertation presents a novel problem inspired by the characteristics of sensor networks. The basic setup through-out the dissertation is that a set of sensor nodes encipher their data without collaboration and without any prior shared secret materials. The challenge is dealt by an eavesdropper who intercepts a subset of the enciphered data and wishes to gain knowledge of the uncoded data. This problem is challenging and novel given that the eavesdropper is assumed to know everything, including secret cryptographic keys used by both the encoders and decoders. We study the above problem using information theoretic models as a necessary first step towards an understanding of the characteristics of this system problem. This dissertation contains four parts. The first part deals with noiseless channels, and the goal is for sensor nodes to both source code and encipher their data. We derive inner and outer regions of the capacity region (i.e the set of all source coding and equivocation rates) for this problem under general distortion constraints. The main conclusion in this part is that unconditional secrecy is unachievable unless the distortion is maximal, rendering the data useless. In the second part we thus provide a practical coding scheme based on distributed source coding using syndromes (DISCUS) that provides secrecy beyond the equivocation measure, i.e. secrecy on each symbol in the message. The third part deals with discrete memoryless channels, and the goal is for sensor nodes to both channel code and encipher their data. We derive inner and outer regions to the secrecy capacity region, i.e. the set of all channel coding rates that achieve (weak) unconditional secrecy. The main conclusion in this part is that interference allows (weak) unconditional secrecy to be achieved in contrast with the first part of this dissertation. The fourth part deals with wireless channels with fading and additive Gaussian noise. We derive a general outer region and an inner region based on an equal SNR assumption, and show that the two are partially tight when the maximum available user powers are admissible

Physical-Layer Security in Wireless Communication Systems

The use of wireless networks has grown significantly in contemporary times, and continues to develop further. The broadcast nature of wireless communications, however, makes them particularly vulnerable to eavesdropping. Unlike traditional solutions, which usually handle security at the application layer, the primary concern of this dissertation is to analyze and develop solutions based on coding techniques at the physical-layer. First, in chapter $2$, we consider a scenario where a source node wishes to broadcast two confidential messages to two receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless communications, where individual secure messages are broadcast over open media and can be received by any illegitimate receiver. The secrecy level is measured by the equivocation rate at the eavesdropper. We first study the general (non-degraded) broadcast channel with an eavesdropper, and present an inner bound on the secrecy capacity region for this model. This inner bound is based on a combination of random binning, and the Gelfand-Pinsker binning. We further study the situation in which the channels are degraded. For the degraded broadcast channel with an eavesdropper, we present the secrecy capacity region. Our achievable coding scheme is based on Cover's superposition scheme and random binning. We refer to this scheme as the Secret Superposition Scheme. Our converse proof is based on a combination of the converse proof of the conventional degraded broadcast channel and Csiszar Lemma. We then assume that the channels are Additive White Gaussian Noise and show that the Secret Superposition Scheme with Gaussian codebook is optimal. The converse proof is based on Costa's entropy power inequality. Finally, we use a broadcast strategy for the slowly fading wire-tap channel when only the eavesdropper's channel is fixed and known at the transmitter. We derive the optimum power allocation for the coding layers, which maximizes the total average rate. Second, in chapter $3$ , we consider the Multiple-Input-Multiple-Output (MIMO) scenario of a broadcast channel where a wiretapper also receives the transmitted signal via another MIMO channel. First, we assume that the channels are degraded and the wiretapper has the worst channel. We establish the capacity region of this scenario. Our achievability scheme is the Secret Superposition Coding. For the outerbound, we use notion of the enhanced channels to show that the secret superposition of Gaussian codes is optimal. We show that we only need to enhance the channels of the legitimate receivers, and the channel of the eavesdropper remains unchanged. We then extend the result of the degraded case to a non-degraded case. We show that the secret superposition of Gaussian codes, along with successive decoding, cannot work when the channels are not degraded. We develop a Secret Dirty Paper Coding scheme and show that it is optimal for this channel. We then present a corollary generalizing the capacity region of the two receivers case to the case of multiple receivers. Finally, we investigate a scenario which frequently occurs in the practice of wireless networks. In this scenario, the transmitter and the eavesdropper have multiple antennae, while both intended receivers have a single antenna (representing resource limited mobile units). We characterize the secrecy capacity region in terms of generalized eigenvalues of the receivers' channels and the eavesdropper's channel. We refer to this configuration as the MISOME case. We then present a corollary generalizing the results of the two receivers case to multiple receivers. In the high SNR regime, we show that the capacity region is a convex closure of rectangular regions. Finally, in chapter $4$, we consider a $K$-user secure Gaussian Multiple-Access-Channel with an external eavesdropper. We establish an achievable rate region for the secure discrete memoryless MAC. Thereafter, we prove the secrecy sum capacity of the degraded Gaussian MIMO MAC using Gaussian codebooks. For the non-degraded Gaussian MIMO MAC, we propose an algorithm inspired by the interference alignment technique to achieve the largest possible total Secure-Degrees-of-Freedom . When all the terminals are equipped with a single antenna, Gaussian codebooks have shown to be inefficient in providing a positive S-DoF. Instead, we propose a novel secure coding scheme to achieve a positive S-DoF in the single antenna MAC. This scheme converts the single-antenna system into a multiple-dimension system with fractional dimensions. The achievability scheme is based on the alignment of signals into a small sub-space at the eavesdropper, and the simultaneous separation of the signals at the intended receiver. We use tools from the field of Diophantine Approximation in number theory to analyze the probability of error in the coding scheme. We prove that the total S-DoF of $\frac{K-1}{K}$ can be achieved for almost all channel gains. For the other channel gains, we propose a multi-layer coding scheme to achieve a positive S-DoF. As a function of channel gains, therefore, the achievable S-DoF is discontinued

Fundamental limitations on communication channels with noisy feedback: information flow, capacity and bounds

Since the success of obtaining the capacity (i.e. the maximal achievable transmission rate under which the message can be recovered with arbitrarily small probability of error) for non-feedback point-to-point communication channels by C. Shannon (in 1948), Information Theory has been proved to be a powerful tool to derive fundamental limitations in communication systems. During the last decade, motivated by the emerging of networked systems, information theorists have turned lots of their attention to communication channels with feedback (through another channel from receiver to transmitter). Under the assumption that the feedback channel is noiseless, a large body of notable results have been derived, although much work still needs to be done. However, when this ideal assumption is removed, i.e., the feedback channel is noisy, only few valuable results can be found in the literature and many challenging problems are still open. This thesis aims to address some of these long-standing noisy feedback problems, with concentration on the channel capacity. First of all, we analyze the fundamental information flow in noisy feedback channels. We introduce a new notion, the residual directed information, in order to characterize the noisy feedback channel capacity for which the standard directed information can not be used. As an illustration, finite-alphabet noisy feedback channels have been studied in details. Next, we provide an information flow decomposition equality which serves as a foundation of other novel results in this thesis. With the result of information flow decomposition in hand, we next investigate time-varying Gaussian channels with additive Gaussian noise feedback. Following the notable Cover-Pombra results in 1989, we define the n-block noisy feedback capacity and derive a pair of n-block upper and lower bounds on the n-block noisy feedback capacity. These bounds can be obtained by efficiently solving convex optimization problems. Under the assumption of stationarity on the additive Gaussian noises, we show that the limits of these n-block bounds can be characterized in a power spectral optimization form. In addition, two computable lower bounds are derived for the Shannon capacity. Next, we consider a class of channels where feedback could not increase the capacity and thus the noisy feedback capacity equals to the non-feedback capacity. We derive a necessary condition (characterized by the directed information) for the capacity-achieving channel codes. The condition implies that using noisy feedback is detrimental to achievable rate, i.e, the capacity can not be achieved by using noisy feedback. Finally, we introduce a new framework of communication channels with noisy feedback where the feedback information received by the transmitter is also available to the decoder with some finite delays. We investigate the capacity and linear coding schemes for this extended noisy feedback channels. To summarize, this thesis firstly provides a foundation (i.e. information flow analysis) for analyzing communications channels with noisy feedback. In light of this analysis, we next present a sequence of novel results, e.g. channel coding theorem, capacity bounds, etc., which result in a significant step forward to address the long-standing noisy feedback problem