Fundamental limitations on communication channels with noisy feedback: information flow, capacity and bounds

Since the success of obtaining the capacity (i.e. the maximal achievable transmission rate under which the message can be recovered with arbitrarily small probability of error) for non-feedback point-to-point communication channels by C. Shannon (in 1948), Information Theory has been proved to be a powerful tool to derive fundamental limitations in communication systems. During the last decade, motivated by the emerging of networked systems, information theorists have turned lots of their attention to communication channels with feedback (through another channel from receiver to transmitter). Under the assumption that the feedback channel is noiseless, a large body of notable results have been derived, although much work still needs to be done. However, when this ideal assumption is removed, i.e., the feedback channel is noisy, only few valuable results can be found in the literature and many challenging problems are still open. This thesis aims to address some of these long-standing noisy feedback problems, with concentration on the channel capacity. First of all, we analyze the fundamental information flow in noisy feedback channels. We introduce a new notion, the residual directed information, in order to characterize the noisy feedback channel capacity for which the standard directed information can not be used. As an illustration, finite-alphabet noisy feedback channels have been studied in details. Next, we provide an information flow decomposition equality which serves as a foundation of other novel results in this thesis. With the result of information flow decomposition in hand, we next investigate time-varying Gaussian channels with additive Gaussian noise feedback. Following the notable Cover-Pombra results in 1989, we define the n-block noisy feedback capacity and derive a pair of n-block upper and lower bounds on the n-block noisy feedback capacity. These bounds can be obtained by efficiently solving convex optimization problems. Under the assumption of stationarity on the additive Gaussian noises, we show that the limits of these n-block bounds can be characterized in a power spectral optimization form. In addition, two computable lower bounds are derived for the Shannon capacity. Next, we consider a class of channels where feedback could not increase the capacity and thus the noisy feedback capacity equals to the non-feedback capacity. We derive a necessary condition (characterized by the directed information) for the capacity-achieving channel codes. The condition implies that using noisy feedback is detrimental to achievable rate, i.e, the capacity can not be achieved by using noisy feedback. Finally, we introduce a new framework of communication channels with noisy feedback where the feedback information received by the transmitter is also available to the decoder with some finite delays. We investigate the capacity and linear coding schemes for this extended noisy feedback channels. To summarize, this thesis firstly provides a foundation (i.e. information flow analysis) for analyzing communications channels with noisy feedback. In light of this analysis, we next present a sequence of novel results, e.g. channel coding theorem, capacity bounds, etc., which result in a significant step forward to address the long-standing noisy feedback problem

Digital Signal Processing Research Program

Contains table of contents for Section 2, an introduction, reports on twenty-two research projects and a list of publications.Sanders, a Lockheed-Martin Corporation Contract BZ4962U.S. Army Research Laboratory Contract DAAL01-96-2-0001U.S. Navy - Office of Naval Research Grant N00014-93-1-0686National Science Foundation Grant MIP 95-02885U.S. Navy - Office of Naval Research Grant N00014-96-1-0930National Defense Science and Engineering FellowshipU.S. Air Force - Office of Scientific Research Grant F49620-96-1-0072U.S. Navy - Office of Naval Research Grant N00014-95-1-0362National Science Foundation Graduate Research FellowshipAT&T Bell Laboratories Graduate Research FellowshipU.S. Army Research Laboratory Contract DAAL01-96-2-0002National Science Foundation Graduate FellowshipU.S. Army Research Laboratory/Advanced Sensors Federated Lab Program Contract DAAL01-96-2-000

The Entropy Gain of Linear Systems and Some of Its Implications

We study the increase in per-sample differential entropy rate of random sequences and processes after being passed through a non minimum-phase (NMP) discrete-time, linear time-invariant (LTI) filter G. For LTI discrete-time filters and random processes, it has long been established by Theorem 14 in Shannon’s seminal paper that this entropy gain, G(G), equals the integral of log|G(ejω)|. In this note, we first show that Shannon’s Theorem 14 does not hold in general. Then, we prove that, when comparing the input differential entropy to that of the entire (longer) output of G, the entropy gain equals G(G). We show that the entropy gain between equal-length input and output sequences is upper bounded by G(G) and arises if and only if there exists an output additive disturbance with finite differential entropy (no matter how small) or a random initial state. Unlike what happens with linear maps, the entropy gain in this case depends on the distribution of all the signals involved. We illustrate some of the consequences of these results by presenting their implications in three different problems. Specifically: conditions for equality in an information inequality of importance in networked control problems; extending to a much broader class of sources the existing results on the rate-distortion function for non-stationary Gaussian sources, and an observation on the capacity of auto-regressive Gaussian channels with feedback

Feedback of channel state information in multi-antenna systems based on quantization of channel Gram matrices

This dissertation deals with the proper design of efficient feedback strategies for Multiple-Input Multiple-Output (MIMO) communication systems. MIMO systems outperform single antenna systems in terms of achievable throughput and are more resilient to noise and interference, which are becoming the limiting factors in the current and future communications. Apart from the clear performance advantages, MIMO systems introduce an additional complexity factor, since they require knowledge of the propagation channel in order to be able to adapt the transmission to the propagation channel’s characteristics and achieve optimum performance. This channel knowledge, also known as Channel State Information (CSI), is estimated at the receiver and sent to the transmitter through a limited feedback link. In this dissertation, first, the minimum channel information necessary at the transmitter for the optimum precoding design is identified. This minimum information for the optimum design of the system corresponds to the channel Gram matrix. It is essential for the design of optimized systems to avoid the transmission of redundant feedback information. Following this idea, a quantization algorithm that exploits the differential geometry of the set of Gram matrices and the correlation in time present in most propagation channels is developed in order to greatly improve the feedback performance. This scheme is applied first to single-user MIMO communications, then to some particular multiuser scenarios, and finally it is extended to general multiuser broadcast communications. To conclude, the feedback link sizing is studied. An analysis of the tradeoff between size of the forward link and size of the feedback link isformulated and the radio resource allocation problem, in terms of transmission energy, time, and bandwidth of the forward and feedback links is presented.En un mundo cada vez más interconectado, donde hay una clara tendencia hacia un mayor número de comunicaciones inalámbricas simultáneas (comunicaciones M2M: Machine to Machine, redes de sensores, etc.) y en el que las necesidades de capacidad de transmisión de los enlaces de comunicaciones aumentan de manera vertiginosa (audio, video, contenidos multimedia, alta definición, etc.) el problema de la interferencia se convierte en uno de los factores limitadores de los enlaces junto con los desvanecimientos del nivel de señal y las pérdidas de propagación. Por este motivo los sistemas que emplean múltiples antenas tanto en la transmisión como en la recepción (los llamados sistemas MIMO: Multiple-Input Multiple-Output) se presentan como una de las soluciones más interesantes para satisfacer los crecientes requisitos de capacidad y comportamiento relativo a interferencias. Los sistemas MIMO permiten obtener un mejor rendimiento en términos de tasa de transmisión de información y a su vez son más robustos frente a ruido e interferencias en el canal. Esto significa que pueden usarse para aumentar la capacidad de los enlaces de comunicaciones actuales o para reducir drásticamente el consumo energético manteniendo las mismas prestaciones. Por otro lado, además de estas claras ventajas, los sistemas MIMO introducen un punto de complejidad adicional puesto que para aprovechar al máximo las posibilidades de estos sistemas es necesario tener conocimiento de la información de estado del canal (CSI: Channel State Information) tanto en el transmisor como en el receptor. Esta CSI se obtiene mediante estimación de canal en el receptor y posteriormente se envía al transmisor a través de un canal de realimentación. Esta tesis trata sobre el diseño del canal de realimentación para la transmisión de CSI, que es un elemento fundamental de los sistemas de comunicaciones del presente y del futuro. Las técnicas de transmisión que consideran activamente el efecto de la interferencia y el ruido requieren adaptarse al canal y, para ello, la realimentación de CSI es necesaria. En esta tesis se identifica, en primer lugar, la mínima información sobre el estado del canal necesaria para implementar un diseño óptimo en el transmisor, con el fin de evitar transmitir información redundante y obtener así un sistema más eficiente. Esta información es la matriz de Gram del canal MIMO. Seguidamente, se desarrolla un algoritmo de cuantificación adaptado a la geometría diferencial del conjunto que contiene la información a cuantificar y que además aprovecha la correlación temporal existente en los canales de propagación inalámbricos. Este algoritmo se implementa y evalúa primero en comunicaciones MIMO punto a punto entre dos usuarios, después se implementa para algunos casos particulares con múltiples usuarios, y finalmente se amplía para el caso general de sistemas broadcast multi-usuario. Adicionalmente, esta tesis también estudia y optimiza el dimensionamiento del canal de realimentación en función de la cantidad de recursos radio disponibles, en términos de ancho de banda, tiempo y potencia de transmisión. Para ello presenta el problema de la distribución óptima de dichos recursos radio entre el enlace de transmisión de datos y el enlace de realimentación para transmisión de información sobre estado del canal como un problema de optimización