Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.Comment: 28 pages, 14 figure

Zero-Delay Joint Source-Channel Coding in the Presence of Interference Known at the Encoder

Zero-delay transmission of a Gaussian source over an additive white Gaussian noise (AWGN) channel is considered in the presence of an additive Gaussian interference signal. The mean squared error (MSE) distortion is minimized under an average power constraint assuming that the interference signal is known at the transmitter. Optimality of simple linear transmission does not hold in this setting due to the presence of the known interference signal. While the optimal encoder-decoder pair remains an open problem, various non-linear transmission schemes are proposed in this paper. In particular, interference concentration (ICO) and one-dimensional lattice (1DL) strategies, using both uniform and non-uniform quantization of the interference signal, are studied. It is shown that, in contrast to typical scalar quantization of Gaussian sources, a non-uniform quantizer, whose quantization intervals become smaller as we go further from zero, improves the performance. Given that the optimal decoder is the minimum MSE (MMSE) estimator, a necessary condition for the optimality of the encoder is derived, and the numerically optimized encoder (NOE) satisfying this condition is obtained. Based on the numerical results, it is shown that 1DL with nonuniform quantization performs closer (compared to the other schemes) to the numerically optimized encoder while requiring significantly lower complexity

Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, thereby providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications

Zero-delay source-channel coding

In this thesis, we investigate the zero-delay transmission of source samples over three different types of communication channel models. First, we consider the zero-delay transmission of a Gaussian source sample over an additive white Gaussian noise (AWGN) channel in the presence of an additive white Gaussian (AWG) interference, which is fully known by the transmitter. We propose three parameterized linear and non-linear transmission schemes for this scenario, and compare the corresponding mean square error (MSE) performances with that of a numerically optimized encoder, obtained using the necessary optimality conditions. Next, we consider the zero-delay transmission of a Gaussian source sample over an AWGN channel with a one-bit analog-to-digital (ADC) front end. We study this problem under two different performance criteria, namely the MSE distortion and the distortion outage probability (DOP), and obtain the optimal encoder and the decoder for both criteria. As generalizations of this scenario, we consider the performance with a K-level ADC front end as well as with multiple one-bit ADC front ends. We derive necessary conditions for the optimal encoder and decoder, which are then used to obtain numerically optimized encoder and decoder mappings. Finally, we consider the transmission of a Gaussian source sample over an AWGN channel with a one-bit ADC front end in the presence of correlated side information at the receiver. Again, we derive the necessary optimality conditions, and using these conditions obtain numerically optimized encoder and decoder mappings. We also consider the scenario in which the side information is available also at the encoder, and obtain the optimal encoder and decoder mappings. The performance of the latter scenario serves as a lower bound on the performance of the case in which the side information is available only at the decoder.Open Acces

Joint Source-Channel Coding Optimized On End-to-End Distortion for Multimedia Source

In order to achieve high efficiency, multimedia source coding usually relies on the use of predictive coding. While more efficient, source coding based on predictive coding has been considered to be more sensitive to errors during communication. With the current volume and importance of multimedia communication, minimizing the overall distortion during communication over an error-prone channel is critical. In addition, for real-time scenarios, it is necessary to consider additional constraints such as fix and small delay for a given bit rate. To comply with these requirements, we seek an efficient joint source-channel coding scheme. In this work, end-to-end distortion is studied for a first order autoregressive synthetic source that represents a general multimedia traffic. This study reveals that predictive coders achieve the same channel-induced distortion performance as memoryless codecs when applying optimal error concealment. We propose a joint source-channel system based on incremental redundancy that satisfies the fixed delay and error-prone channel constraints and combines DPCM as a source encoder and a rate-compatible punctured convolutional (RCPC) error control codec. To calculate the joint source-channel coding rate allocation that minimizes end-to-end distortion, we develop a Markov Decision Process (MDP) approach for delay constrained feedback Hybrid ARQ, and we use a Dynamic Programming (DP) technique. Our simulation results support the improvement in end-to-end distortion compared to a conventional Forward Error Control (FEC) approach with no feedback

Side information aware source and channel coding in wireless networks

Signals in communication networks exhibit significant correlation, which can stem from the physical nature of the underlying sources, or can be created within the system. Current layered network architectures, in which, based on Shannon’s separation theorem, data is compressed and transmitted over independent bit-pipes, are in general not able to exploit such correlation efficiently. Moreover, this strictly layered architecture was developed for wired networks and ignore the broadcast and highly dynamic nature of the wireless medium, creating a bottleneck in the wireless network design. Technologies that exploit correlated information and go beyond the layered network architecture can become a key feature of future wireless networks, as information theory promises significant gains. In this thesis, we study from an information theoretic perspective, three distinct, yet fundamental, problems involving the availability of correlated information in wireless networks and develop novel communication techniques to exploit it efficiently. We first look at two joint source-channel coding problems involving the lossy transmission of Gaussian sources in a multi-terminal and a time-varying setting in which correlated side information is present in the network. In these two problems, the optimality of Shannon’s separation breaks down and separate source and channel coding is shown to perform poorly compared to the proposed joint source-channel coding designs, which are shown to achieve the optimal performance in some setups. Then, we characterize the capacity of a class of orthogonal relay channels in the presence of channel side information at the destination, and show that joint decoding and compression of the received signal at the relay is required to optimally exploit the available side information. Our results in these three different scenarios emphasize the benefits of exploiting correlated side information at the destination when designing a communication system, even though the nature of the side information and the performance measure in the three scenarios are quite different.Open Acces

Distributed Cooperative Spatial Multiplexing System

Multiple-Input-Multiple-Output (MIMO) spatial multiplexing systems can increase the spectral efficiency manyfold, without extra bandwidth or transmit power, however these advantages are based on the assumption that channels between different transmit antenna and receive antenna are independent which requires the elements in antenna array be separated by several wavelengths. For small mobile devices, the requirement is difficult to implement in practice. Cooperative spatial multiplexing (C-SM) system provides a solution: it organizes antennas on distributed mobile stations to form a virtual antenna array (VAA) to support spatial multiplexing. In this thesis, we propose a novel C-SM system design which includes a transmitter with two antennas, a single antenna receiver and a relay group with two single antenna relays. In this design, we assume that the transmitter tries to transmit two coded independent messages to the receiver simultaneously but the transmitter-receiver link is too weak to support the transmission. Thus a relay group is introduced to help with the transmission. After relays receive the messages from the transmitter via a $2\times 2$ MIMO link, they first detect and quantize the received messages, then compress them independently according to the Slepian and Wolf theorem, the compressed messages are sent to the receiver simultaneously where de-compression and de-quantization are performed on the received messages. After that the resulting messages are combined to estimate the original coded messages. The estimated coded messages are decoded to produce the original messages. The basic system structure is studied and an analytical bit error rate expression is derived. Several transmission protocols are also introduced to enhance the system BER performance. The merit of this design is focus on the relay destination link. Because the Slepian and Wolf theorem is applied on the relay-destination link, messages at the relays can be compressed independently and de-compressed jointly at the receiver with arbitrarily small error probability but still achieve the same compression rate as a joint compression scheme does. The Slepian and Wolf theorem is implemented by a joint source-channel code in this thesis. Several schemes are introduced and tested, the testing results and performance analysis are given in this thesis. According to the chief executive officer (CEO) problem in the network information theory, we discover an error floor in this design. An analytical expression for this error floor is derived. A feedback link is also introduced from the receiver to the relays to allow the relays to cooperatively adapt their compression rates to the qualities of the received messages. Two combination schemes at the receiver are introduced, their performances are examined from the information theory point of view, the results and performance analysis are given in this thesis. As we assume that the relay destination link is a multiple access channel (MAC) suffers from block Rayleigh fading and white Gaussian noise, the relationship between the MAC channel capacity and the Slepian and Wolf compression rate region is studied to analyse the system performance

On distributed coding, quantization of channel measurements and faster-than-Nyquist signaling

This dissertation considers three different aspects of modern digital communication systems and is therefore divided in three parts. The first part is distributed coding. This part deals with source and source- channel code design issues for digital communication systems with many transmitters and one receiver or with one transmitter and one receiver but with side information at the receiver, which is not available at the transmitter. Such problems are attracting attention lately, as they constitute a way of extending the classical point-to-point communication theory to networks. In this first part of this dissertation, novel source and source-channel codes are designed by converting each of the considered distributed coding problems into an equivalent classical channel coding or classical source-channel coding problem. The proposed schemes come very close to the theoretical limits and thus, are able to exhibit some of the gains predicted by network information theory. In the other two parts of this dissertation classical point-to-point digital com- munication systems are considered. The second part is quantization of coded chan- nel measurements at the receiver. Quantization is a way to limit the accuracy of continuous-valued measurements so that they can be processed in the digital domain. Depending on the desired type of processing of the quantized data, different quantizer design criteria should be used. In this second part of this dissertation, the quantized received values from the channel are processed by the receiver, which tries to recover the transmitted information. An exhaustive comparison of several quantization cri- teria for this case are studied providing illuminating insight for this quantizer design problem. The third part of this dissertation is faster-than-Nyquist signaling. The Nyquist rate in classical point-to-point bandwidth-limited digital communication systems is considered as the maximum transmission rate or signaling rate and is equal to twice the bandwidth of the channel. In this last part of the dissertation, we question this Nyquist rate limitation by transmitting at higher signaling rates through the same bandwidth. By mitigating the incurred interference due to the faster-than-Nyquist rates, gains over Nyquist rate systems are obtained

Structural Results for Coding Over Communication Networks

We study the structure of optimality achieving codes in network communications. The thesis consists of two parts: in the first part, we investigate the role of algebraic structure in the performance of communication strategies. In chapter two, we provide a linear coding scheme for the multiple-descriptions source coding problem which improves upon the performance of the best known unstructured coding scheme. In chapter three, we propose a new method for lattice-based codebook generation. The new method leads to a simplification in the analysis of the performance of lattice codes in continuous-alphabet communication. In chapter four, we show that although linear codes are necessary to achieve optimality in certain problems, loosening the closure restriction in the codebook leads to gains in other network communication settings. We introduce a new class of structured codes called quasi-linear codes (QLC). These codes cover the whole spectrum between unstructured codes and linear codes. We develop coding strategies in the interference channel and the multiple-descriptions problems using QLCs which outperform the previous schemes. In the second part, which includes the last two chapters, we consider a different structural restriction on codes used in network communication. Namely, we limit the `effective length' of these codes. First, we consider an arbitrary pair of Boolean functions which operate on two sequences of correlated random variables. We derive a new upper-bound on the correlation between the outputs of these functions. The upper-bound is presented as a function of the `dependency spectrum' of the corresponding Boolean functions. Next, we investigate binary block-codes (BBC). A BBC is defined as a vector of Boolean functions. We consider BBCs which are generated randomly, and using single-letter distributions. We characterize the vector of dependency spectrums of these BBCs. This gives an upper-bound on the correlation between the outputs of two distributed BBCs. Finally, the upper-bound is used to show that the large blocklength single-letter coding schemes in the literature are sub-optimal in various multiterminal communication settings.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/137059/1/fshirani_1.pd