Quantization Over Discrete Noisy Channels Under Complexity Constraints

A fundamental problem in communication is the transmission of an information source across a communication channel. According to Shannon's separation principle, this problem can be separated (without loss of optimality) into two different, yet similar, problems: source coding and channel coding. This result, however, holds only when complexity and delay are not an issue. In practical situations, complexity plays a major role in many system designs. When complexity is constrained, treating these two problems jointly may prove to be more fruitful than treating them separately.In this work we consider two approaches to joint source-channel coding of discrete-time, continuous- amplitude sources and discrete memoryless channels when complexity is constrained.In the first approach, we consider the analysis and design of two low-complexity vector quantizer - the tree-structured vector quantizer (TSVQ) and the multistage vector quantizer (MSVQ) - when used over a noisy channel. The resulting schemes are called channel-matched TSVQ and channel- matched MSVQ. These schemes are compared with (i) the ordinary TSVQ and MSVQ which are designed for the noiseless channel and (ii) a tandem source-channel coding scheme in which the source and channel codes are designed separately.In the second approach, we assume a low-complexity quantizer (i.e., source code) is given. Because of its low complexity, the quantizer is sub-optimum and hence certain redundancy remains at its output. Our aim is to make use of this redundancy to combat channel noise. We consider two scenarios: (i) the redundancy is in the form of memory and (ii) it is in the form of a non-uniform distribution.In the second case, we propose the use of a rate- one convolutional code to convert the residual redundancy into a usable form. Comparisons are also made with a tandem source- channel coding scheme

Optimal Detection of Discrete Markov Sources Over Discrete Memoryless Channels - Applications to Combined Sources-Channel Coding

We consider the problem of detecting a discrete Markov source which is transmitted across a discrete memoryless channel. The detection is based upon the maximum a posteriori (MAP) criterion which yields the minimum probability of error for a given observation. Two formulations of this problem are considered: (i) a sequence MAP detection in which the objective is to determine the most probable transmitted sequence given the observed sequence and (ii) an instantaneous MAP detection which is to determine the most probable transmitted symbol at time n given all the observations prior to and including time n. The solution to the first problem results in a "Viterbi-like" implementation of the MAP detector (with large delay) while the later problem results in a recursive (with no delay). For the special case of the binary symmetric Markov source and binary symmetric channel, simulation results are presented and an analysis of these two systems yields explicit critical channel bit error rates above which the MAP detectors become useful.Applications of the MAP detection problem in a combined source-channel coding system are considered. Here it is assumed that the source is highly correlated and that the source encoder (in our case, a vector quantizer (VQ) fails to remove all of the source redundancy. The remaining redundancy at the output of the source encoder is referred to as the "residual" redundancy. It is shown, through simulation, that the residual redundancy can be used by the MAP detectors to combat channel errors. For small block sizes, the proposed system beats Farvardin and Vaishampayan's channel- optimized VQ by wide margins. Finally, it is shown that the instantaneous MAP detector can be combined with the VQ decoder to form a minimum mean-squared error decoder. Simulation results are also given for this case

Quantization Over Discrete Noisy Channels Using Rate-One Convolutional Codes

We consider high-rate scalar quantization of a memoryless source for transmission over a binary symmetric channel. It is assumed that, due to its suboptimality, the quantizer's output is redundant. Our aim is to make use of this redundancy to combat channel noise. A rate-one convolutional code is introduced to convert this natural redundancy into a usable form. at the receiver, a maximum a posteriori decoder is employed. An upper bound on the average distortion of the proposed system is derived. An approximation of this bound is computable and we search for that convolutional code which minimizes the approximate upper bound. simulation results for a generalized Gaussian source with parameter a = 0.5 at rate 4 bits/sample and channel crossover provability 0.005 show improvement of 11.9 dB in signal-to-noise ratio over the Lloyd-Max quantizer and 4.6 dB over Farvardin and Vaishampayan's channel-optimized scalar quantizer

Channel Codes That Exploit the Residual Redundancy in CELP- Encoded Speech

We consider the problem of reliably transmitting CELP-encoded speech over noisy communication channels. Our objective is to design efficient coding/decoding schemes for the transmission of the CELP line spectral parameters (LSP's) over very noisy channels.We begin by quantifying the amount of ﲲesidual redundancy inherent in the LSP's of Federal Standard 1016 CELP. This is done by modeling the LSP's as first and second-order Markov chains. Two models for LSP generation are proposed; the first model characterizes the intra-frame correlation exhibited by the LSP's, while the second model captures both intra-frame and inter-frame correlation. By comparing the entropy rates of the models thus constructed with the CELP rates, it is shown that as many as one-third of the LSP bits in every frame of speech are redundant.We next consider methods by which this residual redundancy can be exploited by an appropriately designed channel decoder. Before transmission, the LSP's are encoded with a forward error control (FEC) code; we consider both block (Reed- Solomon) codes and convolutional codes. Soft-decision decoders that exploit the residual redundancy in the LSP's are implemented assuming additive white Gaussian noise (AWGN) and independent Rayleigh fading environments. Simulation results employing binary phaseshift keying (BPSK) indicate coding gains of 2 to 5 dB over soft-decision decoders that do not exploit the residual redundancy

A United Approach to Tree-Structured and Multi-Stage Vector Quantization for Noisy Channels

Vector quantization (VQ) is a powerful and effective scheme which is widely used in speech and image coding applications. Two basic problems can be associated with VQ: (i) its large encoding complexity, and (ii) its sensitivity to channel errors. These two problems have been independently studied in the past. In this paper, we examine these two problems jointly. Specifically, the performances of two low-complexity VQs-the tree-structured VQ (TSVQ) and the multi-stage VQ (MSVQ) - when used over noisy channels are analyzed. An algorithms is developed for the design of channel-matched TSVQ (CM-TSVQ) and channel-matched MSVQ (CM- MSVQ) under the squared-error criterion. Extensive numerical results are given for the memoryless Gaussian source and the Gauss-Markov source with correlation coefficient 0.9. Comparisons with the ordinary TSVQ and MSVQ designed for the noiseless channel show substantial improvements when the channel is very noisy. The CM-MSVQ, which can be regarded as a block- structured combined source-channel coding scheme, is then compared with a block-structured tandem source-channel coding scheme (with the same block length as the CM-MSVQ). For the Gauss-Markov source, the CM-MSVQ outperforms the tandem scheme in all cases which we have considered. Furthermore, it is demonstrated that the CM-MSVQ is fairly robust to channel mismatch

Quantization of Memoryless and Gauss-Markov Sources Over Binary Markov Channels

Joint source-channel coding for stationary memoryless and Gauss- Markov sources and binary Markov channels is considered. The channel is an additive-noise channel where the noise process is an M-th order Markov chain. Two joint source-channel coding schemes are considered. The first is a channel-optimized vector quantizer - optimized for both source and channel. The second scheme consists of a scalar quantizer and a maximum a posteriori detector. In this scheme, it is assumed that the scalar quantizer output has residual redundancy that can be exploited by the maximum a posteriori detector to combat the correlated channel noise. These two schemes are then compared against two schemes which use channel interleaving. Numerical results show that the proposed schemes outperform the interleaving schemes. For very noisy channels with high noise correlation, gains of 4 to 5 dB in signal-to-noise ratio are possible

Detection of Binary Sources Over Discrete Channels with Additive Markov Noise

We consider the problem of directly transmitting a binary source with an inherent redundancy over a binary channel with additive stationary ergodic Markov noise. Out objective is to design an optimum receiver which fully utilizes the source redundancy in order to combat the channel noise.We investigate the problem of detecting a binary iid non-uniform source transmitted across the Markov channel. Two maximum a posteriori (MAP) formulations are considered: a sequence MAP detection and an instantaneous MAP detection. The two MAP detection problems are implemented using a modified version of the Viterbi decoding algorithm and a recursive algorithm. Necessary and sufficient conditions under which the sequence MAP detector becomes useless as well as simulation results are presented. A comparison between the performance of the proposed system with that of a (substantially more complex) traditional tandem source-channel coding scheme exhibits a better performance for the proposed scheme at relatively high channel bit error rates.The same detection problem is then analyzed for the case of a binary symmetric Markov source. Analytical and simulation results show the existence of a "mismatch" between the source and the channel. This mismatch is reduced by the use of a rate-one convolutional encoder. Finally, the detection problem is generalized for the case of a binary non-symmetric Markov source