Side information exploitation, quality control and low complexity implementation for distributed video coding

Distributed video coding (DVC) is a new video coding methodology that shifts the highly complex motion search components from the encoder to the decoder, such a video coder would have a great advantage in encoding speed and it is still able to achieve similar rate-distortion performance as the conventional coding solutions. Applications include wireless video sensor networks, mobile video cameras and wireless video surveillance, etc. Although many progresses have been made in DVC over the past ten years, there is still a gap in RD performance between conventional video coding solutions and DVC. The latest development of DVC is still far from standardization and practical use. The key problems remain in the areas such as accurate and efficient side information generation and refinement, quality control between Wyner-Ziv frames and key frames, correlation noise modelling and decoder complexity, etc. Under this context, this thesis proposes solutions to improve the state-of-the-art side information refinement schemes, enable consistent quality control over decoded frames during coding process and implement highly efficient DVC codec. This thesis investigates the impact of reference frames on side information generation and reveals that reference frames have the potential to be better side information than the extensively used interpolated frames. Based on this investigation, we also propose a motion range prediction (MRP) method to exploit reference frames and precisely guide the statistical motion learning process. Extensive simulation results show that choosing reference frames as SI performs competitively, and sometimes even better than interpolated frames. Furthermore, the proposed MRP method is shown to significantly reduce the decoding complexity without degrading any RD performance. To minimize the block artifacts and achieve consistent improvement in both subjective and objective quality of side information, we propose a novel side information synthesis framework working on pixel granularity. We synthesize the SI at pixel level to minimize the block artifacts and adaptively change the correlation noise model according to the new SI. Furthermore, we have fully implemented a state-of-the-art DVC decoder with the proposed framework using serial and parallel processing technologies to identify bottlenecks and areas to further reduce the decoding complexity, which is another major challenge for future practical DVC system deployments. The performance is evaluated based on the latest transform domain DVC codec and compared with different standard codecs. Extensive experimental results show substantial and consistent rate-distortion gains over standard video codecs and significant speedup over serial implementation. In order to bring the state-of-the-art DVC one step closer to practical use, we address the problem of distortion variation introduced by typical rate control algorithms, especially in a variable bit rate environment. Simulation results show that the proposed quality control algorithm is capable to meet user defined target distortion and maintain a rather small variation for sequence with slow motion and performs similar to fixed quantization for fast motion sequence at the cost of some RD performance. Finally, we propose the first implementation of a distributed video encoder on a Texas Instruments TMS320DM6437 digital signal processor. The WZ encoder is efficiently implemented, using rate adaptive low-density-parity-check accumulative (LDPCA) codes, exploiting the hardware features and optimization techniques to improve the overall performance. Implementation results show that the WZ encoder is able to encode at 134M instruction cycles per QCIF frame on a TMS320DM6437 DSP running at 700MHz. This results in encoder speed 29 times faster than non-optimized encoder implementation. We also implemented a highly efficient DVC decoder using both serial and parallel technology based on a PC-HPC (high performance cluster) architecture, where the encoder is running in a general purpose PC and the decoder is running in a multicore HPC. The experimental results show that the parallelized decoder can achieve about 10 times speedup under various bit-rates and GOP sizes compared to the serial implementation and significant RD gains with regards to the state-of-the-art DISCOVER codec

ADAPTIVE CHANNEL AND SOURCE CODING USING APPROXIMATE INFERENCE

Channel coding and source coding are two important problems in communications. Although both channel coding and source coding (especially, the distributed source coding (DSC)) can achieve their ultimate performance by knowing the perfect knowledge of channel noise and source correlation, respectively, such information may not be always available at the decoder side. The reasons might be because of the time−varying characteristic of some communication systems and sources themselves, respectively. In this dissertation, I mainly focus on the study of online channel noise estimation and correlation estimation by using both stochastic and deterministic approximation inferences on factor graphs.In channel coding, belief propagation (BP) is a powerful algorithm to decode low−density parity check (LDPC) codes over additive white Gaussian noise (AWGN) channels. However, the traditional BP algorithm cannot adapt efficiently to the statistical change of SNR in an AWGN channel. To solve the problem, two common workarounds in approximate inference are stochastic methods (e.g. particle filtering (PF)) and deterministic methods (e.g. expectation approximation (EP)). Generally, deterministic methods are much faster than stochastic methods. In contrast, stochastic methods are more flexible and suitable for any distribution. In this dissertation, I proposed two adaptive LDPC decoding schemes, which are able to perform online estimation of time−varying channel state information (especially signal to noise ratio (SNR)) at the bit−level by incorporating PF and EP algorithms. Through experimental results, I compare the performance between the proposed PF based and EP based approaches, which shows that the EP based approach obtains the comparable estimation accuracy with less computational complexity than the PF based method for both stationary and time−varying SNR, and enhances the BP decoding performance simultaneously. Moreover, the EP estimator shows a very fast convergence speed, and the additional computational overhead of the proposed decoder is less than 10% of the standard BP decoder.Moreover, since the close relationship between source coding and channel coding, the proposed ideas are extended to source correlation estimation. First, I study the correlation estimation problem in lossless DSC setup, where I consider both asymmetric and non−asymmetric SW coding of two binary correlated sources. The aforementioned PF and EP based approaches are extended to handle the correlation between two binary sources, where the relationship is modeled as a virtual binary symmetric channel (BSC) with a time−varying crossover probability. Besides, to handle the correlation estimation problem of Wyner−Ziv (WZ) coding, a lossy DSC setup, I design a joint bit−plane model, by which the PF based approach can be applied to tracking the correlation between non−binary sources. Through experimental results, the proposed correlation estimation approaches significantly improve the compression performance of DSC.Finally, due to the property of ultra−low encoding complexity, DSC is a promising technique for many tasks, in which the encoder has only limited computing and communication power, e.g. the space imaging systems. In this dissertation, I consider a real−world application of the proposed correlation estimation scheme on the onboard low−complexity compression of solar stereo images, since such solutions are essential to reduce onboard storage, processing, and communication resources. In this dissertation, I propose an adaptive distributed compression solution using PF that tracks the correlation, as well as performs disparity estimation, at the decoder side. The proposed algorithm istested on the stereo solar images captured by the twin satellites systemof NASA’s STEREO project. The experimental results show the significant PSNR improvement over traditional separate bit−plane decoding without dynamic correlation and disparity estimation

Improving the Rate-Distortion Performance in Distributed Video Coding

Distributed video coding is a coding paradigm, which allows encoding of video frames at a complexity that is substantially lower than that in conventional video coding schemes. This feature makes it suitable for some emerging applications such as wireless surveillance video and mobile camera phones. In distributed video coding, a subset of frames in the video sequence, known as the key frames, are encoded using a conventional intra-frame encoder, such as H264/AVC in the intra mode, and then transmitted to the decoder. The remaining frames, known as the Wyner-Ziv frames, are encoded based on the Wyner-Ziv principle by using the channel codes, such as LDPC codes. In the transform-domain distributed video coding, each Wyner-Ziv frame undergoes a 4x4 block DCT transform and the resulting DCT coefficients are grouped into DCT bands. The bitplaines corresponding to each DCT band are encoded by a channel encoder, for example an LDPCA encoder, one after another. The resulting error-correcting bits are retained in a buffer at the encoder and transmitted incrementally as needed by the decoder. At the decoder, the key frames are first decoded. The decoded key frames are then used to generate a side information frame as an initial estimate of the corresponding Wyner-Ziv frame, usually by employing an interpolation method. The difference between the DCT band in the side information frame and the corresponding one in the Wyner-Ziv frame, referred to as the correlation noise, is often modeled by Laplacian distribution. A soft-input information for each bit in the bitplane is obtained using this correlation noise model and the corresponding DCT band of the side information frame. The channel decoder then uses this soft-input information along with some error-correcting bits sent by the encoder to decode the bitplanes of each DCT band in each of the Wyner-Ziv frames. Hence, an accurate estimation of the correlation noise model parameter(s) and generation of high-quality side information are required for reliable soft-input information for the bitplanes in the decoder, which in turn leads to a more efficient decoding. Consequently, less error-correcting bits need to be transmitted from the encoder to the decoder to decode the bitplanes, leading to a better compression efficiency and rate-distortion performance. The correlation noise is not stationary and its statistics vary within each Wyner-Ziv frame and within its corresponding DCT bands. Hence, it is difficult to find an accurate model for the correlation noise and estimate its parameters precisely at the decoder. Moreover, in existing schemes the parameters of the correlation noise for each DCT band are estimated before the decoder starts to decode the bitplanes of that DCT band and they are not modified and kept unchanged during decoding process of the bitplanes. Another problem of concern is that, since side information frame is generated in the decoder using the temporal interpolation between the previously decoded frames, the quality of the side information frames is generally poor when the motions between the frames are non-linear. Hence, generating a high-quality side information is a challenging problem. This thesis is concerned with the study of accurate estimation of correlation noise model parameters and increasing in the quality of the side information from the standpoint of improving the rate-distortion performance in distributed video coding. A new scheme is proposed for the estimation of the correlation noise parameters wherein the decoder decodes simultaneously all the bitplanes of a DCT band in a Wyner-Ziv frame and then refines the parameters of the correlation noise model of the band in an iterative manner. This process is carried out on an augmented factor graph using a new recursive message passing algorithm, with the side information generated and kept unchanged during the decoding of the Wyner-Ziv frame. Extensive simulations are carried out showing that the proposed decoder leads to an improved rate-distortion performance in comparison to the original DISCOVER codec and in another DVC codec employing side information frame refinement, particularly for video sequences with high motion content. In the second part of this work, a new algorithm for the generation of the side information is proposed to refine the initial side information frame using the additional information obtained after decoding the previous DCT bands of a Wyner-Ziv frame. The simulations are carried out demonstrating that the proposed algorithm provides a performance superior to that of schemes employing the other side information refinement mechanisms. Finally, it is shown that incorporating the proposed algorithm for refining the side information into the decoder proposed in the first part of the thesis leads to a further improvement in the rate-distortion performance of the DVC codec

Intra-Key-Frame Coding and Side Information Generation Schemes in Distributed Video Coding

In this thesis investigation has been made to propose improved schemes for intra-key-frame coding and side information (SI) generation in a distributed video coding (DVC) framework. From the DVC developments in last few years it has been observed that schemes put more thrust on intra-frame coding and better quality side information (SI) generation. In fact both are interrelated as SI generation is dependent on decoded key frame quality. Hence superior quality key frames generated through intra-key frame coding will in turn are utilized to generate good quality SI frames. As a result, DVC needs less number of parity bits to reconstruct the WZ frames at the decoder. Keeping this in mind, we have proposed two schemes for intra-key frame coding namely, (a) Borrows Wheeler Transform based H.264/AVC (Intra) intra-frame coding (BWT-H.264/AVC(Intra)) (b) Dictionary based H.264/AVC (Intra) intra-frame coding using orthogonal matching pursuit (DBOMP-H.264/AVC (Intra)) BWT-H.264/AVC (Intra) scheme is a modified version of H.264/AVC (Intra) scheme where a regularized bit stream is generated prior to compression. This scheme results in higher compression efficiency as well as high quality decoded key frames. DBOMP-H.264/AVC (Intra) scheme is based on an adaptive dictionary and H.264/AVC (Intra) intra-frame coding. The traditional transform is replaced with a dictionary trained with K-singular value decomposition (K-SVD) algorithm. The dictionary elements are coded using orthogonal matching pursuit (OMP). Further, two side information generation schemes have been suggested namely, (a) Multilayer Perceptron based side information generation (MLP - SI) (b) Multivariable support vector regression based side information generation (MSVR-SI) MLP-SI scheme utilizes a multilayer perceptron (MLP) to estimate SI frames from the decoded key frames block-by-block. The network is trained offline using training patterns from different frames collected from standard video sequences. MSVR-SI scheme uses an optimized multi variable support vector regression (M-SVR) to generate SI frames from decoded key frames block-by-block. Like MLP, the training for M-SVR is made offline with known training patterns apriori. Both intra-key-frame coding and SI generation schemes are embedded in the Stanford based DVC architecture and studied individually to compare performances with their competitive schemes. Visual as well as quantitative evaluations have been made to show the efficacy of the schemes. To exploit the usefulness of intra-frame coding schemes in SI generation, four hybrid schemes have been formulated by combining the aforesaid suggested schemes as follows: (a) BWT-MLP scheme that uses BWT-H.264/AVC (Intra) intra-frame coding scheme and MLP-SI side information generation scheme. (b) BWT-MSVR scheme, where we utilize BWT-H.264/AVC (Intra) for intra-frame coding followed by MSVR-SI based side information generation. (c) DBOMP-MLP scheme is an outcome of putting DBOMP-H.264/AVC (Intra) intra-frame coding and MLP-SI side information generation schemes. (d) DBOMP-MSVR scheme deals with DBOMP-H.264/AVC (Intra) intra-frame coding and MSVR-SI side information generation together. The hybrid schemes are also incorporated into the Stanford based DVC architecture and simulation has been carried out on standard video sequences. The performance analysis with respect to overall rate distortion, number requests per SI frame, temporal evaluation, and decoding time requirement has been made to derive an overall conclusion