Improvement and optimization of H.264 video codec.

Tang, Kai Lam.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references.Abstracts in English and Chinese.Acknowledgement --- p.iAbstract --- p.iiContents --- p.ivPublication List --- p.viiChapter Chapter 1 --- Introduction --- p.1-1Chapter 1.1 --- Video Coding --- p.1-1Chapter 1.1.1 --- Temporal prediction --- p.1-5Chapter 1.1.2 --- Transform Coding --- p.1-9Chapter 1.1.3 --- Quantization --- p.1-12Chapter 1.1.4 --- Entropy Coding --- p.1-14Chapter 1.2 --- H.264/MPEG-4 Part 10 --- p.1-15Chapter 1.2.1 --- Overview --- p.1-16Chapter 1.2.2 --- Intra Prediction --- p.1-19Chapter 1.2.3 --- Inter Prediction --- p.1-20Chapter 1.2.4 --- Transform and Quantization --- p.1-23Chapter 1.2.5 --- Entropy Coding --- p.1-25Chapter 1.2.6 --- Deblocking Filter --- p.1-29Chapter 1.3 --- Organization of the Thesis --- p.1-32Chapter 1.3.1 --- Review of Motion Estimation Techniques --- p.1-32Chapter 1.3.2 --- The Proposed Algorithms --- p.1-33Chapter 1.3.3 --- Optimization of the Codec --- p.1-34Chapter 1.4 --- Contributions --- p.1-35Chapter Chapter 2 --- Review of Motion Estimation Techniques --- p.2-1Chapter 2.1 --- Fast Full Search --- p.2-2Chapter 2.2 --- Hybrid Unsymmetrical-cross Multi-Hexagon-grid Search --- p.2-4Chapter 2.3 --- Center biased Fractional Pel Search --- p.2-6Chapter 2.4 --- Enhanced Predictive Zonal Search --- p.2-7Chapter Chapter 3 --- Enhancement Techniques for Intra Block Matching --- p.3-1Chapter 3.1 --- Introduction --- p.3-1Chapter 3.1.1 --- Fundamental Principles --- p.3-1Chapter 3.1.2 --- Variable Block Size Intra Block Matching --- p.3-3Chapter 3.2 --- Proposed Techniques --- p.3-5Chapter 3.2.1 --- Padding --- p.3-5Chapter 3.2.2 --- Modes --- p.3-9Chapter 3.2.3 --- Performance Enhancement Tools --- p.3-12Chapter 3.2.3.1 --- Multiple Best Matches --- p.3-12Chapter 3.2.3.2 --- Adaptive Integer and Sub-pixel Intra Block Matching --- p.3-13Chapter 3.2.4 --- Pseudo Intra Block Matching --- p.3-14Chapter 3.3 --- Proposed Fast Algorithms --- p.3-16Chapter 3.3.1 --- Fast Intra Block Matching Decision --- p.3-16Chapter 3.3.2 --- Skipping some Intra Block Matching Processes --- p.3-18Chapter 3.3.3 --- Early Termination --- p.3-19Chapter 3.3.4 --- SAD Reuse Techniques --- p.3-21Chapter 3.4 --- Experimental Results --- p.3-22Chapter Chapter 4 --- Enhanced SAD Reuse Fast Motion Estimation --- p.4-1Chapter 4.1 --- Introduction --- p.4-1Chapter 4.2 --- Proposed Fast Motion Estimation Algorithm --- p.4-3Chapter 4.2.1 --- Best Initial Motion Vector --- p.4-3Chapter 4.2.2 --- Initial Search Pattern --- p.4-4Chapter 4.2.3 --- Initial Search Process and Search Pattern Improvement Process --- p.4-7Chapter 4.2.3.1 --- BISPCSP Motion Estimation or Refinement Process Decision --- p.4-8Chapter 4.2.3.2 --- ISP Motion Estimation or Refinement Process Decision --- p.4-9Chapter 4.2.4 --- Motion Estimation Process and Refinement Process --- p.4-9Chapter 4.2.4.1 --- Motion Estimation Process --- p.4-9Chapter 4.2.4.2 --- Refinement Process --- p.4-11Chapter 4.2.5 --- Motion Estimation Skip Process for B Pictures --- p.4-12Chapter 4.3 --- Experimental Results --- p.4-13Chapter Chapter 5 --- Development of Real-Time H.264 Codec on Pocket PC --- p.5-1Chapter 5.1 --- Algorithmic Optimizations --- p.5-2Chapter 5.1.1 --- Fast Sub-Pixel Motion Estimation --- p.5-2Chapter 5.1.2 --- Interpolation --- p.5-5Chapter 5.1.2.1 --- Revision of Luma Interpolation --- p.5-5Chapter 5.1.2.2 --- Fast Interpolation --- p.5-8Chapter 5.1.3 --- Skipping Inverse ICT and Inverse Quantization Depends on Coded Block Pattern --- p.5-10Chapter 5. 2 --- Code Level Optimizations --- p.5-12Chapter 5.2.1 --- Merging Loops --- p.5-12Chapter 5.2.2 --- Moving Independent Code outside the Loop --- p.5-13Chapter 5.2.3 --- Unrolling Loops --- p.5-14Chapter 5.3 --- Experimental Results --- p.5-16Chapter 5.4 --- Applications --- p.5-26Chapter Chapter 6 --- Conclusions and Future Development --- p.6-1Chapter 6.1 --- Conclusions --- p.6-1Chapter 6.1.1 --- Enhancement Techniques for Intra Block Matching --- p.6-1Chapter 6.1.2 --- Enhanced SAD Reuse Fast Motion Estimation --- p.6-1Chapter 6.1.3 --- Development of Real-Time H.264 Codec on Pocket PC --- p.6-2Chapter 6.2 --- Future Development --- p.6-3Bibliography --- p.

3D high definition video coding on a GPU-based heterogeneous system

H.264/MVC is a standard for supporting the sensation of 3D, based on coding from 2 (stereo) to N views. H.264/MVC adopts many coding options inherited from single view H.264/AVC, and thus its complexity is even higher, mainly because the number of processing views is higher. In this manuscript, we aim at an efficient parallelization of the most computationally intensive video encoding module for stereo sequences. In particular, inter prediction and its collaborative execution on a heterogeneous platform. The proposal is based on an efficient dynamic load balancing algorithm and on breaking encoding dependencies. Experimental results demonstrate the proposed algorithm's ability to reduce the encoding time for different stereo high definition sequences. Speed-up values of up to 90× were obtained when compared with the reference encoder on the same platform. Moreover, the proposed algorithm also provides a more energy-efficient approach and hence requires less energy than the sequential reference algorith

Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding

H.264/AVC video compression standard achieved significant improvements in coding efficiency, but the computational complexity of the H.264/AVC encoder is drastically high. The main complexity of encoder comes from variable block size motion estimation (ME) and rate-distortion optimized (RDO) mode decision methods. This dissertation proposes three different methods to reduce computation of motion estimation. Firstly, the computation of each distortion measure is reduced by proposing a novel two step edge based partial distortion search (TS-EPDS) algorithm. In this algorithm, the entire macroblock is divided into different sub-blocks and the calculation order of partial distortion is determined based on the edge strength of the sub-blocks. Secondly, we have developed an early termination algorithm that features an adaptive threshold based on the statistical characteristics of rate-distortion (RD) cost regarding current block and previously processed blocks and modes. Thirdly, this dissertation presents a novel adaptive search area selection method by utilizing the information of the previously computed motion vector differences (MVDs). In H.264/AVC intra coding, DC mode is used to predict regions with no unified direction and the predicted pixel values are same and thus smooth varying regions are not well de-correlated. This dissertation proposes an improved DC prediction (IDCP) mode based on the distance between the predicted and reference pixels. On the other hand, using the nine prediction modes in intra 4x4 and 8x8 block units needs a lot of overhead bits. In order to reduce the number of overhead bits, an intra mode bit rate reduction method is suggested. This dissertation also proposes an enhanced algorithm to estimate the most probable mode (MPM) of each block. The MPM is derived from the prediction mode direction of neighboring blocks which have different weights according to their positions. This dissertation also suggests a fast enhanced cost function for mode decision of intra encoder. The enhanced cost function uses sum of absolute Hadamard-transformed differences (SATD) and mean absolute deviation of the residual block to estimate distortion part of the cost function. A threshold based large coefficients count is also used for estimating the bit-rate part

Complexity adaptation in H.264/AVC video coder for static cameras

H.264/AVC uses variable block size motion estimation (VBSME) to improve coding gain. However, its complexity is significant and fixed regardless of the required quality or of the scene characteristics. In this paper, we propose an adaptive complexity algorithm based on using the Walsh Hadamard Transform (WHT). VBS automatic partition and skip mode detection algorithms also are proposed. Experimental results show that 70% - 5% of the computation of H.264/AVC is required to achieve the same PSNR

Dense and accurate motion and strain estimation in high resolution speckle images using an image-adaptive approach

Digital image processing methods represent a viable and well acknowledged alternative to strain gauges and interferometric techniques for determining full-field displacements and strains in materials under stress. This paper presents an image adaptive technique for dense motion and strain estimation using high-resolution speckle images that show the analyzed material in its original and deformed states. The algorithm starts by dividing the speckle image showing the original state into irregular cells taking into consideration both spatial and gradient image information present. Subsequently the Newton-Raphson digital image correlation technique is applied to calculate the corresponding motion for each cell. Adaptive spatial regularization in the form of the Geman-McClure robust spatial estimator is employed to increase the spatial consistency of the motion components of a cell with respect to the components of neighbouring cells. To obtain the final strain information, local least-squares fitting using a linear displacement model is performed on the horizontal and vertical displacement fields. To evaluate the presented image partitioning and strain estimation techniques two numerical and two real experiments are employed. The numerical experiments simulate the deformation of a specimen with constant strain across the surface as well as small rigid-body rotations present while real experiments consist specimens that undergo uniaxial stress. The results indicate very good accuracy of the recovered strains as well as better rotation insensitivity compared to classical techniques

Dense and accurate motion and strain estimation in high resolution speckle images using an image-adaptive approach

Digital image processing methods represent a viable and well acknowledged alternative to strain gauges and interferometric techniques for determining full-field displacements and strains in materials under stress. This paper presents an image adaptive technique for dense motion and strain estimation using high-resolution speckle images that show the analyzed material in its original and deformed states. The algorithm starts by dividing the speckle image showing the original state into irregular cells taking into consideration both spatial and gradient image information present. Subsequently the Newton-Raphson digital image correlation technique is applied to calculate the corresponding motion for each cell. Adaptive spatial regularization in the form of the Geman-McClure robust spatial estimator is employed to increase the spatial consistency of the motion components of a cell with respect to the components of neighbouring cells. To obtain the final strain information, local least-squares fitting using a linear displacement model is performed on the horizontal and vertical displacement fields. To evaluate the presented image partitioning and strain estimation techniques two numerical and two real experiments are employed. The numerical experiments simulate the deformation of a specimen with constant strain across the surface as well as small rigid-body rotations present while real experiments consist specimens that undergo uniaxial stress. The results indicate very good accuracy of the recovered strains as well as better rotation insensitivity compared to classical techniques

Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing

Free-viewpoint video conferencing allows a participant to observe the remote 3D scene from any freely chosen viewpoint. An intermediate virtual viewpoint image is commonly synthesized using two pairs of transmitted texture and depth maps from two neighboring captured viewpoints via depth-image-based rendering (DIBR). To maintain high quality of synthesized images, it is imperative to contain the adverse effects of network packet losses that may arise during texture and depth video transmission. Towards this end, we develop an integrated approach that exploits the representation redundancy inherent in the multiple streamed videos a voxel in the 3D scene visible to two captured views is sampled and coded twice in the two views. In particular, at the receiver we first develop an error concealment strategy that adaptively blends corresponding pixels in the two captured views during DIBR, so that pixels from the more reliable transmitted view are weighted more heavily. We then couple it with a sender-side optimization of reference picture selection (RPS) during real-time video coding, so that blocks containing samples of voxels that are visible in both views are more error-resiliently coded in one view only, given adaptive blending will erase errors in the other view. Further, synthesized view distortion sensitivities to texture versus depth errors are analyzed, so that relative importance of texture and depth code blocks can be computed for system-wide RPS optimization. Experimental results show that the proposed scheme can outperform the use of a traditional feedback channel by up to 0.82 dB on average at 8% packet loss rate, and by as much as 3 dB for particular frames