Depth sequence coding with hierarchical partitioning and spatial-domain quantization

Depth coding in 3D-HEVC deforms object shapes due to block-level edge-approximation and lacks efficient techniques to exploit the statistical redundancy, due to the frame-level clustering tendency in depth data, for higher coding gain at near-lossless quality. This paper presents a standalone mono-view depth sequence coder, which preserves edges implicitly by limiting quantization to the spatial-domain and exploits the frame-level clustering tendency efficiently with a novel binary tree-based decomposition (BTBD) technique. The BTBD can exploit the statistical redundancy in frame-level syntax, motion components, and residuals efficiently with fewer block-level prediction/coding modes and simpler context modeling for context-adaptive arithmetic coding. Compared with the depth coder in 3D-HEVC, the proposed one has achieved significantly lower bitrate at lossless to near-lossless quality range for mono-view coding and rendered superior quality synthetic views from the depth maps, compressed at the same bitrate, and the corresponding texture frames. © 1991-2012 IEEE

Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing

Free-viewpoint video conferencing allows a participant to observe the remote 3D scene from any freely chosen viewpoint. An intermediate virtual viewpoint image is commonly synthesized using two pairs of transmitted texture and depth maps from two neighboring captured viewpoints via depth-image-based rendering (DIBR). To maintain high quality of synthesized images, it is imperative to contain the adverse effects of network packet losses that may arise during texture and depth video transmission. Towards this end, we develop an integrated approach that exploits the representation redundancy inherent in the multiple streamed videos a voxel in the 3D scene visible to two captured views is sampled and coded twice in the two views. In particular, at the receiver we first develop an error concealment strategy that adaptively blends corresponding pixels in the two captured views during DIBR, so that pixels from the more reliable transmitted view are weighted more heavily. We then couple it with a sender-side optimization of reference picture selection (RPS) during real-time video coding, so that blocks containing samples of voxels that are visible in both views are more error-resiliently coded in one view only, given adaptive blending will erase errors in the other view. Further, synthesized view distortion sensitivities to texture versus depth errors are analyzed, so that relative importance of texture and depth code blocks can be computed for system-wide RPS optimization. Experimental results show that the proposed scheme can outperform the use of a traditional feedback channel by up to 0.82 dB on average at 8% packet loss rate, and by as much as 3 dB for particular frames

Improved depth coding for HEVC focusing on depth edge approximation

The latest High Efficiency Video Coding (HEVC) standard has greatly improved the coding efficiency compared to its predecessor H.264. An important share of which is the adoption of hierarchical block partitioning structures and an extended number of modes. The structure of existing inter-modes is appropriate mainly to handle the rectangular and square aligned motion patterns. However, they could not be suitable for the block partitioning of depth objects having partial foreground motion with irregular edges and background. In such cases, the HEVC reference test model (HM) normally explores finer level block partitioning that requires more bits and encoding time to compensate large residuals. Since motion detection is the underlying criteria for mode selection, in this work, we use the energy concentration ratio feature of phase correlation to capture different types of motion in depth object. For better motion modeling focusing at depth edges, the proposed technique also uses an extra pattern mode comprising a group of templates with various rectangular and non-rectangular object shapes and edges. As the pattern mode could save bits by encoding only the foreground areas and beat all other inter-modes in a block once selected, the proposed technique could improve the rate-distortion performance. It could also reduce encoding time by skipping further branching using the pattern mode and selecting a subset of modes using innovative pre-processing criteria. Experimentally it could save 29% average encoding time and improve 0.10 dB Bjontegaard Delta peak signal-to-noise ratio compared to the HM

Video coding for compression and content-based functionality

The lifetime of this research project has seen two dramatic developments in the area of digital video coding. The first has been the progress of compression research leading to a factor of two improvement over existing standards, much wider deployment possibilities and the development of the new international ITU-T Recommendation H.263. The second has been a radical change in the approach to video content production with the introduction of the content-based coding concept and the addition of scene composition information to the encoded bit-stream. Content-based coding is central to the latest international standards efforts from the ISO/IEC MPEG working group. This thesis reports on extensions to existing compression techniques exploiting a priori knowledge about scene content. Existing, standardised, block-based compression coding techniques were extended with work on arithmetic entropy coding and intra-block prediction. These both form part of the H.263 and MPEG-4 specifications respectively. Object-based coding techniques were developed within a collaborative simulation model, known as SIMOC, then extended with ideas on grid motion vector modelling and vector accuracy confidence estimation. An improved confidence measure for encouraging motion smoothness is proposed. Object-based coding ideas, with those from other model and layer-based coding approaches, influenced the development of content-based coding within MPEG-4. This standard made considerable progress in this newly adopted content based video coding field defining normative techniques for arbitrary shape and texture coding. The means to generate this information, the analysis problem, for the content to be coded was intentionally not specified. Further research work in this area concentrated on video segmentation and analysis techniques to exploit the benefits of content based coding for generic frame based video. The work reported here introduces the use of a clustering algorithm on raw data features for providing initial segmentation of video data and subsequent tracking of those image regions through video sequences. Collaborative video analysis frameworks from COST 21 l qual and MPEG-4, combining results from many other segmentation schemes, are also introduced

Non-expansive symmetrically extended wavelet transform for arbitrarily shaped video object plane.

by Lai Chun Kit.Thesis (M.Phil.)--Chinese University of Hong Kong, 1998.Includes bibliographical references (leaves 68-70).Abstract also in Chinese.ACKNOWLEDGMENTS --- p.IVABSTRACT --- p.vChapter Chapter 1 --- Traditional Image and Video Coding --- p.1Chapter 1.1 --- Introduction --- p.1Chapter 1.2 --- Fundamental Principle of Compression --- p.1Chapter 1.3 --- Entropy - Value of Information --- p.2Chapter 1.4 --- Performance Measure --- p.3Chapter 1.5 --- Image Coding Overview --- p.4Chapter 1.5.1 --- Digital Image Formation --- p.4Chapter 1.5.2 --- Needs of Image Compression --- p.4Chapter 1.5.3 --- Classification of Image Compression --- p.5Chapter 1.5.4 --- Transform Coding --- p.6Chapter 1.6 --- Video Coding Overview --- p.8Chapter Chapter 2 --- Discrete Wavelets Transform (DWT) and Subband Coding --- p.11Chapter 2.1 --- Subband Coding --- p.11Chapter 2.1.1 --- Introduction --- p.11Chapter 2.1.2 --- Quadrature Mirror Filters (QMFs) --- p.12Chapter 2.1.3 --- Subband Coding for Image --- p.13Chapter 2.2 --- Discrete Wavelets Transformation (DWT) --- p.15Chapter 2.2.1 --- Introduction --- p.15Chapter 2.2.2 --- Wavelet Theory --- p.15Chapter 2.2.3 --- Comparison Between Fourier Transform and Wavelet Transform --- p.16Chapter Chapter 3 --- Non-expansive Symmetric Extension --- p.19Chapter 3.1 --- Introduction --- p.19Chapter 3.2 --- Types of extension scheme --- p.19Chapter 3.3 --- Non-expansive Symmetric Extension and Symmetric Sub-sampling --- p.21Chapter Chapter 4 --- Content-based Video Coding in MPEG-4 Purposed Standard --- p.24Chapter 4.1 --- Introduction --- p.24Chapter 4.2 --- Motivation of the new MPEG-4 standard --- p.25Chapter 4.2.1 --- Changes in the production of audio-visual material --- p.25Chapter 4.2.2 --- Changes in the consumption of multimedia information --- p.25Chapter 4.2.3 --- Reuse of audio-visual material --- p.26Chapter 4.2.4 --- Changes in mode of implementation --- p.26Chapter 4.3 --- Objective of MPEG-4 standard --- p.27Chapter 4.4 --- Technical Description of MPEG-4 --- p.28Chapter 4.4.1 --- Overview of MPEG-4 coding system --- p.28Chapter 4.4.2 --- Shape Coding --- p.29Chapter 4.4.3 --- Shape Adaptive Texture Coding --- p.33Chapter 4.4.4 --- Motion Estimation and Compensation (ME/MC) --- p.35Chapter Chapter 5 --- Shape Adaptive Wavelet Transformation Coding Scheme (SA WT) --- p.36Chapter 5.1 --- Shape Adaptive Wavelet Transformation --- p.36Chapter 5.1.1 --- Introduction --- p.36Chapter 5.1.2 --- Description of Transformation Scheme --- p.37Chapter 5.2 --- Quantization --- p.40Chapter 5.3 --- Entropy Coding --- p.42Chapter 5.3.1 --- Introduction --- p.42Chapter 5.3.2 --- Stack Run Algorithm --- p.42Chapter 5.3.3 --- ZeroTree Entropy (ZTE) Coding Algorithm --- p.45Chapter 5.4 --- Binary Shape Coding --- p.49Chapter Chapter 6 --- Simulation --- p.51Chapter 6.1 --- Introduction --- p.51Chapter 6.2 --- SSAWT-Stack Run --- p.52Chapter 6.3 --- SSAWT-ZTR --- p.53Chapter 6.4 --- Simulation Results --- p.55Chapter 6.4.1 --- SSAWT - STACK --- p.55Chapter 6.4.2 --- SSAWT ´ؤ ZTE --- p.56Chapter 6.4.3 --- Comparison Result - Cjpeg and Wave03. --- p.57Chapter 6.5 --- Shape Coding Result --- p.61Chapter 6.6 --- Analysis --- p.63Chapter Chapter 7 --- Conclusion --- p.64Appendix A: Image Segmentation --- p.65Reference --- p.6

Object-based video representations: shape compression and object segmentation

Object-based video representations are considered to be useful for easing the process of multimedia content production and enhancing user interactivity in multimedia productions. Object-based video presents several new technical challenges, however. Firstly, as with conventional video representations, compression of the video data is a requirement. For object-based representations, it is necessary to compress the shape of each video object as it moves in time. This amounts to the compression of moving binary images. This is achieved by the use of a technique called context-based arithmetic encoding. The technique is utilised by applying it to rectangular pixel blocks and as such it is consistent with the standard tools of video compression. The blockbased application also facilitates well the exploitation of temporal redundancy in the sequence of binary shapes. For the first time, context-based arithmetic encoding is used in conjunction with motion compensation to provide inter-frame compression. The method, described in this thesis, has been thoroughly tested throughout the MPEG-4 core experiment process and due to favourable results, it has been adopted as part of the MPEG-4 video standard. The second challenge lies in the acquisition of the video objects. Under normal conditions, a video sequence is captured as a sequence of frames and there is no inherent information about what objects are in the sequence, not to mention information relating to the shape of each object. Some means for segmenting semantic objects from general video sequences is required. For this purpose, several image analysis tools may be of help and in particular, it is believed that video object tracking algorithms will be important. A new tracking algorithm is developed based on piecewise polynomial motion representations and statistical estimation tools, e.g. the expectationmaximisation method and the minimum description length principle