Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing

Free-viewpoint video conferencing allows a participant to observe the remote 3D scene from any freely chosen viewpoint. An intermediate virtual viewpoint image is commonly synthesized using two pairs of transmitted texture and depth maps from two neighboring captured viewpoints via depth-image-based rendering (DIBR). To maintain high quality of synthesized images, it is imperative to contain the adverse effects of network packet losses that may arise during texture and depth video transmission. Towards this end, we develop an integrated approach that exploits the representation redundancy inherent in the multiple streamed videos a voxel in the 3D scene visible to two captured views is sampled and coded twice in the two views. In particular, at the receiver we first develop an error concealment strategy that adaptively blends corresponding pixels in the two captured views during DIBR, so that pixels from the more reliable transmitted view are weighted more heavily. We then couple it with a sender-side optimization of reference picture selection (RPS) during real-time video coding, so that blocks containing samples of voxels that are visible in both views are more error-resiliently coded in one view only, given adaptive blending will erase errors in the other view. Further, synthesized view distortion sensitivities to texture versus depth errors are analyzed, so that relative importance of texture and depth code blocks can be computed for system-wide RPS optimization. Experimental results show that the proposed scheme can outperform the use of a traditional feedback channel by up to 0.82 dB on average at 8% packet loss rate, and by as much as 3 dB for particular frames

Impact of packet losses in scalable light field video coding

Light field imaging technology has been recently attracting the attention of the research community and the industry. However, to effectively transmit light field content to the end-user over error-prone networks—e.g., wireless networks or the Internet—error resilience techniques are required to mitigate the impact of data impairments in the user quality perception. In this context, this chapter analyzes the impact of packet losses when using a three-layer display scalable light field video coding architecture, which has been presented in Chap. 6. For this, a simple error concealment algorithm is used, which makes use of inter-layer redundancy between multiview and light field content and the inherent correlation of the light field content to estimate lost data. Furthermore, a study of the influence of 2D views generation parameters used in lower layers on the performance of the used error concealment algorithm is also presented.info:eu-repo/semantics/acceptedVersio

Robust Adaptive Intra Refresh for Multiview Video

Transmission error propagation in wireless multimedia communication systems has become a recurring problem. This persistent problem has led to grave consequences on the visual quality of the decoded video. It is against this backdrop that, we present an adaptive intra refresh (AIR) error-resilient coding tool to mitigate the effect of transmission error propagation in 3D video communications. This work utilizes periodic insertion of intra macroblocks in badly error-infected frames temporally as well as related frames in the multi view video scheme. Our objective is to maximize the transmission efficiency while ensuring the transmission robustness of the coded bitstream. The selection of periodic macroblocks is based on areas with high motion above a pre-set threshold. The coding modes of the macroblocks are based on the distortion expectation due to transmission errors. Extensive simulation results show significant improvement in both objective and subjective video quality at different intra refresh rates

Impact of packet losses in scalable 3D holoscopic video coding

Holoscopic imaging became a prospective glassless 3D technology to provide more natural 3D viewing experiences to the end user. Additionally, holoscopic systems also allow new post-production degrees of freedom, such as controlling the plane of focus or the viewing angle presented to the user. However, to successfully introduce this technology into the consumer market, a display scalable coding approach is essential to achieve backward compatibility with legacy 2D and 3D displays. Moreover, to effectively transmit 3D holoscopic content over error-prone networks, e.g., wireless networks or the Internet, error resilience techniques are required to mitigate the impact of data impairments in the user quality perception. Therefore, it is essential to deeply understand the impact of packet losses in terms of decoding video quality for the specific case of 3D holoscopic content, notably when a scalable approach is used. In this context, this paper studies the impact of packet losses when using a three-layer display scalable 3D holoscopic video coding architecture previously proposed, where each layer represents a different level of display scalability (i.e., L0 - 2D, L1 - stereo or multiview, and L2 - full 3D holoscopic). For this, a simple error concealment algorithm is used, which makes use of inter-layer redundancy between multiview and 3D holoscopic content and the inherent correlation of the 3D holoscopic content to estimate lost data. Furthermore, a study of the influence of 2D views generation parameters used in lower layers on the performance of the used error concealment algorithm is also presented.info:eu-repo/semantics/acceptedVersio

Error concealment techniques for H.264/MVC encoded sequences

This work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union–European Social Fund (ESF 1.25).The H.264/MVC standard offers good compression ratios for multi-view sequences by exploiting spatial, temporal and interview image dependencies. This works well in error-free channels, however in the event of transmission errors, it leads to the propagation of the distorted macro-blocks, degrading the quality of experience of the user. This paper reviews the state-of-the-art error concealment solutions and proposes a low complexity concealment method that can be used with multi-view video coding. The error resilience techniques used to aid error concealment are also identified. Results obtained demonstrate that good multi-view video reconstruction can be obtained with this approach.peer-reviewe

Video Traffic Characteristics of Modern Encoding Standards: H.264/AVC with SVC and MVC Extensions and H.265/HEVC

abstract: Video encoding for multimedia services over communication networks has significantly advanced in recent years with the development of the highly efficient and flexible H.264/AVC video coding standard and its SVC extension. The emerging H.265/HEVC video coding standard as well as 3D video coding further advance video coding for multimedia communications. This paper first gives an overview of these new video coding standards and then examines their implications for multimedia communications by studying the traffic characteristics of long videos encoded with the new coding standards. We review video coding advances from MPEG-2 and MPEG-4 Part 2 to H.264/AVC and its SVC and MVC extensions as well as H.265/HEVC. For single-layer (nonscalable) video, we compare H.265/HEVC and H.264/AVC in terms of video traffic and statistical multiplexing characteristics. Our study is the first to examine the H.265/HEVC traffic variability for long videos. We also illustrate the video traffic characteristics and statistical multiplexing of scalable video encoded with the SVC extension of H.264/AVC as well as 3D video encoded with the MVC extension of H.264/AVC.View the article as published at https://www.hindawi.com/journals/tswj/2014/189481

Performance of enhanced error concealment techniques in multi-view video coding systems

This research work is partially funded by the Strategic Educational Pathways Scholarship Scheme (STEPS-Malta). This scholarship is partly financed by the European Union - European Social Fund (ESF 1.25).Transmission of multi-view video encoded bit-streams over error-prone channels demands robust error concealment techniques. This paper studies the performance of solutions that exploit the neighbourhood spatial, temporal and inter-view information for this scope. Furthermore, different boundary distortion measurements, motion compensation refinement and temporal error concealment of Anchor frames were exploited to improve the results obtained by the basic error concealment techniques. Results show that a gain in performance is obtained with the implementation of each independent concealment technique. Furthermore, Peak Signal-to-Noise Ratio (PSNR) gains of about 4dB relative to the standard were achieved when adopting a hybrid error concealment approach.peer-reviewe

Reference picture selection using checkerboard pattern for resilient video coding

The improved compression efficiency achieved by the High Efficiency Video Coding (HEVC) standard has the counter-effect of decreasing error resilience in transmission over error-prone channels. To increase the error resilience of HEVC streams, this paper proposes a checkerboard reference picture selection method in order to reduce the prediction mismatch at the decoder in case of frame losses. The proposed approach not only allows to reduce the error propagation at the decoder, but also enhances the quality of reconstructed frames by selectively constraining the choice of reference pictures used for temporal prediction. The underlying approach is to increase the amount of accurate temporal information at the decoder when transmission errors occur, to improve the video quality by using an efficient combination of diverse motion fields. The proposed method compensates for the small loss of coding efficiency at frame loss rates as low as 3%. For a single frame-loss event the proposed method can achieve up to 2 dB of gain in the affected frames and an average quality gain of 0:84 dB for different error prone conditions