REAL-TIME AND PARALLEL SHVC HYBRID CODEC AVC TO HEVC DECODER

International audienc

Decoding Time Prediction for Versatile Video Coding

International audienceEnergy consumption in video decoding is a complex interplay of codec efficiency, hardware design, software optimization, and other factors such as the processor clock frequency for software implementation. The Dynamic voltage and frequency scaling (DVFS) allows adjusting dynamically the processor clock frequency according to estimated frame requirements, enabling efficient resource management for decoding tasks. However, decoding time can vary considerably from one frame to another due to variations in frame complexity. Building upon this concept, this paper proposes a machine-learning model that estimates the decoding time of individual frames. The proposed model is built on the ExtraTrees regressor that accurately predicts the decoding time of 1080p video frames with a low relative error of 5.58% and a high R2 score of 94%. Our proposal entails the utilization of frame-related data that is readily reachable by the decoder, making it highly suitable for real-time scenarios

Real-Time Adaptive Multiple Transforms for the Next Generation Software Video Decoders

International audienceThe next generation of video codecs incorporate multiple transforms as a part of their transform core. However, some of the new transform types introduced in state-of-the-art codecs are still lacking efficient optimization methods. This paper focuses on the optimization of Adaptive Multiple Transforms (AMTs) in the next generation of video decoders. Our solution consists in three main contributions aiming towards a real time software implementation of an AMTs inverse core transform. We first investigate AMTs usage and properties in a transform and quantization video coding context. In a second contribution we design a unified transform core decoding algorithm for AMTs relying on pruning strategies. Finally, in a third contribution we present a Single Input Multiple Data (SIMD) implementation using AVX2 extension set targeting x86 architecture. To the best of our knowledge this is the first software implementation of AMTs achieving real-time performance in a software video decoder

Backward Compatible Layered Video Coding for 360° Video Broadcast

International audienc

OpenVVC: a Lightweight Software Decoder for the Versatile Video Coding Standard

In the recent years, users requirements for higher resolution, coupled with the apparition of new multimedia applications, have created the need for a new video coding standard. The new generation video coding standard, called Versatile Video Coding (VVC), has been developed by the Joint Video Experts Team, and offers coding capability beyond the previous generation High Efficiency Video Coding (HEVC) standard. Due to the incorporation of more advanced and complex tools, the decoding complexity of VVC standard compared to HEVC has approximately doubled. This complexity increase raises new research challenges to achieve live software decoding. In this context, we developed OpenVVC, an open-source software decoder that supports a broad range of VVC functionalities. This paper presents the OpenVVC software architecture, its parallelism strategy as well as a detailed set of experimental results. By combining extensive data level parallelism with frame level parallelism, OpenVVC achieves real-time decoding of UHD video content. Moreover, the memory required by OpenVVC is remarkably low, which presents a great advantage for its integration on embedded platforms with low memory resources. The code of the OpenVVC decoder is publicly available at https://github.com/OpenVVC/OpenVV

Energy Efficient VVC Decoding on Mobile Platform

International audienceRecently, global demand for high-resolution videos and new multimedia applications have created the need for a new video coding standard. Hence, in July 2020 the Versatile Video Coding (VVC) standard was released providing up to 40% bit-rate saving for the same video quality compared to its predecessor High Efficiency Video Coding (HEVC). However, this bit-rate saving comes at the cost of high computational complexity, particularly for live applications and on resource-constraint embedded devices. This paper presents an power-efficient VVC decoder implementation designed for low-resource platforms. This latter exploits optimization techniques such as data level parallelism using Single Instruction Multiple Data (SIMD) instructions and functional level parallelism using frame, tile and slice-based parallelisms. The results showed that the OpenVVC decoder achieve real-time decoding of Full High Definition (FHD) resolution at 30 fps targeting a platform with 8 cores with a maximum frequency of 2.2 Ghz and High Definition (HD) real-time decoding at 30 fps for platforms using 4 cores with a maximum frequency of 1.8 Ghz. In terms of average consumed power, OpenVVC showed around 5.6 watts and 1.7 watts for the 8 and 4 cores platforms, respectively. In addition, it comes with the best trade-off between what is achievable in real-time and the power consumed in comparison to the state-of-the-art implementation

Performance and Computational Complexity of the Future Video Coding

International audienceThe drastic increasing of multimedia applications, such as IPTV, Virtual Reality (VR, 360°) and Light Field videos has led to a high computing complexity in video compression and content quality assessment. In the last five years, HEVC standard has been widely used in the industrial community due to its bit-rate gain compared to its predecessor H.264/AVC. Recently, a new coding tools have been developed under the Joint Exploration Model (JEM) software, with the main goal to provide high bit rate saving compared to the HEVC standard. In this paper we investigate the performance and the associated computational complexity of these emerging video coding tools, from both encoding and decoding sides. Two spatial resolutions (HD 4K) and several video contents have been used in this study. Results have shown that despite the bit-rate saving, a considerable computational complexity can be noticed. A bit-rate saving up to 37% and quality enhancements up to 30% can be achieved by JEM. However, these emerging tools are time consuming between x5 and x12 times compared to the HM reference software, depending on video sequence and encoding/decoding processes. © 2018 IEEE

4K Real Time Software Solution of Scalable HEVC for Broadcast Video Application

International audienceScalable High efficiency Video Coding (SHVC) is the scalable extension of the High Efficiency Video Coding (HEVC) standard. SHVC enables spatial, quality, bit-depth, color gamut and codec scalability. The architecture of the SHVC encoder is based on multiple instances of the HEVC encoder where each instance encodes one video layer. This architecture offers several advantages of being modular and close to the native HEVC coding block scheme. However, the close-loop SHVC architecture requires the complete decoding of the reference lower-layer frames to decode a higher quality layer, which considerably increases the complexity of both encoder and decoder processes. In this paper, we propose an end-to-end 4K real time SHVC solution, including both software encoder and decoder, for video broadcast applications. The SHVC codec relies on low level optimizations for specific Intel x86 platform and parallel processing to speed-up the encoding and decoding processes. The proposed encoder enables a real time processing of 4Kp30 video in 2x spatial scalability on the 4x10-cores Intel Xeon processor (E5-4627V3) running at 2.6 GHz. In addition, the SHVC decoder enables to decode, respectively, the lower quality layer in full HD (1920x1080p30) resolution, on ARM Neon mobile platform, and the enhancement layer in UHD (3840x2160p30), on a laptop fitted, with 4 cores Intel i7 processor running at 2.7 GHz. Finally, experimental results have shown that the proposed solution can reach a high rate-distortion performance close to the reference SHVC reference software Model (SHM) with a speed-up of 37 and 66 in Intra and Inter coding configurations

Scalable Video Coding for Backward-Compatible 360° Video Delivery Over Broadcast Networks

International audienceRecently, the coding and transmission of immersive 360 • video has been intensely studied. The technologies provided by standards developing organizations mainly address requirements coming from over-the-top services. The terrestrial broadcast remains in many countries the mainstream medium to deliver high quality contents to a wide audience. To enable seamless introduction of immersive 360 • video services over terrestrial broadcast, the deployed technologies shall fulfill requirements such as backward compatibility to legacy receivers and high bandwidth efficiency. While bandwidth efficiency is addressed by existing techniques, none of them enables legacy video services decoding. In this paper, a novel scalable coding scheme is proposed to enable immersive 360 • video services introduction over broadcast networks. The experiments show that the proposed scalable coding scheme provides substantial coding gains of 14.99% compared to simulcast coding and introduces a limited coding overhead of 5.15% compared to 360 • single-layer coding. A real-time decoding implementation is proposed, highlighting the relevance of the proposed design. Eventually, an end-to-end demonstrator illustrates how the proposed solution could be integrated in a real terrestrial broadcast environment

Demo: 4K Real-Time Video Streaming in Hybrid Codec Scalability SHVC configuration

International audienc