New visual coding exploration in MPEG: Super-MultiView and free navigation in free viewpoint TV

ISO/IEC MPEG and ITU-T VCEG have recently jointly issued a new multiview video compression standard, called 3D-HEVC, which reaches unpreceded compression performances for linear,dense camera arrangements. In view of supporting future highquality,auto-stereoscopic 3D displays and Free Navigation virtual/augmented reality applications with sparse, arbitrarily arranged camera setups, innovative depth estimation and virtual view synthesis techniques with global optimizations over all camera views should be developed. Preliminary studies in response to the MPEG-FTV (Free viewpoint TV) Call for Evidence suggest these targets are within reach, with at least 6% bitrate gains over 3DHEVC technology

Video compression by chroma prediction using semantic communications

Conventional video coding is evolving to meet unprecedented consumer device requirements, but the statistical signal processing based approach may find limitations in handling new media contents. Deep neural network and semantic communication based video compression systems show potential to be used as video encoders and decoders, but reaching the rate distortion performance of state-of-the-art conventional video coding systems remains to be achieved. A novel video compression system by predicting the chroma components of video using the semantically encoded luma component and reference intra-coded frames is proposed and tested against high efficiency video coding (HEVC) for bit rate comparison and rate-distortion performance evaluation. The proposed system demonstrated 18% to 30% saving of bit rate for high and medium motion videos without significant reductions of rate-distortion with the saving increasing with higher group of picture sizes, but low motion videos only demonstrated negligible savings

Algorithms for compression of high dynamic range images and video

The recent advances in sensor and display technologies have brought upon the High Dynamic Range (HDR) imaging capability. The modern multiple exposure HDR sensors can achieve the dynamic range of 100-120 dB and LED and OLED display devices have contrast ratios of 10^5:1 to 10^6:1. Despite the above advances in technology the image/video compression algorithms and associated hardware are yet based on Standard Dynamic Range (SDR) technology, i.e. they operate within an effective dynamic range of up to 70 dB for 8 bit gamma corrected images. Further the existing infrastructure for content distribution is also designed for SDR, which creates interoperability problems with true HDR capture and display equipment. The current solutions for the above problem include tone mapping the HDR content to fit SDR. However this approach leads to image quality associated problems, when strong dynamic range compression is applied. Even though some HDR-only solutions have been proposed in literature, they are not interoperable with current SDR infrastructure and are thus typically used in closed systems. Given the above observations a research gap was identified in the need for efficient algorithms for the compression of still images and video, which are capable of storing full dynamic range and colour gamut of HDR images and at the same time backward compatible with existing SDR infrastructure. To improve the usability of SDR content it is vital that any such algorithms should accommodate different tone mapping operators, including those that are spatially non-uniform. In the course of the research presented in this thesis a novel two layer CODEC architecture is introduced for both HDR image and video coding. Further a universal and computationally efficient approximation of the tone mapping operator is developed and presented. It is shown that the use of perceptually uniform colourspaces for internal representation of pixel data enables improved compression efficiency of the algorithms. Further proposed novel approaches to the compression of metadata for the tone mapping operator is shown to improve compression performance for low bitrate video content. Multiple compression algorithms are designed, implemented and compared and quality-complexity trade-offs are identified. Finally practical aspects of implementing the developed algorithms are explored by automating the design space exploration flow and integrating the high level systems design framework with domain specific tools for synthesis and simulation of multiprocessor systems. The directions for further work are also presented

A semantic communication and VVC based hybrid video coding system

Requirements of next-generation video applications are becoming a challenge for conven-tional video coding systems, although they have evolved over decades to accommodate the most demanding of current video applications. Semantic communications, built on the concept of transmitting just the semantics of a message and allowing the receiver to reconstruct the message based on a shared context, is a non-conventional approach being considered to overcome these challenges and improve performance of video coding systems. In this paper, a first such semantic communication-based video coding system in hybrid mode is proposed, which uses an autoencoder-based semantic encoder for inter coding, augmented by the intra coding capabilities of Versatile Video Coding (VVC) to encode key frames that form the context for the semantic communication and the residuals for improving the fidelity of the output frames. For a range of videos with differing levels of complexity, the proposed system consistently outperforms High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC) in terms of rate distortion metrics quantified by Bjontegaard Delta Rates. It also outperforms Versatile Video Coding with videos with low or high complexity, but slightly falls behind with videos with medium complexity, which can be improved by addressing the open research areas that stem from this work. The proposed system demonstrates the potential of semantic communication based video coding systems to consistently outperform state-of-the-art conventional video coding systems over a wide range video applications