JND-Based Perceptual Video Coding for 4:4:4 Screen Content Data in HEVC

The JCT-VC standardized Screen Content Coding (SCC) extension in the HEVC HM RExt + SCM reference codec offers an impressive coding efficiency performance when compared with HM RExt alone; however, it is not significantly perceptually optimized. For instance, it does not include advanced HVS-based perceptual coding methods, such as JND-based spatiotemporal masking schemes. In this paper, we propose a novel JND-based perceptual video coding technique for HM RExt + SCM. The proposed method is designed to further improve the compression performance of HM RExt + SCM when applied to YCbCr 4:4:4 SC video data. In the proposed technique, luminance masking and chrominance masking are exploited to perceptually adjust the Quantization Step Size (QStep) at the Coding Block (CB) level. Compared with HM RExt 16.10 + SCM 8.0, the proposed method considerably reduces bitrates (Kbps), with a maximum reduction of 48.3%. In addition to this, the subjective evaluations reveal that SC-PAQ achieves visually lossless coding at very low bitrates.Comment: Preprint: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2018

New approaches and a subjective database for video quality assessment

Video quality assessment plays an important role in multimedia systems that process digital images/videos such as video codec, video streaming server. The use of video quality assessment algorithm helps optimize system parameters, increase quality of service, and satisfy customers' demands. Traditional method that recruits human subjects to judge video quality often comes with the expense of time, money, and effort while objective method, which uses computer and built-in algorithms to judge video quality, offers a more affordable way. This dissertation report provides an efficient approach to develop objective video quality assessment algorithm.Algorithms in video quality assessment aim to predict quality of videos in a manner that agrees with subjective ratings of quality judged by human subjects. From that, two important factors are required for the research of video quality assessment. The first factor is an algorithm that is able to predict video quality. Our approach to develop such an algorithm bases on the analyses of spatial and spatiotemporal slices in two separate stages. The first stage estimates perceived quality degradation due to spatial distortion; this stage operates by adaptively applying our previous image quality assessment algorithm on a frame basis with an extension to account for temporal masking. The second stage estimates perceived quality degradation due to joint spatial and temporal distortion; this stage operates by measuring the dissimilarity between the two-dimensional spatiotemporal slices created by taking time-based slices of the original and distorted videos. The combination of these two estimates serves as an overall estimate of perceived quality degradation.The second important factor in the research of video quality assessment is a video-quality database with collected subjective ratings used to validate the algorithms' performance. We create our own video-quality database that consists of more videos ($216$ videos) and more distortion types (six) comparing to the currently available video-quality databases. The experiment to collect subjective ratings of quality is conducted by 40 different subjects following the SAMVIQ methodology.Acknowledge that in many applications, the original video is not available; we develop another video quality assessment algorithm that can predict quality of a processed video without information of the original video. This algorithm, specifically designed for videos compressed by Motion JPEG2000 compression standard, consists of two analyses of quality degradation in the edge/near-edge regions and the non-edge regions of the videos. The algorithm shows promise in the first step of developing a general no-reference algorithm for video quality assessment

Visually lossless coding in HEVC : a high bit depth and 4:4:4 capable JND-based perceptual quantisation technique for HEVC

Due to the increasing prevalence of high bit depth and YCbCr 4:4:4 video data, it is desirable to develop a JND-based visually lossless coding technique which can account for high bit depth 4:4:4 data in addition to standard 8-bit precision chroma subsampled data. In this paper, we propose a Coding Block (CB)-level JND-based luma and chroma perceptual quantisation technique for HEVC named Pixel-PAQ. Pixel-PAQ exploits both luminance masking and chrominance masking to achieve JND-based visually lossless coding; the proposed method is compatible with high bit depth YCbCr 4:4:4 video data of any resolution. When applied to YCbCr 4:4:4 high bit depth video data, Pixel-PAQ can achieve vast bitrate reductions – of up to 75% (68.6% over four QP data points) – compared with a state-of-the-art luma-based JND method for HEVC named IDSQ. Moreover, the participants in the subjective evaluations confirm that visually lossless coding is successfully achieved by Pixel-PAQ (at a PSNR value of 28.04 dB in one test)

Spatiotemporal Video Quality Assessment Method via Multiple Feature Mappings

Progressed video quality assessment (VQA) methods aim to evaluate the perceptual quality of videos in many applications but often prompt to increase computational complexity. Problems derive from the complexity of the distorted videos that are of significant concern in the communication industry, as well as the spatial-temporal content of the two-fold (spatial and temporal) distortion. Therefore, the findings of the study indicate that the information in the spatiotemporal slice (STS) images are useful in measuring video distortion. This paper mainly focuses on developing on a full reference video quality assessment algorithm estimator that integrates several features of spatiotemporal slices (STSS) of frames to form a high-performance video quality. This research work aims to evaluate video quality by utilizing several VQA databases by the following steps: (1) we first arrange the reference and test video sequences into a spatiotemporal slice representation. A collection of spatiotemporal feature maps were computed on each reference-test video. These response features are then processed by using a Structural Similarity (SSIM) to form a local frame quality.  (2) To further enhance the quality assessment, we combine the spatial feature maps with the spatiotemporal feature maps and propose the VQA model, named multiple map similarity feature deviation (MMSFD-STS). (3) We apply a sequential pooling strategy to assemble the quality indices of frames in the video quality scoring. (4) Extensive evaluations on video quality databases show that the proposed VQA algorithm achieves better/competitive performance as compared with other state- of- the- art methods

Hybrid video quality prediction: reviewing video quality measurement for widening application scope

A tremendous number of objective video quality measurement algorithms have been developed during the last two decades. Most of them either measure a very limited aspect of the perceived video quality or they measure broad ranges of quality with limited prediction accuracy. This paper lists several perceptual artifacts that may be computationally measured in an isolated algorithm and some of the modeling approaches that have been proposed to predict the resulting quality from those algorithms. These algorithms usually have a very limited application scope but have been verified carefully. The paper continues with a review of some standardized and well-known video quality measurement algorithms that are meant for a wide range of applications, thus have a larger scope. Their individual artifacts prediction accuracy is usually lower but some of them were validated to perform sufficiently well for standardization. Several difficulties and shortcomings in developing a general purpose model with high prediction performance are identified such as a common objective quality scale or the behavior of individual indicators when confronted with stimuli that are out of their prediction scope. The paper concludes with a systematic framework approach to tackle the development of a hybrid video quality measurement in a joint research collaboration.Polish National Centre for Research and Development (NCRD) SP/I/1/77065/10, Swedish Governmental Agency for Innovation Systems (Vinnova

SpatioTemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment

Perceptual video quality assessment models are either frame-based or video-based, i.e., they apply spatiotemporal filtering or motion estimation to capture temporal video distortions. Despite their good performance on video quality databases, video-based approaches are time-consuming and harder to efficiently deploy. To balance between high performance and computational efficiency, Netflix developed the Video Multi-method Assessment Fusion (VMAF) framework, which integrates multiple quality-aware features to predict video quality. Nevertheless, this fusion framework does not fully exploit temporal video quality measurements which are relevant to temporal video distortions. To this end, we propose two improvements to the VMAF framework: SpatioTemporal VMAF and Ensemble VMAF. Both algorithms exploit efficient temporal video features which are fed into a single or multiple regression models. To train our models, we designed a large subjective database and evaluated the proposed models against state-of-the-art approaches. The compared algorithms will be made available as part of the open source package in https://github.com/Netflix/vmaf