24 research outputs found
JND-Based Perceptual Video Coding for 4:4:4 Screen Content Data in HEVC
The JCT-VC standardized Screen Content Coding (SCC) extension in the HEVC HM
RExt + SCM reference codec offers an impressive coding efficiency performance
when compared with HM RExt alone; however, it is not significantly perceptually
optimized. For instance, it does not include advanced HVS-based perceptual
coding methods, such as JND-based spatiotemporal masking schemes. In this
paper, we propose a novel JND-based perceptual video coding technique for HM
RExt + SCM. The proposed method is designed to further improve the compression
performance of HM RExt + SCM when applied to YCbCr 4:4:4 SC video data. In the
proposed technique, luminance masking and chrominance masking are exploited to
perceptually adjust the Quantization Step Size (QStep) at the Coding Block (CB)
level. Compared with HM RExt 16.10 + SCM 8.0, the proposed method considerably
reduces bitrates (Kbps), with a maximum reduction of 48.3%. In addition to
this, the subjective evaluations reveal that SC-PAQ achieves visually lossless
coding at very low bitrates.Comment: Preprint: 2018 IEEE International Conference on Acoustics, Speech and
Signal Processing (ICASSP 2018
Visually lossless coding in HEVC : a high bit depth and 4:4:4 capable JND-based perceptual quantisation technique for HEVC
Due to the increasing prevalence of high bit depth and YCbCr 4:4:4 video data, it is desirable to develop a JND-based visually lossless coding technique which can account for high bit depth 4:4:4 data in addition to standard 8-bit precision chroma subsampled data. In this paper, we propose a Coding Block (CB)-level JND-based luma and chroma perceptual quantisation technique for HEVC named Pixel-PAQ. Pixel-PAQ exploits both luminance masking and chrominance masking to achieve JND-based visually lossless coding; the proposed method is compatible with high bit depth YCbCr 4:4:4 video data of any resolution. When applied to YCbCr 4:4:4 high bit depth video data, Pixel-PAQ can achieve vast bitrate reductions – of up to 75% (68.6% over four QP data points) – compared with a state-of-the-art luma-based JND method for HEVC named IDSQ. Moreover, the participants in the subjective evaluations confirm that visually lossless coding is successfully achieved by Pixel-PAQ (at a PSNR value of 28.04 dB in one test)
Subjective image quality assessment with boosted triplet comparisons.
In subjective full-reference image quality assessment, a reference image is distorted at increasing distortion levels. The differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcomes these problems. In addition, PC is more sensitive than DCR, and it can provide scale values in fractional, just noticeable difference (JND) units that express a precise perceptional interpretation. Still, the comparison of images of nearly the same quality can be difficult. We introduce boosting techniques embedded in more general triplet comparisons (TC) that increase the sensitivity even more. Boosting amplifies the artefacts of distorted images, enlarges their visual representation by zooming, increases the visibility of the distortions by a flickering effect, or combines some of the above. Experimental results show the effectiveness of boosted TC for seven types of distortion (color diffusion, jitter, high sharpen, JPEG 2000 compression, lens blur, motion blur, multiplicative noise). For our study, we crowdsourced over 1.7 million responses to triplet questions. We give a detailed analysis of the data in terms of scale reconstructions, accuracy, detection rates, and sensitivity gain. Generally, boosting increases the discriminatory power and allows to reduce the number of subjective ratings without sacrificing the accuracy of the resulting relative image quality values. Our technique paves the way to fine-grained image quality datasets, allowing for more distortion levels, yet with high-quality subjective annotations. We also provide the details for Thurstonian scale reconstruction from TC and our annotated dataset, KonFiG-IQA , containing 10 source images, processed using 7 distortion types at 12 or even 30 levels, uniformly spaced over a span of 3 JND units
Localization of Just Noticeable Difference for Image Compression
The just noticeable difference (JND) is the minimal difference between
stimuli that can be detected by a person. The picture-wise just noticeable
difference (PJND) for a given reference image and a compression algorithm
represents the minimal level of compression that causes noticeable differences
in the reconstruction. These differences can only be observed in some specific
regions within the image, dubbed as JND-critical regions. Identifying these
regions can improve the development of image compression algorithms. Due to the
fact that visual perception varies among individuals, determining the PJND
values and JND-critical regions for a target population of consumers requires
subjective assessment experiments involving a sufficiently large number of
observers. In this paper, we propose a novel framework for conducting such
experiments using crowdsourcing. By applying this framework, we created a novel
PJND dataset, KonJND++, consisting of 300 source images, compressed versions
thereof under JPEG or BPG compression, and an average of 43 ratings of PJND and
129 self-reported locations of JND-critical regions for each source image. Our
experiments demonstrate the effectiveness and reliability of our proposed
framework, which is easy to be adapted for collecting a large-scale dataset.
The source code and dataset are available at
https://github.com/angchen-dev/LocJND
Localization of just noticeable difference for image compression
The just noticeable difference (JND) is the minimal difference between stimuli that can be detected by a person. The picture-wise just noticeable difference (PJND) for a given reference image and a compression algorithm represents the minimal level of compression that causes noticeable differences in the reconstruction. These differences can only be observed in some specific regions within the image, dubbed as JND-critical regions. Identifying these regions can improve the development of image compression algorithms. Due to the fact that visual perception varies among individuals, determining the PJND values and JND-critical regions for a target population of consumers requires subjective assessment experiments involving a sufficiently large number of observers. In this paper, we propose a novel framework for conducting such experiments using crowdsourcing. By applying this framework, we created a novel PJND dataset, KonJND++, consisting of 300 source images, compressed versions thereof under JPEG or BPG compression, and an average of 43 ratings of PJND and 129 self-reported locations of JND-critical regions for each source image. Our experiments demonstrate the effectiveness and reliability of our proposed framework, which is easy to be adapted for collecting a large-scale dataset. The source code and dataset are available at https://github.com/angchen-dev/LocJND.</p
Training and Predicting Visual Error for Real-Time Applications
Visual error metrics play a fundamental role in the quantification of
perceived image similarity. Most recently, use cases for them in real-time
applications have emerged, such as content-adaptive shading and shading reuse
to increase performance and improve efficiency. A wide range of different
metrics has been established, with the most sophisticated being capable of
capturing the perceptual characteristics of the human visual system. However,
their complexity, computational expense, and reliance on reference images to
compare against prevent their generalized use in real-time, restricting such
applications to using only the simplest available metrics. In this work, we
explore the abilities of convolutional neural networks to predict a variety of
visual metrics without requiring either reference or rendered images.
Specifically, we train and deploy a neural network to estimate the visual error
resulting from reusing shading or using reduced shading rates. The resulting
models account for 70%-90% of the variance while achieving up to an order of
magnitude faster computation times. Our solution combines image-space
information that is readily available in most state-of-the-art deferred shading
pipelines with reprojection from previous frames to enable an adequate estimate
of visual errors, even in previously unseen regions. We describe a suitable
convolutional network architecture and considerations for data preparation for
training. We demonstrate the capability of our network to predict complex error
metrics at interactive rates in a real-time application that implements
content-adaptive shading in a deferred pipeline. Depending on the portion of
unseen image regions, our approach can achieve up to performance
compared to state-of-the-art methods.Comment: Published at Proceedings of the ACM in Computer Graphics and
Interactive Techniques. 14 Pages, 16 Figures, 3 Tables. For paper website and
higher quality figures, see https://jaliborc.github.io/rt-percept