Visibility metrics and their applications in visually lossless image compression

Visibility metrics are image metrics that predict the probability that a human observer can detect differences between a pair of images. These metrics can provide localized information in the form of visibility maps, in which each value represents a probability of detection. An important application of the visibility metric is visually lossless image compression that aims at compressing a given image to the lowest fraction of bit per pixel while keeping the compression artifacts invisible at the same time. In previous works, most visibility metrics were modeled based on largely simplified assumptions and mathematical models of human visual systems. This approach generally fits well into experimental data measured with simple stimuli, such as Gabor patches. However, it cannot predict complex non-linear effects, such as contrast masking in natural images, particularly well. To predict visibility of image differences accurately, we collected the largest visibility dataset under fixed viewing conditions for calibrating existing visibility metrics and proposed a deep neural network-based visibility metric. We demonstrated in our experiments that the deep neural network-based visibility metric significantly outperformed existing visibility metrics. However, the deep neural network-based visibility metric cannot predict visibility under varying viewing conditions, such as display brightness and viewing distances that have great impacts on the visibility of distortions. To extend the deep neural network-based visibility metric to varying viewing conditions, we collected the largest visibility dataset under varying display brightness and viewing distances. We proposed incorporating white-box modules, in other words, luminance masking and viewing distance adaptation, into the black-box deep neural network, and we found that the combination of white-box modules and black-box deep neural networks could generalize our proposed visibility metric to varying viewing conditions. To demonstrate the application of our proposed deep neural network-based visibility metric to visually lossless image compression, we collected the visually lossless image compression dataset under fixed viewing conditions and significantly improved the deep neural network-based visibility metric's accuracy of predicting visually lossless image compression threshold by pre-training the visibility metric with a synthetic dataset generated by the state-of-the-art white-box visibility metric---HDR-VDP \cite{Mantiuk2011}. In a large-scale study of 1000 images, we found that with our improved visibility metric, we can save around 60\% to 70\% bits for visually lossless image compression encoding as compared to the default visually lossless quality level of 90. Because predicting image visibility and predicting image quality are closely related research topics, we also proposed a trained perceptually uniform transform for high dynamic range images and videos quality assessments by training a perceptual encoding function on a set of subjective quality assessment datasets. We have shown that when combining the trained perceptual encoding function with standard dynamic range image quality metrics, such as peak-signal-noise-ratio (PSNR), better performance was achieved compared to the untrained version

JNCD-based perceptual compression of RGB 4:4:4 image data

In contemporary lossy image coding applications, a desired aim is to decrease, as much as possible, bits per pixel without inducing perceptually conspicuous distortions in RGB image data. In this paper, we propose a novel color-based perceptual compression technique, named RGB-PAQ. RGB-PAQ is based on CIELAB Just Noticeable Color Difference (JNCD) and Human Visual System (HVS) spectral sensitivity. We utilize CIELAB JNCD and HVS spectral sensitivity modeling to separately adjust quantization levels at the Coding Block (CB) level. In essence, our method is designed to capitalize on the inability of the HVS to perceptually differentiate photons in very similar wavelength bands. In terms of application, the proposed technique can be used with RGB (4:4:4) image data of various bit depths and spatial resolutions including, for example, true color and deep color images in HD and Ultra HD resolutions. In the evaluations, we compare RGB-PAQ with a set of anchor methods; namely, HEVC, JPEG, JPEG 2000 and Google WebP. Compared with HEVC HM RExt, RGB-PAQ achieves up to 77.8% bits reductions. The subjective evaluations confirm that the compression artifacts induced by RGB-PAQ proved to be either imperceptible (MOS = 5) or near-imperceptible (MOS = 4) in the vast majority of cases

A just noticeable distortion based perceptually lossless image compression codec

In this study thesis a JND (Just-Noticeable-Distortion)-Measurement will be implemented on top of JPEG-LS while only considering a grayscale bit depth of 8 Bit. This is sufficient to show a proof of concept of combining the JND approach with JPEG-LS. JPEG-LS is a widely used and relatively simple coding mechanism for lossless and near-lossless image compression. The JND measurement will be defined, implemented and integrated into JPEG-LS. Therefore a modified approach of [CL95] will be used. The quantization step size (QSS) will be adapted dynamically according to the JND value so that the compression ratio compared to a standard JPEG-LS implementation can be improved. By that, a near-lossless variable bit-rate (VBR) is introduced into the coding flow. The encoder and decoder are implemented in MATLAB and after defining the visual quality criteria the results are evaluated and analysed in matter of compression quality and performance. Two test picture sets are used and the perceptual quality of the codec will be evaluated by an Mean-Opinion-Score (MOS) test and Multi-Scale Structural Similarity (MS-SSIM) test

Dataset and metrics for predicting local visible differences

A large number of imaging and computer graphics applications require localized information on the visibility of image distortions. Existing image quality metrics are not suitable for this task as they provide a single quality value per image. Existing visibility metrics produce visual difference maps, and are specifically designed for detecting just noticeable distortions but their predictions are often inaccurate. In this work, we argue that the key reason for this problem is the lack of large image collections with a good coverage of possible distortions that occur in different applications. To address the problem, we collect an extensive dataset of reference and distorted image pairs together with user markings indicating whether distortions are visible or not. We propose a statistical model that is designed for the meaningful interpretation of such data, which is affected by visual search and imprecision of manual marking. We use our dataset for training existing metrics and we demonstrate that their performance significantly improves. We show that our dataset with the proposed statistical model can be used to train a new CNN-based metric, which outperforms the existing solutions. We demonstrate the utility of such a metric in visually lossless JPEG compression, super-resolution and watermarking.</jats:p

Subjective Assessment of Image Compression Artefacts on Stereoscopic Display

Image and video quality are important to depict any pictorial information vividly and correctly. With the advancement of technology, we can produce high-quality images and can display those in advanced high-resolution displays. But as high-quality images continue to increase in size, transmitting these exceeds the limited bandwidth of display links. To cope, we need to compress the images but desire that the user cannot perceive any difference between the compressed and uncompressed images. In my thesis, psychophysical experiments with a flicker paradigm were undertaken to do a subjective assessment of the visibility of compression artefacts of two sets of images with two codecs viewed on a stereoscopic display. For one set of images the result shows that artefacts can be silenced in some stereo images relative to 2D while testing with the other set of images was inconclusive. This thesis documented evidence for silencing of artefacts in 3D displays. Other differences between stereoscopic and 2D presentation can be predicted but were not observed here (perhaps due to floor effects). Further large-scale subjective assessment with challenging images may help to get a concrete conclusion

Visually lossless coding in HEVC : a high bit depth and 4:4:4 capable JND-based perceptual quantisation technique for HEVC

Due to the increasing prevalence of high bit depth and YCbCr 4:4:4 video data, it is desirable to develop a JND-based visually lossless coding technique which can account for high bit depth 4:4:4 data in addition to standard 8-bit precision chroma subsampled data. In this paper, we propose a Coding Block (CB)-level JND-based luma and chroma perceptual quantisation technique for HEVC named Pixel-PAQ. Pixel-PAQ exploits both luminance masking and chrominance masking to achieve JND-based visually lossless coding; the proposed method is compatible with high bit depth YCbCr 4:4:4 video data of any resolution. When applied to YCbCr 4:4:4 high bit depth video data, Pixel-PAQ can achieve vast bitrate reductions – of up to 75% (68.6% over four QP data points) – compared with a state-of-the-art luma-based JND method for HEVC named IDSQ. Moreover, the participants in the subjective evaluations confirm that visually lossless coding is successfully achieved by Pixel-PAQ (at a PSNR value of 28.04 dB in one test)

LOCMIC:LOw Complexity Multi-resolution Image Compression

Image compression is a well-established and extensively researched field. The huge interest in it has been aroused by the rapid enhancements introduced in imaging techniques and the various applications that use high-resolution images (e.g. medical, astronomical, Internet applications). The image compression algorithms should not only give state-of-art performance, they should also provide other features and functionalities such as progressive transmission. Often, a rough approximation (thumbnail) of an image is sufficient for the user to decide whether to continue the image transmission or to abort; which accordingly helps to reduce time and bandwidth. That in turn necessitated the development of multi-resolution image compression schemes. The existed multi-resolution schemes (e.g., Multi-Level Progressive method) have shown high computational efficiency, but with a lack of the compression performance, in general. In this thesis, a LOw Complexity Multi-resolution Image Compression (LOCMIC) based on the Hierarchical INTerpolation (HINT) framework is presented. Moreover, a novel integration of the Just Noticeable Distortion (JND) for perceptual coding with the HINT framework to achieve a visual-lossless multi-resolution scheme has been proposed. In addition, various prediction formulas, a context-based prediction correction model and a multi-level Golomb parameter adaption approach have been investigated. The proposed LOCMIC (the lossless and the visual lossless) has contributed to the compression performance. The lossless LOCMIC has achieved a 3% reduced bit rate over LOCO-I, about 1% over JPEG2000, 3% over SPIHT, and 2% over CALIC. The Perceptual LOCMIC has been better in terms of bit rate than near-lossless JPEG-LS (at NEAR=2) with about 4.7%. Moreover, the decorrelation efficiency of the LOCMIC in terms of entropy has shown an advance of 2.8%, 4.5% than the MED and the conventional HINT respectively

Combined Industry, Space and Earth Science Data Compression Workshop

The sixth annual Space and Earth Science Data Compression Workshop and the third annual Data Compression Industry Workshop were held as a single combined workshop. The workshop was held April 4, 1996 in Snowbird, Utah in conjunction with the 1996 IEEE Data Compression Conference, which was held at the same location March 31 - April 3, 1996. The Space and Earth Science Data Compression sessions seek to explore opportunities for data compression to enhance the collection, analysis, and retrieval of space and earth science data. Of particular interest is data compression research that is integrated into, or has the potential to be integrated into, a particular space or earth science data information system. Preference is given to data compression research that takes into account the scien- tist's data requirements, and the constraints imposed by the data collection, transmission, distribution and archival systems