High Dynamic Range Visual Content Compression

This thesis addresses the research questions of High Dynamic Range (HDR) visual contents compression. The HDR representations are intended to represent the actual physical value of the light rather than exposed value. The current HDR compression schemes are the extension of legacy Low Dynamic Range (LDR) compressions, by using Tone-Mapping Operators (TMO) to reduce the dynamic range of the HDR contents. However, introducing TMO increases the overall computational complexity, and it causes the temporal artifacts. Furthermore, these compression schemes fail to compress non-salient region differently than the salient region, when Human Visual System (HVS) perceives them differently. The main contribution of this thesis is to propose a novel Mapping-free visual saliency-guided HDR content compression scheme. Firstly, the relationship of Discrete Wavelet Transform (DWT) lifting steps and TMO are explored. A novel approach to compress HDR image by Joint Photographic Experts Group (JPEG) 2000 codec while backward compatible to LDR is proposed. This approach exploits the reversibility of tone mapping and scalability of DWT. Secondly, the importance of the TMO in the HDR compression is evaluated in this thesis. A mapping-free post HDR image compression based on JPEG and JPEG2000 standard codecs for current HDR image formats is proposed. This approach exploits the structure of HDR formats. It has an equivalent compression performance and the lowest computational complexity compared to the existing HDR lossy compressions (50% lower than the state-of-the-art). Finally, the shortcomings of the current HDR visual saliency models, and HDR visual saliency-guided compression are explored in this thesis. A spatial saliency model for HDR visual content outperform others by 10% for spatial visual prediction task with 70% lower computational complexity is proposed. Furthermore, the experiment suggested more than 90% temporal saliency is predicted by the proposed spatial model. Moreover, the proposed saliency model can be used to guide the HDR compression by applying different quantization factor according to the intensity of predicted saliency map

Algorithms for compression of high dynamic range images and video

The recent advances in sensor and display technologies have brought upon the High Dynamic Range (HDR) imaging capability. The modern multiple exposure HDR sensors can achieve the dynamic range of 100-120 dB and LED and OLED display devices have contrast ratios of 10^5:1 to 10^6:1. Despite the above advances in technology the image/video compression algorithms and associated hardware are yet based on Standard Dynamic Range (SDR) technology, i.e. they operate within an effective dynamic range of up to 70 dB for 8 bit gamma corrected images. Further the existing infrastructure for content distribution is also designed for SDR, which creates interoperability problems with true HDR capture and display equipment. The current solutions for the above problem include tone mapping the HDR content to fit SDR. However this approach leads to image quality associated problems, when strong dynamic range compression is applied. Even though some HDR-only solutions have been proposed in literature, they are not interoperable with current SDR infrastructure and are thus typically used in closed systems. Given the above observations a research gap was identified in the need for efficient algorithms for the compression of still images and video, which are capable of storing full dynamic range and colour gamut of HDR images and at the same time backward compatible with existing SDR infrastructure. To improve the usability of SDR content it is vital that any such algorithms should accommodate different tone mapping operators, including those that are spatially non-uniform. In the course of the research presented in this thesis a novel two layer CODEC architecture is introduced for both HDR image and video coding. Further a universal and computationally efficient approximation of the tone mapping operator is developed and presented. It is shown that the use of perceptually uniform colourspaces for internal representation of pixel data enables improved compression efficiency of the algorithms. Further proposed novel approaches to the compression of metadata for the tone mapping operator is shown to improve compression performance for low bitrate video content. Multiple compression algorithms are designed, implemented and compared and quality-complexity trade-offs are identified. Finally practical aspects of implementing the developed algorithms are explored by automating the design space exploration flow and integrating the high level systems design framework with domain specific tools for synthesis and simulation of multiprocessor systems. The directions for further work are also presented

다중 노출 입력의 피쳐 분해를 통한 하이 다이나믹 레인지 영상 생성 방법

학위논문(석사) -- 서울대학교대학원 : 공과대학 협동과정 인공지능전공, 2022. 8. 조남익.Multi-exposure high dynamic range (HDR) imaging aims to generate an HDR image from multiple differently exposed low dynamic range (LDR) images. Multi-exposure HDR imaging is a challenging task due to two major problems. One is misalignments among the input LDR images, which can cause ghosting artifacts on result HDR, and the other is missing information on LDR images due to under-/over-exposed region. Although previous methods tried to align input LDR images with traditional methods(e.g., homography, optical flow), they still suffer undesired artifacts on the result HDR image due to estimation errors that occurred in aligning step. In this dissertation, disentangled feature-guided HDR network (DFGNet) is proposed to alleviate the above-stated problems. Specifically, exposure features and spatial features are first extracted from input LDR images, and they are disentangled from each other. Then, these features are processed through the proposed DFG modules, which produce a high-quality HDR image. The proposed DFGNet shows outstanding performance compared to previous methods, achieving the PSNR-ℓ of 41.89dB and the PSNR-μ of 44.19dB.다중 노출(Multiple-exposure) 하이 다이나믹 레인지(High Dynamic Range, HDR) 이미징은 각각 다른 노출 정도로 촬영된 다수의 로우 다이나믹 레인지(Low Dynamic Range, LDR) 이미지를 사용하여 하나의 HDR 이미지를 생성하는 것을 목표로 한다. 다중 노출 HDR 이미징은 두 가지 주요 문제점 때문에 어려움이 있는데, 하나는 입력 LDR 이미지들이 정렬되지 않아 결과 HDR 이미지에서 고스트 아티팩트(Ghosting Artifact)가 발생할 수 있다는 점과, 또 다른 하나는 LDR 이미지들의 과소노출(Under-exposure) 및 과다노출(Over-exposure) 된 영역에서 정보 손실이 발생한다는 점이다. 과거의 방법들이 고전적인 이미지 정렬 방법들(e.g., homography, optical flow)을 사용하여 입력 LDR 이미지들을 전처리 과정에서 정렬하 여 병합하는 시도를 했지만, 이 과정에서 발생하는 추정 오류로 인해 이후 단계에 악영항을 미침으로써 발생하는 여러가지 부적절한 아티팩트들이 결과 HDR 이미지에서 나타나고 있다. 본 심사에서는 피쳐 분해를 응용한 HDR 네트워크를 제안하여, 언급된 문제들을 경감하고자 한다. 구체적으로, 먼저 LDR 이미지들을 노출 피쳐와 공간 피쳐로 분해하고, 분해된 피쳐를 HDR 네트워크에서 활용함으로써 고품질의 HDR 이미지 를 생성할 수 있도록 한다. 제안한 네트워크는 성능 지표인 PSNR-ℓ과 PSNR-μ에서 각각 41.89dB, 44.19dB의 성능을 달성함으로써, 기존 방법들보다 우수함을 입증한다.1 Introduction 1 2 Related Works 4 2.1 Single-frame HDR imaging 4 2.2 Multi-frame HDR imaging with dynamic scenes 6 3 Proposed Method 10 3.1 Disentangle Network for Feature Extraction 10 3.2 Disentangle Features Guided Network 16 4 Experimental Results 22 4.1 Implementation and Details 22 4.2 Comparison with State-of-the-art Methods 22 5 Ablation Study 30 5.1 Impact of Proposed Modules 30 6 Conclusion 32 Abstract (In Korean) 39석