200 research outputs found
GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving
as a partial observation of the High Dynamic Range (HDR) visual world. Despite
limited dynamic range, these LDR images are often captured with different
exposures, implicitly containing information about the underlying HDR image
distribution. Inspired by this intuition, in this work we present, to the best
of our knowledge, the first method for learning a generative model of HDR
images from in-the-wild LDR image collections in a fully unsupervised manner.
The key idea is to train a generative adversarial network (GAN) to generate HDR
images which, when projected to LDR under various exposures, are
indistinguishable from real LDR images. The projection from HDR to LDR is
achieved via a camera model that captures the stochasticity in exposure and
camera response function. Experiments show that our method GlowGAN can
synthesize photorealistic HDR images in many challenging cases such as
landscapes, lightning, or windows, where previous supervised generative models
produce overexposed images. We further demonstrate the new application of
unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does
not need HDR images or paired multi-exposure images for training, yet it
reconstructs more plausible information for overexposed regions than
state-of-the-art supervised learning models trained on such data
GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, servingas a partial observation of the High Dynamic Range (HDR) visual world. Despitelimited dynamic range, these LDR images are often captured with differentexposures, implicitly containing information about the underlying HDR imagedistribution. Inspired by this intuition, in this work we present, to the bestof our knowledge, the first method for learning a generative model of HDRimages from in-the-wild LDR image collections in a fully unsupervised manner.The key idea is to train a generative adversarial network (GAN) to generate HDRimages which, when projected to LDR under various exposures, areindistinguishable from real LDR images. The projection from HDR to LDR isachieved via a camera model that captures the stochasticity in exposure andcamera response function. Experiments show that our method GlowGAN cansynthesize photorealistic HDR images in many challenging cases such aslandscapes, lightning, or windows, where previous supervised generative modelsproduce overexposed images. We further demonstrate the new application ofunsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method doesnot need HDR images or paired multi-exposure images for training, yet itreconstructs more plausible information for overexposed regions thanstate-of-the-art supervised learning models trained on such data.<br
Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation
Modern displays are capable of rendering video content with high dynamic
range (HDR) and wide color gamut (WCG). However, the majority of available
resources are still in standard dynamic range (SDR). As a result, there is
significant value in transforming existing SDR content into the HDRTV standard.
In this paper, we define and analyze the SDRTV-to-HDRTV task by modeling the
formation of SDRTV/HDRTV content. Our analysis and observations indicate that a
naive end-to-end supervised training pipeline suffers from severe gamut
transition errors. To address this issue, we propose a novel three-step
solution pipeline called HDRTVNet++, which includes adaptive global color
mapping, local enhancement, and highlight refinement. The adaptive global color
mapping step uses global statistics as guidance to perform image-adaptive color
mapping. A local enhancement network is then deployed to enhance local details.
Finally, we combine the two sub-networks above as a generator and achieve
highlight consistency through GAN-based joint training. Our method is primarily
designed for ultra-high-definition TV content and is therefore effective and
lightweight for processing 4K resolution images. We also construct a dataset
using HDR videos in the HDR10 standard, named HDRTV1K that contains 1235 and
117 training images and 117 testing images, all in 4K resolution. Besides, we
select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Our
final results demonstrate state-of-the-art performance both quantitatively and
visually. The code, model and dataset are available at
https://github.com/xiaom233/HDRTVNet-plus.Comment: Extended version of HDRTVNe
JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video
Joint learning of super-resolution (SR) and inverse tone-mapping (ITM) has
been explored recently, to convert legacy low resolution (LR) standard dynamic
range (SDR) videos to high resolution (HR) high dynamic range (HDR) videos for
the growing need of UHD HDR TV/broadcasting applications. However, previous
CNN-based methods directly reconstruct the HR HDR frames from LR SDR frames,
and are only trained with a simple L2 loss. In this paper, we take a
divide-and-conquer approach in designing a novel GAN-based joint SR-ITM
network, called JSI-GAN, which is composed of three task-specific subnets: an
image reconstruction subnet, a detail restoration (DR) subnet and a local
contrast enhancement (LCE) subnet. We delicately design these subnets so that
they are appropriately trained for the intended purpose, learning a pair of
pixel-wise 1D separable filters via the DR subnet for detail restoration and a
pixel-wise 2D local filter by the LCE subnet for contrast enhancement.
Moreover, to train the JSI-GAN effectively, we propose a novel detail GAN loss
alongside the conventional GAN loss, which helps enhancing both local details
and contrasts to reconstruct high quality HR HDR results. When all subnets are
jointly trained well, the predicted HR HDR results of higher quality are
obtained with at least 0.41 dB gain in PSNR over those generated by the
previous methods.Comment: The first two authors contributed equally to this work. Accepted at
AAAI 2020. (Camera-ready version
๋ค์ค ๋ ธ์ถ ์ ๋ ฅ์ ํผ์ณ ๋ถํด๋ฅผ ํตํ ํ์ด ๋ค์ด๋๋ฏน ๋ ์ธ์ง ์์ ์์ฑ ๋ฐฉ๋ฒ
ํ์๋
ผ๋ฌธ(์์ฌ) -- ์์ธ๋ํ๊ต๋ํ์ : ๊ณต๊ณผ๋ํ ํ๋๊ณผ์ ์ธ๊ณต์ง๋ฅ์ ๊ณต, 2022. 8. ์กฐ๋จ์ต.Multi-exposure high dynamic range (HDR) imaging aims to generate an HDR image from multiple differently exposed low dynamic range (LDR) images. Multi-exposure HDR imaging is a challenging task due to two major problems. One is misalignments among the input LDR images, which can cause ghosting artifacts on result HDR, and the other is missing information on LDR images due to under-/over-exposed region. Although previous methods tried to align input LDR images with traditional methods(e.g., homography, optical flow), they still suffer undesired artifacts on the result HDR image due to estimation errors that occurred in aligning step.
In this dissertation, disentangled feature-guided HDR network (DFGNet) is proposed to alleviate the above-stated problems. Specifically, exposure features and spatial features are first extracted from input LDR images, and they are disentangled from each other. Then, these features are processed through the proposed DFG modules, which produce a high-quality HDR image. The proposed DFGNet shows outstanding performance compared to previous methods, achieving the PSNR-โ of 41.89dB and the PSNR-ฮผ of 44.19dB.๋ค์ค ๋
ธ์ถ(Multiple-exposure) ํ์ด ๋ค์ด๋๋ฏน ๋ ์ธ์ง(High Dynamic Range, HDR) ์ด๋ฏธ์ง์ ๊ฐ๊ฐ ๋ค๋ฅธ ๋
ธ์ถ ์ ๋๋ก ์ดฌ์๋ ๋ค์์ ๋ก์ฐ ๋ค์ด๋๋ฏน ๋ ์ธ์ง(Low Dynamic Range, LDR) ์ด๋ฏธ์ง๋ฅผ ์ฌ์ฉํ์ฌ ํ๋์ HDR ์ด๋ฏธ์ง๋ฅผ ์์ฑํ๋ ๊ฒ์ ๋ชฉํ๋ก ํ๋ค. ๋ค์ค ๋
ธ์ถ HDR ์ด๋ฏธ์ง์ ๋ ๊ฐ์ง ์ฃผ์ ๋ฌธ์ ์ ๋๋ฌธ์ ์ด๋ ค์์ด ์๋๋ฐ, ํ๋๋ ์
๋ ฅ LDR ์ด๋ฏธ์ง๋ค์ด ์ ๋ ฌ๋์ง ์์ ๊ฒฐ๊ณผ HDR ์ด๋ฏธ์ง์์ ๊ณ ์คํธ ์ํฐํฉํธ(Ghosting Artifact)๊ฐ ๋ฐ์ํ ์ ์๋ค๋ ์ ๊ณผ, ๋ ๋ค๋ฅธ ํ๋๋ LDR ์ด๋ฏธ์ง๋ค์ ๊ณผ์๋
ธ์ถ(Under-exposure) ๋ฐ ๊ณผ๋ค๋
ธ์ถ(Over-exposure) ๋ ์์ญ์์ ์ ๋ณด ์์ค์ด ๋ฐ์ํ๋ค๋ ์ ์ด๋ค. ๊ณผ๊ฑฐ์ ๋ฐฉ๋ฒ๋ค์ด ๊ณ ์ ์ ์ธ ์ด๋ฏธ์ง ์ ๋ ฌ ๋ฐฉ๋ฒ๋ค(e.g., homography, optical flow)์ ์ฌ์ฉํ์ฌ ์
๋ ฅ LDR ์ด๋ฏธ์ง๋ค์ ์ ์ฒ๋ฆฌ ๊ณผ์ ์์ ์ ๋ ฌํ ์ฌ ๋ณํฉํ๋ ์๋๋ฅผ ํ์ง๋ง, ์ด ๊ณผ์ ์์ ๋ฐ์ํ๋ ์ถ์ ์ค๋ฅ๋ก ์ธํด ์ดํ ๋จ๊ณ์ ์
์ํญ์ ๋ฏธ์นจ์ผ๋ก์จ ๋ฐ์ํ๋ ์ฌ๋ฌ๊ฐ์ง ๋ถ์ ์ ํ ์ํฐํฉํธ๋ค์ด ๊ฒฐ๊ณผ HDR ์ด๋ฏธ์ง์์ ๋ํ๋๊ณ ์๋ค.
๋ณธ ์ฌ์ฌ์์๋ ํผ์ณ ๋ถํด๋ฅผ ์์ฉํ HDR ๋คํธ์ํฌ๋ฅผ ์ ์ํ์ฌ, ์ธ๊ธ๋ ๋ฌธ์ ๋ค์ ๊ฒฝ๊ฐํ๊ณ ์ ํ๋ค. ๊ตฌ์ฒด์ ์ผ๋ก, ๋จผ์ LDR ์ด๋ฏธ์ง๋ค์ ๋
ธ์ถ ํผ์ณ์ ๊ณต๊ฐ ํผ์ณ๋ก ๋ถํดํ๊ณ , ๋ถํด๋ ํผ์ณ๋ฅผ HDR ๋คํธ์ํฌ์์ ํ์ฉํจ์ผ๋ก์จ ๊ณ ํ์ง์ HDR ์ด๋ฏธ์ง
๋ฅผ ์์ฑํ ์ ์๋๋ก ํ๋ค. ์ ์ํ ๋คํธ์ํฌ๋ ์ฑ๋ฅ ์งํ์ธ PSNR-โ๊ณผ PSNR-ฮผ์์ ๊ฐ๊ฐ 41.89dB, 44.19dB์ ์ฑ๋ฅ์ ๋ฌ์ฑํจ์ผ๋ก์จ, ๊ธฐ์กด ๋ฐฉ๋ฒ๋ค๋ณด๋ค ์ฐ์ํจ์ ์
์ฆํ๋ค.1 Introduction 1
2 Related Works 4
2.1 Single-frame HDR imaging 4
2.2 Multi-frame HDR imaging with dynamic scenes 6
3 Proposed Method 10
3.1 Disentangle Network for Feature Extraction 10
3.2 Disentangle Features Guided Network 16
4 Experimental Results 22
4.1 Implementation and Details 22
4.2 Comparison with State-of-the-art Methods 22
5 Ablation Study 30
5.1 Impact of Proposed Modules 30
6 Conclusion 32
Abstract (In Korean) 39์
- โฆ