17,625 research outputs found
Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications
Recent modern displays are now able to render high dynamic range (HDR), high
resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently,
UHD HDR broadcasting and streaming have emerged as high quality premium
services. However, due to the lack of original UHD HDR video content,
appropriate conversion technologies are urgently needed to transform the legacy
low resolution (LR) standard dynamic range (SDR) videos into UHD HDR versions.
In this paper, we propose a joint super-resolution (SR) and inverse
tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct
mapping from LR SDR video to their HR HDR version. Joint SR and ITM is an
intricate task, where high frequency details must be restored for SR, jointly
with the local contrast, for ITM. Our network is able to restore fine details
by decomposing the input image and focusing on the separate base (low
frequency) and detail (high frequency) layers. Moreover, the proposed
modulation blocks apply location-variant operations to enhance local contrast.
The Deep SR-ITM shows good subjective quality with increased contrast and
details, outperforming the previous joint SR-ITM method.Comment: Accepted at ICCV 2019 (Oral
Learning an Inverse Tone Mapping Network with a Generative Adversarial Regularizer
Transferring a low-dynamic-range (LDR) image to a high-dynamic-range (HDR)
image, which is the so-called inverse tone mapping (iTM), is an important
imaging technique to improve visual effects of imaging devices. In this paper,
we propose a novel deep learning-based iTM method, which learns an inverse tone
mapping network with a generative adversarial regularizer. In the framework of
alternating optimization, we learn a U-Net-based HDR image generator to
transfer input LDR images to HDR ones, and a simple CNN-based discriminator to
classify the real HDR images and the generated ones. Specifically, when
learning the generator we consider the content-related loss and the generative
adversarial regularizer jointly to improve the stability and the robustness of
the generated HDR images. Using the learned generator as the proposed inverse
tone mapping network, we achieve superior iTM results to the state-of-the-art
methods consistently
Fast Multi-Layer Laplacian Enhancement
A novel, fast and practical way of enhancing images is introduced in this
paper. Our approach builds on Laplacian operators of well-known edge-aware
kernels, such as bilateral and nonlocal means, and extends these filter's
capabilities to perform more effective and fast image smoothing, sharpening and
tone manipulation. We propose an approximation of the Laplacian, which does not
require normalization of the kernel weights. Multiple Laplacians of the
affinity weights endow our method with progressive detail decomposition of the
input image from fine to coarse scale. These image components are blended by a
structure mask, which avoids noise/artifact magnification or detail loss in the
output image. Contributions of the proposed method to existing image editing
tools are: (1) Low computational and memory requirements, making it appropriate
for mobile device implementations (e.g. as a finish step in a camera pipeline),
(2) A range of filtering applications from detail enhancement to denoising with
only a few control parameters, enabling the user to apply a combination of
various (and even opposite) filtering effects
Text2Light: Zero-Shot Text-Driven HDR Panorama Generation
High-quality HDRIs(High Dynamic Range Images), typically HDR panoramas, are
one of the most popular ways to create photorealistic lighting and 360-degree
reflections of 3D scenes in graphics. Given the difficulty of capturing HDRIs,
a versatile and controllable generative model is highly desired, where layman
users can intuitively control the generation process. However, existing
state-of-the-art methods still struggle to synthesize high-quality panoramas
for complex scenes. In this work, we propose a zero-shot text-driven framework,
Text2Light, to generate 4K+ resolution HDRIs without paired training data.
Given a free-form text as the description of the scene, we synthesize the
corresponding HDRI with two dedicated steps: 1) text-driven panorama generation
in low dynamic range(LDR) and low resolution, and 2) super-resolution inverse
tone mapping to scale up the LDR panorama both in resolution and dynamic range.
Specifically, to achieve zero-shot text-driven panorama generation, we first
build dual codebooks as the discrete representation for diverse environmental
textures. Then, driven by the pre-trained CLIP model, a text-conditioned global
sampler learns to sample holistic semantics from the global codebook according
to the input text. Furthermore, a structure-aware local sampler learns to
synthesize LDR panoramas patch-by-patch, guided by holistic semantics. To
achieve super-resolution inverse tone mapping, we derive a continuous
representation of 360-degree imaging from the LDR panorama as a set of
structured latent codes anchored to the sphere. This continuous representation
enables a versatile module to upscale the resolution and dynamic range
simultaneously. Extensive experiments demonstrate the superior capability of
Text2Light in generating high-quality HDR panoramas. In addition, we show the
feasibility of our work in realistic rendering and immersive VR.Comment: SIGGRAPH Asia 2022; Project Page
https://frozenburning.github.io/projects/text2light/ Codes are available at
https://github.com/FrozenBurning/Text2Ligh
Fully-automatic inverse tone mapping algorithm based on dynamic mid-level tone mapping
High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show LDR content on HDR displays, it needs to be up-scaled using a so-called inverse tone mapping algorithm. Several techniques for inverse tone mapping have been proposed in the last years, going from simple approaches based on global and local operators to more advanced algorithms such as neural networks. Some of the drawbacks of existing techniques for inverse tone mapping are the need for human intervention, the high computation time for more advanced algorithms, limited low peak brightness, and the lack of the preservation of the artistic intentions. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing. Our proposed algorithm allows expanding LDR images into HDR images with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using the full-reference objective quality metrics HDR-VDP-2.2 and DRIM, and carrying out a subjective pair-wise comparison experiment. We compared our results with those obtained with the most recent methods found in the literature. Experimental results demonstrate that our proposed method outperforms the current state-of-the-art of simple inverse tone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques
Learning High Dynamic Range from Outdoor Panoramas
Outdoor lighting has extremely high dynamic range. This makes the process of
capturing outdoor environment maps notoriously challenging since special
equipment must be used. In this work, we propose an alternative approach. We
first capture lighting with a regular, LDR omnidirectional camera, and aim to
recover the HDR after the fact via a novel, learning-based inverse tonemapping
method. We propose a deep autoencoder framework which regresses linear, high
dynamic range data from non-linear, saturated, low dynamic range panoramas. We
validate our method through a wide set of experiments on synthetic data, as
well as on a novel dataset of real photographs with ground truth. Our approach
finds applications in a variety of settings, ranging from outdoor light capture
to image matching.Comment: 8 pages + 2 pages of citations, 10 figures. Accepted as an oral paper
at ICCV 201
Rendition: Reclaiming what a black box takes away
The premise of our work is deceptively familiar: A black box has
altered an image . Recover the image
. This black box might be any number of simple or complicated
things: a linear or non-linear filter, some app on your phone, etc. The latter
is a good canonical example for the problem we address: Given only "the app"
and an image produced by the app, find the image that was fed to the app. You
can run the given image (or any other image) through the app as many times as
you like, but you can not look inside the (code for the) app to see how it
works. At first blush, the problem sounds a lot like a standard inverse
problem, but it is not in the following sense: While we have access to the
black box and can run any image through it and observe the output,
we do not know how the block box alters the image. Therefore we have no
explicit form or model of . Nor are we necessarily interested in the
internal workings of the black box. We are simply happy to reverse its effect
on a particular image, to whatever extent possible. This is what we call the
"rendition" (rather than restoration) problem, as it does not fit the mold of
an inverse problem (blind or otherwise). We describe general conditions under
which rendition is possible, and provide a remarkably simple algorithm that
works for both contractive and expansive black box operators. The principal and
novel take-away message from our work is this surprising fact: One simple
algorithm can reliably undo a wide class of (not too violent) image
distortions.
A higher quality pdf of this paper is available at http://www.milanfar.or
Exposure: A White-Box Photo Post-Processing Framework
Retouching can significantly elevate the visual appeal of photos, but many
casual photographers lack the expertise to do this well. To address this
problem, previous works have proposed automatic retouching systems based on
supervised learning from paired training images acquired before and after
manual editing. As it is difficult for users to acquire paired images that
reflect their retouching preferences, we present in this paper a deep learning
approach that is instead trained on unpaired data, namely a set of photographs
that exhibits a retouching style the user likes, which is much easier to
collect. Our system is formulated using deep convolutional neural networks that
learn to apply different retouching operations on an input image. Network
training with respect to various types of edits is enabled by modeling these
retouching operations in a unified manner as resolution-independent
differentiable filters. To apply the filters in a proper sequence and with
suitable parameters, we employ a deep reinforcement learning approach that
learns to make decisions on what action to take next, given the current state
of the image. In contrast to many deep learning systems, ours provides users
with an understandable solution in the form of conventional retouching edits,
rather than just a "black-box" result. Through quantitative comparisons and
user studies, we show that this technique generates retouching results
consistent with the provided photo set.Comment: ACM Transaction on Graphics (Accepted with minor revisions
Generation of High Dynamic Range Illumination from a Single Image for the Enhancement of Undesirably Illuminated Images
This paper presents an algorithm that enhances undesirably illuminated images
by generating and fusing multi-level illuminations from a single image.The
input image is first decomposed into illumination and reflectance components by
using an edge-preserving smoothing filter. Then the reflectance component is
scaled up to improve the image details in bright areas. The illumination
component is scaled up and down to generate several illumination images that
correspond to certain camera exposure values different from the original. The
virtual multi-exposure illuminations are blended into an enhanced illumination,
where we also propose a method to generate appropriate weight maps for the tone
fusion. Finally, an enhanced image is obtained by multiplying the equalized
illumination and enhanced reflectance. Experiments show that the proposed
algorithm produces visually pleasing output and also yields comparable
objective results to the conventional enhancement methods, while requiring
modest computational loads
Semi-Global Weighted Least Squares in Image Filtering
Solving the global method of Weighted Least Squares (WLS) model in image
filtering is both time- and memory-consuming. In this paper, we present an
alternative approximation in a time- and memory- efficient manner which is
denoted as Semi-Global Weighed Least Squares (SG-WLS). Instead of solving a
large linear system, we propose to iteratively solve a sequence of subsystems
which are one-dimensional WLS models. Although each subsystem is
one-dimensional, it can take two-dimensional neighborhood information into
account due to the proposed special neighborhood construction. We show such a
desirable property makes our SG-WLS achieve close performance to the original
two-dimensional WLS model but with much less time and memory cost. While
previous related methods mainly focus on the 4-connected/8-connected
neighborhood system, our SG-WLS can handle a more general and larger
neighborhood system thanks to the proposed fast solution. We show such a
generalization can achieve better performance than the 4-connected/8-connected
neighborhood system in some applications. Our SG-WLS is times faster
than the WLS model. For an image of , the memory cost of SG-WLS is
at most at the magnitude of of that of the
WLS model. We show the effectiveness and efficiency of our SG-WLS in a range of
applications. The code is publicly available at:
https://github.com/wliusjtu/Semi-Global-Weighted-Least-Squares-in-Image-Filtering.Comment: Appearing in Proc. Int. Conf.Computer Vision (ICCV), 201
- …