1,815 research outputs found
Learnable Differencing Center for Nighttime Depth Perception
Depth completion is the task of recovering dense depth maps from sparse ones,
usually with the help of color images. Existing image-guided methods perform
well on daytime depth perception self-driving benchmarks, but struggle in
nighttime scenarios with poor visibility and complex illumination. To address
these challenges, we propose a simple yet effective framework called LDCNet.
Our key idea is to use Recurrent Inter-Convolution Differencing (RICD) and
Illumination-Affinitive Intra-Convolution Differencing (IAICD) to enhance the
nighttime color images and reduce the negative effects of the varying
illumination, respectively. RICD explicitly estimates global illumination by
differencing two convolutions with different kernels, treating the
small-kernel-convolution feature as the center of the large-kernel-convolution
feature in a new perspective. IAICD softly alleviates local relative light
intensity by differencing a single convolution, where the center is dynamically
aggregated based on neighboring pixels and the estimated illumination map in
RICD. On both nighttime depth completion and depth estimation tasks, extensive
experiments demonstrate the effectiveness of our LDCNet, reaching the state of
the art.Comment: 10 page
Instance Segmentation in the Dark
Existing instance segmentation techniques are primarily tailored for
high-visibility inputs, but their performance significantly deteriorates in
extremely low-light environments. In this work, we take a deep look at instance
segmentation in the dark and introduce several techniques that substantially
boost the low-light inference accuracy. The proposed method is motivated by the
observation that noise in low-light images introduces high-frequency
disturbances to the feature maps of neural networks, thereby significantly
degrading performance. To suppress this ``feature noise", we propose a novel
learning method that relies on an adaptive weighted downsampling layer, a
smooth-oriented convolutional block, and disturbance suppression learning.
These components effectively reduce feature noise during downsampling and
convolution operations, enabling the model to learn disturbance-invariant
features. Furthermore, we discover that high-bit-depth RAW images can better
preserve richer scene information in low-light conditions compared to typical
camera sRGB outputs, thus supporting the use of RAW-input algorithms. Our
analysis indicates that high bit-depth can be critical for low-light instance
segmentation. To mitigate the scarcity of annotated RAW datasets, we leverage a
low-light RAW synthetic pipeline to generate realistic low-light data. In
addition, to facilitate further research in this direction, we capture a
real-world low-light instance segmentation dataset comprising over two thousand
paired low/normal-light images with instance-level pixel-wise annotations.
Remarkably, without any image preprocessing, we achieve satisfactory
performance on instance segmentation in very low light (4~\% AP higher than
state-of-the-art competitors), meanwhile opening new opportunities for future
research.Comment: Accepted by International Journal of Computer Vision (IJCV) 202
Lighting up NeRF via Unsupervised Decomposition and Enhancement
Neural Radiance Field (NeRF) is a promising approach for synthesizing novel
views, given a set of images and the corresponding camera poses of a scene.
However, images photographed from a low-light scene can hardly be used to train
a NeRF model to produce high-quality results, due to their low pixel
intensities, heavy noise, and color distortion. Combining existing low-light
image enhancement methods with NeRF methods also does not work well due to the
view inconsistency caused by the individual 2D enhancement process. In this
paper, we propose a novel approach, called Low-Light NeRF (or LLNeRF), to
enhance the scene representation and synthesize normal-light novel views
directly from sRGB low-light images in an unsupervised manner. The core of our
approach is a decomposition of radiance field learning, which allows us to
enhance the illumination, reduce noise and correct the distorted colors jointly
with the NeRF optimization process. Our method is able to produce novel view
images with proper lighting and vivid colors and details, given a collection of
camera-finished low dynamic range (8-bits/channel) images from a low-light
scene. Experiments demonstrate that our method outperforms existing low-light
enhancement methods and NeRF methods.Comment: ICCV 2023. Project website: https://whyy.site/paper/llner
I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images
Image dehazing has become an important computational imaging topic in the
recent years. However, due to the lack of ground truth images, the comparison
of dehazing methods is not straightforward, nor objective. To overcome this
issue we introduce a new dataset -named I-HAZE- that contains 35 image pairs of
hazy and corresponding haze-free (ground-truth) indoor images. Different from
most of the existing dehazing databases, hazy images have been generated using
real haze produced by a professional haze machine. For easy color calibration
and improved assessment of dehazing algorithms, each scene include a MacBeth
color checker. Moreover, since the images are captured in a controlled
environment, both haze-free and hazy images are captured under the same
illumination conditions. This represents an important advantage of the I-HAZE
dataset that allows us to objectively compare the existing image dehazing
techniques using traditional image quality metrics such as PSNR and SSIM
Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks
The limited dynamic range of commercial compact camera sensors results in an
inaccurate representation of scenes with varying illumination conditions,
adversely affecting image quality and subsequently limiting the performance of
underlying image processing algorithms. Current state-of-the-art (SoTA)
convolutional neural networks (CNN) are developed as post-processing techniques
to independently recover under-/over-exposed images. However, when applied to
images containing real-world degradations such as glare, high-beam, color
bleeding with varying noise intensity, these algorithms amplify the
degradations, further degrading image quality. We propose a lightweight
two-stage image enhancement algorithm sequentially balancing illumination and
noise removal using frequency priors for structural guidance to overcome these
limitations. Furthermore, to ensure realistic image quality, we leverage the
relationship between frequency and spatial domain properties of an image and
propose a Fourier spectrum-based adversarial framework (AFNet) for consistent
image enhancement under varying illumination conditions. While current
formulations of image enhancement are envisioned as post-processing techniques,
we examine if such an algorithm could be extended to integrate the
functionality of the Image Signal Processing (ISP) pipeline within the camera
sensor benefiting from RAW sensor data and lightweight CNN architecture. Based
on quantitative and qualitative evaluations, we also examine the practicality
and effects of image enhancement techniques on the performance of common
perception tasks such as object detection and semantic segmentation in varying
illumination conditions.Comment: Accepted in BMVC 202
Learning to Dehaze from Realistic Scene with A Fast Physics-based Dehazing Network
Dehazing is a popular computer vision topic for long. A real-time dehazing
method with reliable performance is highly desired for many applications such
as autonomous driving. While recent learning-based methods require datasets
containing pairs of hazy images and clean ground truth references, it is
generally impossible to capture accurate ground truth in real scenes. Many
existing works compromise this difficulty to generate hazy images by rendering
the haze from depth on common RGBD datasets using the haze imaging model.
However, there is still a gap between the synthetic datasets and real hazy
images as large datasets with high-quality depth are mostly indoor and depth
maps for outdoor are imprecise. In this paper, we complement the existing
datasets with a new, large, and diverse dehazing dataset containing real
outdoor scenes from High-Definition (HD) 3D movies. We select a large number of
high-quality frames of real outdoor scenes and render haze on them using depth
from stereo. Our dataset is more realistic than existing ones and we
demonstrate that using this dataset greatly improves the dehazing performance
on real scenes. In addition to the dataset, we also propose a light and
reliable dehazing network inspired by the physics model. Our approach
outperforms other methods by a large margin and becomes the new
state-of-the-art method. Moreover, the light-weight design of the network
enables our method to run at a real-time speed, which is much faster than other
baseline methods
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
When enhancing low-light images, many deep learning algorithms are based on
the Retinex theory. However, the Retinex model does not consider the
corruptions hidden in the dark or introduced by the light-up process. Besides,
these methods usually require a tedious multi-stage training pipeline and rely
on convolutional neural networks, showing limitations in capturing long-range
dependencies. In this paper, we formulate a simple yet principled One-stage
Retinex-based Framework (ORF). ORF first estimates the illumination information
to light up the low-light image and then restores the corruption to produce the
enhanced image. We design an Illumination-Guided Transformer (IGT) that
utilizes illumination representations to direct the modeling of non-local
interactions of regions with different lighting conditions. By plugging IGT
into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative
and qualitative experiments demonstrate that our Retinexformer significantly
outperforms state-of-the-art methods on thirteen benchmarks. The user study and
application on low-light object detection also reveal the latent practical
values of our method. Code, models, and results are available at
https://github.com/caiyuanhao1998/RetinexformerComment: ICCV 2023; The first Transformer-based method for low-light image
enhancemen
- …