19 research outputs found
DarkVisionNet: Low-Light Imaging via RGB-NIR Fusion with Deep Inconsistency Prior
RGB-NIR fusion is a promising method for low-light imaging. However,
high-intensity noise in low-light images amplifies the effect of structure
inconsistency between RGB-NIR images, which fails existing algorithms. To
handle this, we propose a new RGB-NIR fusion algorithm called Dark Vision Net
(DVN) with two technical novelties: Deep Structure and Deep Inconsistency Prior
(DIP). The Deep Structure extracts clear structure details in deep multiscale
feature space rather than raw input space, which is more robust to noisy
inputs. Based on the deep structures from both RGB and NIR domains, we
introduce the DIP to leverage the structure inconsistency to guide the fusion
of RGB-NIR. Benefiting from this, the proposed DVN obtains high-quality
lowlight images without the visual artifacts. We also propose a new dataset
called Dark Vision Dataset (DVD), consisting of aligned RGB-NIR image pairs, as
the first public RGBNIR fusion benchmark. Quantitative and qualitative results
on the proposed benchmark show that DVN significantly outperforms other
comparison algorithms in PSNR and SSIM, especially in extremely low light
conditions.Comment: Accepted to AAAI 202
GIA-Net: Global Information Aware Network for Low-light Imaging
It is extremely challenging to acquire perceptually plausible images under
low-light conditions due to low SNR. Most recently, U-Nets have shown promising
results for low-light imaging. However, vanilla U-Nets generate images with
artifacts such as color inconsistency due to the lack of global color
information. In this paper, we propose a global information aware (GIA) module,
which is capable of extracting and integrating the global information into the
network to improve the performance of low-light imaging. The GIA module can be
inserted into a vanilla U-Net with negligible extra learnable parameters or
computational cost. Moreover, a GIA-Net is constructed, trained and evaluated
on a large scale real-world low-light imaging dataset. Experimental results
show that the proposed GIA-Net outperforms the state-of-the-art methods in
terms of four metrics, including deep metrics that measure perceptual
similarities. Extensive ablation studies have been conducted to verify the
effectiveness of the proposed GIA-Net for low-light imaging by utilizing global
information.Comment: 16 pages 6 figures; accepted to AIM at ECCV 202
Efficient Deep Image Denoising via Class Specific Convolution
Deep neural networks have been widely used in image denoising during the past
few years. Even though they achieve great success on this problem, they are
computationally inefficient which makes them inappropriate to be implemented in
mobile devices. In this paper, we propose an efficient deep neural network for
image denoising based on pixel-wise classification. Despite using a
computationally efficient network cannot effectively remove the noises from any
content, it is still capable to denoise from a specific type of pattern or
texture. The proposed method follows such a divide and conquer scheme. We first
use an efficient U-net to pixel-wisely classify pixels in the noisy image based
on the local gradient statistics. Then we replace part of the convolution
layers in existing denoising networks by the proposed Class Specific
Convolution layers (CSConv) which use different weights for different classes
of pixels. Quantitative and qualitative evaluations on public datasets
demonstrate that the proposed method can reduce the computational costs without
sacrificing the performance compared to state-of-the-art algorithms.Comment: The Thirty-Fifth AAAI Conference on Artificial Intelligence(AAAI-21
Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency
Recently, image enhancement and restoration have become important
applications on mobile devices, such as super-resolution and image deblurring.
However, most state-of-the-art networks present extremely high computational
complexity. This makes them difficult to be deployed on mobile devices with
acceptable latency. Moreover, when deploying to different mobile devices, there
is a large latency variation due to the difference and limitation of deep
learning accelerators on mobile devices. In this paper, we conduct a search of
portable network architectures for better quality-latency trade-off across
mobile devices. We further present the effectiveness of widely used network
optimizations for image deblurring task. This paper provides comprehensive
experiments and comparisons to uncover the in-depth analysis for both latency
and image quality. Through all the above works, we demonstrate the successful
deployment of image deblurring application on mobile devices with the
acceleration of deep learning accelerators. To the best of our knowledge, this
is the first paper that addresses all the deployment issues of image deblurring
task across mobile devices. This paper provides practical
deployment-guidelines, and is adopted by the championship-winning team in NTIRE
2020 Image Deblurring Challenge on Smartphone Track.Comment: CVPR 2020 Workshop on New Trends in Image Restoration and Enhancement
(NTIRE
Single Cell Training on Architecture Search for Image Denoising
Neural Architecture Search (NAS) for automatically finding the optimal
network architecture has shown some success with competitive performances in
various computer vision tasks. However, NAS in general requires a tremendous
amount of computations. Thus reducing computational cost has emerged as an
important issue. Most of the attempts so far has been based on manual
approaches, and often the architectures developed from such efforts dwell in
the balance of the network optimality and the search cost. Additionally, recent
NAS methods for image restoration generally do not consider dynamic operations
that may transform dimensions of feature maps because of the dimensionality
mismatch in tensor calculations. This can greatly limit NAS in its search for
optimal network structure. To address these issues, we re-frame the optimal
search problem by focusing at component block level. From previous work, it's
been shown that an effective denoising block can be connected in series to
further improve the network performance. By focusing at block level, the search
space of reinforcement learning becomes significantly smaller and evaluation
process can be conducted more rapidly. In addition, we integrate an innovative
dimension matching modules for dealing with spatial and channel-wise mismatch
that may occur in the optimal design search. This allows much flexibility in
optimal network search within the cell block. With these modules, then we
employ reinforcement learning in search of an optimal image denoising network
at a module level. Computational efficiency of our proposed Denoising Prior
Neural Architecture Search (DPNAS) was demonstrated by having it complete an
optimal architecture search for an image restoration task by just one day with
a single GPU