1,737 research outputs found
MemNet: A Persistent Memory Network for Image Restoration
Recently, very deep convolutional neural networks (CNNs) have been attracting
considerable attention in image restoration. However, as the depth grows, the
long-term dependency problem is rarely realized for these very deep models,
which results in the prior states/layers having little influence on the
subsequent ones. Motivated by the fact that human thoughts have persistency, we
propose a very deep persistent memory network (MemNet) that introduces a memory
block, consisting of a recursive unit and a gate unit, to explicitly mine
persistent memory through an adaptive learning process. The recursive unit
learns multi-level representations of the current state under different
receptive fields. The representations and the outputs from the previous memory
blocks are concatenated and sent to the gate unit, which adaptively controls
how much of the previous states should be reserved, and decides how much of the
current state should be stored. We apply MemNet to three image restoration
tasks, i.e., image denosing, super-resolution and JPEG deblocking.
Comprehensive experiments demonstrate the necessity of the MemNet and its
unanimous superiority on all three tasks over the state of the arts. Code is
available at https://github.com/tyshiwo/MemNet.Comment: Accepted by ICCV 2017 (Spotlight presentation
Noise-Level Estimation from Single Color Image Using Correlations Between Textures in RGB Channels
We propose a simple method for estimating noise level from a single color
image. In most image-denoising algorithms, an accurate noise-level estimate
results in good denoising performance; however, it is difficult to estimate
noise level from a single image because it is an ill-posed problem. We tackle
this problem by using prior knowledge that textures are highly correlated
between RGB channels and noise is uncorrelated to other signals. We also
extended our method for RAW images because they are available in almost all
digital cameras and often used in practical situations. Experiments show the
high noise-estimation performance of our method in synthetic noisy images. We
also applied our method to natural images including RAW images and achieved
better noise-estimation performance than conventional methods.Comment: 9 pages, 11 figure
An End-to-End Compression Framework Based on Convolutional Neural Networks
Deep learning, e.g., convolutional neural networks (CNNs), has achieved great
success in image processing and computer vision especially in high level vision
applications such as recognition and understanding. However, it is rarely used
to solve low-level vision problems such as image compression studied in this
paper. Here, we move forward a step and propose a novel compression framework
based on CNNs. To achieve high-quality image compression at low bit rates, two
CNNs are seamlessly integrated into an end-to-end compression framework. The
first CNN, named compact convolutional neural network (ComCNN), learns an
optimal compact representation from an input image, which preserves the
structural information and is then encoded using an image codec (e.g., JPEG,
JPEG2000 or BPG). The second CNN, named reconstruction convolutional neural
network (RecCNN), is used to reconstruct the decoded image with high-quality in
the decoding end. To make two CNNs effectively collaborate, we develop a
unified end-to-end learning algorithm to simultaneously learn ComCNN and
RecCNN, which facilitates the accurate reconstruction of the decoded image
using RecCNN. Such a design also makes the proposed compression framework
compatible with existing image coding standards. Experimental results validate
that the proposed compression framework greatly outperforms several compression
frameworks that use existing image coding standards with state-of-the-art
deblocking or denoising post-processing methods.Comment: Submitted to IEEE Transactions on Circuits and Systems for Video
Technolog
Quality Adaptive Low-Rank Based JPEG Decoding with Applications
Small compression noises, despite being transparent to human eyes, can
adversely affect the results of many image restoration processes, if left
unaccounted for. Especially, compression noises are highly detrimental to
inverse operators of high-boosting (sharpening) nature, such as deblurring and
superresolution against a convolution kernel. By incorporating the non-linear
DCT quantization mechanism into the formulation for image restoration, we
propose a new sparsity-based convex programming approach for joint compression
noise removal and image restoration. Experimental results demonstrate
significant performance gains of the new approach over existing image
restoration methods
CFSNet: Toward a Controllable Feature Space for Image Restoration
Deep learning methods have witnessed the great progress in image restoration
with specific metrics (e.g., PSNR, SSIM). However, the perceptual quality of
the restored image is relatively subjective, and it is necessary for users to
control the reconstruction result according to personal preferences or image
characteristics, which cannot be done using existing deterministic networks.
This motivates us to exquisitely design a unified interactive framework for
general image restoration tasks. Under this framework, users can control
continuous transition of different objectives, e.g., the perception-distortion
trade-off of image super-resolution, the trade-off between noise reduction and
detail preservation. We achieve this goal by controlling the latent features of
the designed network. To be specific, our proposed framework, named
Controllable Feature Space Network (CFSNet), is entangled by two branches based
on different objectives. Our framework can adaptively learn the coupling
coefficients of different layers and channels, which provides finer control of
the restored image quality. Experiments on several typical image restoration
tasks fully validate the effective benefits of the proposed method. Code is
available at https://github.com/qibao77/CFSNet.Comment: Accepted by ICCV 201
On learning optimized reaction diffusion processes for effective image restoration
For several decades, image restoration remains an active research topic in
low-level computer vision and hence new approaches are constantly emerging.
However, many recently proposed algorithms achieve state-of-the-art performance
only at the expense of very high computation time, which clearly limits their
practical relevance. In this work, we propose a simple but effective approach
with both high computational efficiency and high restoration quality. We extend
conventional nonlinear reaction diffusion models by several parametrized linear
filters as well as several parametrized influence functions. We propose to
train the parameters of the filters and the influence functions through a loss
based approach. Experiments show that our trained nonlinear reaction diffusion
models largely benefit from the training of the parameters and finally lead to
the best reported performance on common test datasets for image restoration.
Due to their structural simplicity, our trained models are highly efficient and
are also well-suited for parallel computation on GPUs.Comment: 9 pages, 3 figures, 3 tables. CVPR2015 oral presentation together
with the supplemental material of 13 pages, 8 pages (Notes on diffusion
networks
Toward Convolutional Blind Denoising of Real Photographs
While deep convolutional neural networks (CNNs) have achieved impressive
success in image denoising with additive white Gaussian noise (AWGN), their
performance remains limited on real-world noisy photographs. The main reason is
that their learned models are easy to overfit on the simplified AWGN model
which deviates severely from the complicated real-world noise model. In order
to improve the generalization ability of deep CNN denoisers, we suggest
training a convolutional blind denoising network (CBDNet) with more realistic
noise model and real-world noisy-clean image pairs. On the one hand, both
signal-dependent noise and in-camera signal processing pipeline is considered
to synthesize realistic noisy images. On the other hand, real-world noisy
photographs and their nearly noise-free counterparts are also included to train
our CBDNet. To further provide an interactive strategy to rectify denoising
result conveniently, a noise estimation subnetwork with asymmetric learning to
suppress under-estimation of noise level is embedded into CBDNet. Extensive
experimental results on three datasets of real-world noisy photographs clearly
demonstrate the superior performance of CBDNet over state-of-the-arts in terms
of quantitative metrics and visual quality. The code has been made available at
https://github.com/GuoShi28/CBDNet
Reconfiguring the Imaging Pipeline for Computer Vision
Advancements in deep learning have ignited an explosion of research on
efficient hardware for embedded computer vision. Hardware vision acceleration,
however, does not address the cost of capturing and processing the image data
that feeds these algorithms. We examine the role of the image signal processing
(ISP) pipeline in computer vision to identify opportunities to reduce
computation and save energy. The key insight is that imaging pipelines should
be designed to be configurable: to switch between a traditional photography
mode and a low-power vision mode that produces lower-quality image data
suitable only for computer vision. We use eight computer vision algorithms and
a reversible pipeline simulation tool to study the imaging system's impact on
vision performance. For both CNN-based and classical vision algorithms, we
observe that only two ISP stages, demosaicing and gamma compression, are
critical for task performance. We propose a new image sensor design that can
compensate for skipping these stages. The sensor design features an adjustable
resolution and tunable analog-to-digital converters (ADCs). Our proposed
imaging system's vision mode disables the ISP entirely and configures the
sensor to produce subsampled, lower-precision image data. This vision mode can
save ~75% of the average energy of a baseline photography mode while having
only a small impact on vision task accuracy
Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration
In this paper, we propose a new control framework called the moving endpoint
control to restore images corrupted by different degradation levels in one
model. The proposed control problem contains a restoration dynamics which is
modeled by an RNN. The moving endpoint, which is essentially the terminal time
of the associated dynamics, is determined by a policy network. We call the
proposed model the dynamically unfolding recurrent restorer (DURR). Numerical
experiments show that DURR is able to achieve state-of-the-art performances on
blind image denoising and JPEG image deblocking. Furthermore, DURR can well
generalize to images with higher degradation levels that are not included in
the training stage.Comment: The first two authors contributed equall
Loss Functions for Neural Networks for Image Processing
Neural networks are becoming central in several areas of computer vision and
image processing and different architectures have been proposed to solve
specific problems. The impact of the loss layer of neural networks, however,
has not received much attention in the context of image processing: the default
and virtually only choice is L2. In this paper, we bring attention to
alternative choices for image restoration. In particular, we show the
importance of perceptually-motivated losses when the resulting image is to be
evaluated by a human observer. We compare the performance of several losses,
and propose a novel, differentiable error function. We show that the quality of
the results improves significantly with better loss functions, even when the
network architecture is left unchanged.Comment: This paper was published in IEEE Transactions on Computational
Imaging on December 23, 201
- …