2,000 research outputs found
"Zero-Shot" Super-Resolution using Deep Internal Learning
Deep Learning has led to a dramatic leap in Super-Resolution (SR) performance
in the past few years. However, being supervised, these SR methods are
restricted to specific training data, where the acquisition of the
low-resolution (LR) images from their high-resolution (HR) counterparts is
predetermined (e.g., bicubic downscaling), without any distracting artifacts
(e.g., sensor noise, image compression, non-ideal PSF, etc). Real LR images,
however, rarely obey these restrictions, resulting in poor SR results by SotA
(State of the Art) methods. In this paper we introduce "Zero-Shot" SR, which
exploits the power of Deep Learning, but does not rely on prior training. We
exploit the internal recurrence of information inside a single image, and train
a small image-specific CNN at test time, on examples extracted solely from the
input image itself. As such, it can adapt itself to different settings per
image. This allows to perform SR of real old photos, noisy images, biological
data, and other images where the acquisition process is unknown or non-ideal.
On such images, our method outperforms SotA CNN-based SR methods, as well as
previous unsupervised SR methods. To the best of our knowledge, this is the
first unsupervised CNN-based SR method
No-reference Image Denoising Quality Assessment
A wide variety of image denoising methods are available now. However, the
performance of a denoising algorithm often depends on individual input noisy
images as well as its parameter setting. In this paper, we present a
no-reference image denoising quality assessment method that can be used to
select for an input noisy image the right denoising algorithm with the optimal
parameter setting. This is a challenging task as no ground truth is available.
This paper presents a data-driven approach to learn to predict image denoising
quality. Our method is based on the observation that while individual existing
quality metrics and denoising models alone cannot robustly rank denoising
results, they often complement each other. We accordingly design denoising
quality features based on these existing metrics and models and then use Random
Forests Regression to aggregate them into a more powerful unified metric. Our
experiments on images with various types and levels of noise show that our
no-reference denoising quality assessment method significantly outperforms the
state-of-the-art quality metrics. This paper also provides a method that
leverages our quality assessment method to automatically tune the parameter
settings of a denoising algorithm for an input noisy image to produce an
optimal denoising result.Comment: 17 pages, 41 figures, accepted by Computer Vision Conference (CVC)
201
Neural Nearest Neighbors Networks
Non-local methods exploiting the self-similarity of natural signals have been
well studied, for example in image analysis and restoration. Existing
approaches, however, rely on k-nearest neighbors (KNN) matching in a fixed
feature space. The main hurdle in optimizing this feature space w.r.t.
application performance is the non-differentiability of the KNN selection rule.
To overcome this, we propose a continuous deterministic relaxation of KNN
selection that maintains differentiability w.r.t. pairwise distances, but
retains the original KNN as the limit of a temperature parameter approaching
zero. To exploit our relaxation, we propose the neural nearest neighbors block
(N3 block), a novel non-local processing layer that leverages the principle of
self-similarity and can be used as building block in modern neural network
architectures. We show its effectiveness for the set reasoning task of
correspondence classification as well as for image restoration, including image
denoising and single image super-resolution, where we outperform strong
convolutional neural network (CNN) baselines and recent non-local models that
rely on KNN selection in hand-chosen features spaces.Comment: to appear at NIPS*2018, code available at
https://github.com/visinf/n3net
Recurrent Attention Models for Depth-Based Person Identification
We present an attention-based model that reasons on human body shape and
motion dynamics to identify individuals in the absence of RGB information,
hence in the dark. Our approach leverages unique 4D spatio-temporal signatures
to address the identification problem across days. Formulated as a
reinforcement learning task, our model is based on a combination of
convolutional and recurrent neural networks with the goal of identifying small,
discriminative regions indicative of human identity. We demonstrate that our
model produces state-of-the-art results on several published datasets given
only depth images. We further study the robustness of our model towards
viewpoint, appearance, and volumetric changes. Finally, we share insights
gleaned from interpretable 2D, 3D, and 4D visualizations of our model's
spatio-temporal attention.Comment: Computer Vision and Pattern Recognition (CVPR) 201
MOGAN: Morphologic-structure-aware Generative Learning from a Single Image
In most interactive image generation tasks, given regions of interest (ROI)
by users, the generated results are expected to have adequate diversities in
appearance while maintaining correct and reasonable structures in original
images. Such tasks become more challenging if only limited data is available.
Recently proposed generative models complete training based on only one image.
They pay much attention to the monolithic feature of the sample while ignoring
the actual semantic information of different objects inside the sample. As a
result, for ROI-based generation tasks, they may produce inappropriate samples
with excessive randomicity and without maintaining the related objects' correct
structures. To address this issue, this work introduces a
MOrphologic-structure-aware Generative Adversarial Network named MOGAN that
produces random samples with diverse appearances and reliable structures based
on only one image. For training for ROI, we propose to utilize the data coming
from the original image being augmented and bring in a novel module to
transform such augmented data into knowledge containing both structures and
appearances, thus enhancing the model's comprehension of the sample. To learn
the rest areas other than ROI, we employ binary masks to ensure the generation
isolated from ROI. Finally, we set parallel and hierarchical branches of the
mentioned learning process. Compared with other single image GAN schemes, our
approach focuses on internal features including the maintenance of rational
structures and variation on appearance. Experiments confirm a better capacity
of our model on ROI-based image generation tasks than its competitive peers
- …