3,272 research outputs found
Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling
High-resolution (HR) images are usually downscaled to low-resolution (LR)
ones for better display and afterward upscaled back to the original size to
recover details. Recent work in image rescaling formulates downscaling and
upscaling as a unified task and learns a bijective mapping between HR and LR
via invertible networks. However, in real-world applications (e.g., social
media), most images are compressed for transmission. Lossy compression will
lead to irreversible information loss on LR images, hence damaging the inverse
upscaling procedure and degrading the reconstruction accuracy. In this paper,
we propose the Self-Asymmetric Invertible Network (SAIN) for compression-aware
image rescaling. To tackle the distribution shift, we first develop an
end-to-end asymmetric framework with two separate bijective mappings for
high-quality and compressed LR images, respectively. Then, based on empirical
analysis of this framework, we model the distribution of the lost information
(including downscaling and compression) using isotropic Gaussian mixtures and
propose the Enhanced Invertible Block to derive high-quality/compressed LR
images in one forward pass. Besides, we design a set of losses to regularize
the learned LR images and enhance the invertibility. Extensive experiments
demonstrate the consistent improvements of SAIN across various image rescaling
datasets in terms of both quantitative and qualitative evaluation under
standard image compression formats (i.e., JPEG and WebP).Comment: Accepted by AAAI 2023. Code is available at
https://github.com/yang-jin-hai/SAI
Downscaled Representation Matters: Improving Image Rescaling with Collaborative Downscaled Images
Deep networks have achieved great success in image rescaling (IR) task that
seeks to learn the optimal downscaled representations, i.e., low-resolution
(LR) images, to reconstruct the original high-resolution (HR) images. Compared
with super-resolution methods that consider a fixed downscaling scheme, e.g.,
bicubic, IR often achieves significantly better reconstruction performance
thanks to the learned downscaled representations. This highlights the
importance of a good downscaled representation in image reconstruction tasks.
Existing IR methods mainly learn the downscaled representation by jointly
optimizing the downscaling and upscaling models. Unlike them, we seek to
improve the downscaled representation through a different and more direct way:
optimizing the downscaled image itself instead of the down-/upscaling models.
Specifically, we propose a collaborative downscaling scheme that directly
generates the collaborative LR examples by descending the gradient w.r.t. the
reconstruction loss on them to benefit the IR process. Furthermore, since LR
images are downscaled from the corresponding HR images, one can also improve
the downscaled representation if we have a better representation in the HR
domain. Inspired by this, we propose a Hierarchical Collaborative Downscaling
(HCD) method that performs gradient descent in both HR and LR domains to
improve the downscaled representations. Extensive experiments show that our HCD
significantly improves the reconstruction performance both quantitatively and
qualitatively. Moreover, we also highlight the flexibility of our HCD since it
can generalize well across diverse IR models.Comment: 11 pages, 8 figure
DynaVSR: Dynamic Adaptive Blind Video Super-Resolution
Most conventional supervised super-resolution (SR) algorithms assume that
low-resolution (LR) data is obtained by downscaling high-resolution (HR) data
with a fixed known kernel, but such an assumption often does not hold in real
scenarios. Some recent blind SR algorithms have been proposed to estimate
different downscaling kernels for each input LR image. However, they suffer
from heavy computational overhead, making them infeasible for direct
application to videos. In this work, we present DynaVSR, a novel
meta-learning-based framework for real-world video SR that enables efficient
downscaling model estimation and adaptation to the current input. Specifically,
we train a multi-frame downscaling module with various types of synthetic blur
kernels, which is seamlessly combined with a video SR network for input-aware
adaptation. Experimental results show that DynaVSR consistently improves the
performance of the state-of-the-art video SR models by a large margin, with an
order of magnitude faster inference time compared to the existing blind SR
approaches
Nonlocal Co-occurrence for Image Downscaling
Image downscaling is one of the widely used operations in image processing
and computer graphics. It was recently demonstrated in the literature that
kernel-based convolutional filters could be modified to develop efficient image
downscaling algorithms. In this work, we present a new downscaling technique
which is based on kernel-based image filtering concept. We propose to use
pairwise co-occurrence similarity of the pixelpairs as the range kernel
similarity in the filtering operation. The co-occurrence of the pixel-pair is
learned directly from the input image. This co-occurrence learning is performed
in a neighborhood based fashion all over the image. The proposed method can
preserve the high-frequency structures, which were present in the input image,
into the downscaled image. The resulting images retain visually important
details and do not suffer from edge-blurring artifact. We demonstrate the
effectiveness of our proposed approach with extensive experiments on a large
number of images downscaled with various downscaling factors.Comment: 9 pages, 8 figure
- …