7 research outputs found
DynaVSR: Dynamic Adaptive Blind Video Super-Resolution
Most conventional supervised super-resolution (SR) algorithms assume that
low-resolution (LR) data is obtained by downscaling high-resolution (HR) data
with a fixed known kernel, but such an assumption often does not hold in real
scenarios. Some recent blind SR algorithms have been proposed to estimate
different downscaling kernels for each input LR image. However, they suffer
from heavy computational overhead, making them infeasible for direct
application to videos. In this work, we present DynaVSR, a novel
meta-learning-based framework for real-world video SR that enables efficient
downscaling model estimation and adaptation to the current input. Specifically,
we train a multi-frame downscaling module with various types of synthetic blur
kernels, which is seamlessly combined with a video SR network for input-aware
adaptation. Experimental results show that DynaVSR consistently improves the
performance of the state-of-the-art video SR models by a large margin, with an
order of magnitude faster inference time compared to the existing blind SR
approaches
Bilevel Fast Scene Adaptation for Low-Light Image Enhancement
Enhancing images in low-light scenes is a challenging but widely concerned
task in the computer vision. The mainstream learning-based methods mainly
acquire the enhanced model by learning the data distribution from the specific
scenes, causing poor adaptability (even failure) when meeting real-world
scenarios that have never been encountered before. The main obstacle lies in
the modeling conundrum from distribution discrepancy across different scenes.
To remedy this, we first explore relationships between diverse low-light scenes
based on statistical analysis, i.e., the network parameters of the encoder
trained in different data distributions are close. We introduce the bilevel
paradigm to model the above latent correspondence from the perspective of
hyperparameter optimization. A bilevel learning framework is constructed to
endow the scene-irrelevant generality of the encoder towards diverse scenes
(i.e., freezing the encoder in the adaptation and testing phases). Further, we
define a reinforced bilevel learning framework to provide a meta-initialization
for scene-specific decoder to further ameliorate visual quality. Moreover, to
improve the practicability, we establish a Retinex-induced architecture with
adaptive denoising and apply our built learning framework to acquire its
parameters by using two training losses including supervised and unsupervised
forms. Extensive experimental evaluations on multiple datasets verify our
adaptability and competitive performance against existing state-of-the-art
works. The code and datasets will be available at
https://github.com/vis-opt-group/BL
Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction
Image super-resolution (SR) aims to learn a mapping from low-resolution (LR)
to high-resolution (HR) using paired HR-LR training images. Conventional SR
methods typically gather the paired training data by synthesizing LR images
from HR images using a predetermined degradation model, e.g., Bicubic
down-sampling. However, the realistic degradation type of test images may
mismatch with the training-time degradation type due to the dynamic changes of
the real-world scenarios, resulting in inferior-quality SR images. To address
this, existing methods attempt to estimate the degradation model and train an
image-specific model, which, however, is quite time-consuming and impracticable
to handle rapidly changing domain shifts. Moreover, these methods largely
concentrate on the estimation of one degradation type (e.g., blur degradation),
overlooking other degradation types like noise and JPEG in real-world test-time
scenarios, thus limiting their practicality. To tackle these problems, we
present an efficient test-time adaptation framework for SR, named SRTTA, which
is able to quickly adapt SR models to test domains with different/unknown
degradation types. Specifically, we design a second-order degradation scheme to
construct paired data based on the degradation type of the test image, which is
predicted by a pre-trained degradation classifier. Then, we adapt the SR model
by implementing feature-level reconstruction learning from the initial test
image to its second-order degraded counterparts, which helps the SR model
generate plausible HR images. Extensive experiments are conducted on newly
synthesized corrupted DIV2K datasets with 8 different degradations and several
real-world datasets, demonstrating that our SRTTA framework achieves an
impressive improvement over existing methods with satisfying speed. The source
code is available at https://github.com/DengZeshuai/SRTTA.Comment: Accepted by 37th Conference on Neural Information Processing Systems
(NeurIPS 2023