24 research outputs found
Lightweight Image Super-Resolution with Information Multi-distillation Network
In recent years, single image super-resolution (SISR) methods using deep
convolution neural network (CNN) have achieved impressive results. Thanks to
the powerful representation capabilities of the deep networks, numerous
previous ways can learn the complex non-linear mapping between low-resolution
(LR) image patches and their high-resolution (HR) versions. However, excessive
convolutions will limit the application of super-resolution technology in low
computing power devices. Besides, super-resolution of any arbitrary scale
factor is a critical issue in practical applications, which has not been well
solved in the previous approaches. To address these issues, we propose a
lightweight information multi-distillation network (IMDN) by constructing the
cascaded information multi-distillation blocks (IMDB), which contains
distillation and selective fusion parts. Specifically, the distillation module
extracts hierarchical features step-by-step, and fusion module aggregates them
according to the importance of candidate features, which is evaluated by the
proposed contrast-aware channel attention mechanism. To process real images
with any sizes, we develop an adaptive cropping strategy (ACS) to super-resolve
block-wise image patches using the same well-trained model. Extensive
experiments suggest that the proposed method performs favorably against the
state-of-the-art SR algorithms in term of visual quality, memory footprint, and
inference time. Code is available at \url{https://github.com/Zheng222/IMDN}.Comment: To be appear in ACM Multimedia 2019, https://github.com/Zheng222/IMD
Enhanced collapsible linear blocks for arbitrary sized image super-resolution
Image up-scaling and super-resolution (SR) techniques have been a hot research topic for many years due to its large impact in the field of medical imaging, surveillance etc. Especially single image super-resolution (SISR) become very popular because of the fast development of deep convolution neural network (DCNN) and the low requirement on the input. They are achieving outstanding performance. However, there are still problems in the state-of-the-art works, especially from two perspectives: 1. failed at exploiting the hierarchical characteristics from the input, resulting in loss of information and artifacts in the final high resolution (HR) image; 2. failed to handle arbitrary-sized images; the existing research works are focused on fixed size input images. To address these challenges, this paper proposed a residual dense network (RDN) and multi-scale sub-pixel convolution network (MSSPCN) which are integrated into a Collapsible Linear Block Super Efficient Super-Resolution (SESR) network. The RDNs aims to tackle the first challenge, carrying the hierarchical features from end-to-end. An adaptive cropping strategy (ACS) technique is introduced before feature extraction targeting at the image size challenge. The novelty of this work is extracting the hierarchical features and integrating RDNs with MSSPCNs. The proposed network can upscale any arbitrary-sized image (1080p) to ×2 (4K) and ×4 (8K). To secure ground truth for evaluation, this paper follows the opposite flow, generating the input LR images by down-sampling the given HR images (ground truth). To evaluate the performance, the proposed algorithm is compared with eight state-of-the-art algorithms, both quantitatively and qualitatively. The results are verified on six benchmark datasets. The extensive experiments justify that the proposed architecture performs better than other methods and upscales the images satisfactorily
Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution
Convolutional neural network (CNN) has achieved great success on image
super-resolution (SR). However, most deep CNN-based SR models take massive
computations to obtain high performance. Downsampling features for
multi-resolution fusion is an efficient and effective way to improve the
performance of visual recognition. Still, it is counter-intuitive in the SR
task, which needs to project a low-resolution input to high-resolution. In this
paper, we propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing
an efficient and effective downsampling module into the SR task. The network
contains pixel-unshuffled downsampling and Self-Residual Depthwise Separable
Convolutions. Specifically, we utilize pixel-unshuffle operation to downsample
the input features and use grouped convolution to reduce the channels. Besides,
we enhance the depthwise convolution's performance by adding the input feature
to its output. Experiments on benchmark datasets show that our HPUN achieves
and surpasses the state-of-the-art reconstruction performance with fewer
parameters and computation costs
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
Compression plays an important role on the efficient transmission and storage
of images and videos through band-limited systems such as streaming services,
virtual reality or videogames. However, compression unavoidably leads to
artifacts and the loss of the original information, which may severely degrade
the visual quality. For these reasons, quality enhancement of compressed images
has become a popular research topic. While most state-of-the-art image
restoration methods are based on convolutional neural networks, other
transformers-based methods such as SwinIR, show impressive performance on these
tasks.
In this paper, we explore the novel Swin Transformer V2, to improve SwinIR
for image super-resolution, and in particular, the compressed input scenario.
Using this method we can tackle the major issues in training transformer vision
models, such as training instability, resolution gaps between pre-training and
fine-tuning, and hunger on data. We conduct experiments on three representative
tasks: JPEG compression artifacts removal, image super-resolution (classical
and lightweight), and compressed image super-resolution. Experimental results
demonstrate that our method, Swin2SR, can improve the training convergence and
performance of SwinIR, and is a top-5 solution at the "AIM 2022 Challenge on
Super-Resolution of Compressed Image and Video".Comment: European Conference on Computer Vision (ECCV 2022) Workshop