22 research outputs found
A Deep Residual Star Generative Adversarial Network for multi-domain Image Super-Resolution
Recently, most of state-of-the-art single image super-resolution (SISR)
methods have attained impressive performance by using deep convolutional neural
networks (DCNNs). The existing SR methods have limited performance due to a
fixed degradation settings, i.e. usually a bicubic downscaling of
low-resolution (LR) image. However, in real-world settings, the LR degradation
process is unknown which can be bicubic LR, bilinear LR, nearest-neighbor LR,
or real LR. Therefore, most SR methods are ineffective and inefficient in
handling more than one degradation settings within a single network. To handle
the multiple degradation, i.e. refers to multi-domain image super-resolution,
we propose a deep Super-Resolution Residual StarGAN (SR2*GAN), a novel and
scalable approach that super-resolves the LR images for the multiple LR domains
using only a single model. The proposed scheme is trained in a StarGAN like
network topology with a single generator and discriminator networks. We
demonstrate the effectiveness of our proposed approach in quantitative and
qualitative experiments compared to other state-of-the-art methods.Comment: 5 pages, 6th International Conference on Smart and Sustainable
Technologies 2021. arXiv admin note: text overlap with arXiv:2009.03693,
arXiv:2005.0095
Multi-Frequency-Aware Patch Adversarial Learning for Neural Point Cloud Rendering
We present a neural point cloud rendering pipeline through a novel
multi-frequency-aware patch adversarial learning framework. The proposed
approach aims to improve the rendering realness by minimizing the spectrum
discrepancy between real and synthesized images, especially on the
high-frequency localized sharpness information which causes image blur
visually. Specifically, a patch multi-discriminator scheme is proposed for the
adversarial learning, which combines both spectral domain (Fourier Transform
and Discrete Wavelet Transform) discriminators as well as the spatial (RGB)
domain discriminator to force the generator to capture global and local
spectral distributions of the real images. The proposed multi-discriminator
scheme not only helps to improve rendering realness, but also enhance the
convergence speed and stability of adversarial learning. Moreover, we introduce
a noise-resistant voxelisation approach by utilizing both the appearance
distance and spatial distance to exclude the spatial outlier points caused by
depth noise. Our entire architecture is fully differentiable and can be learned
in an end-to-end fashion. Extensive experiments show that our method produces
state-of-the-art results for neural point cloud rendering by a significant
margin. Our source code will be made public at a later date.Comment: 8 pages, 4 figure
Guided Frequency Loss for Image Restoration
Image Restoration has seen remarkable progress in recent years. Many
generative models have been adapted to tackle the known restoration cases of
images. However, the interest in benefiting from the frequency domain is not
well explored despite its major factor in these particular cases of image
synthesis. In this study, we propose the Guided Frequency Loss (GFL), which
helps the model to learn in a balanced way the image's frequency content
alongside the spatial content. It aggregates three major components that work
in parallel to enhance learning efficiency; a Charbonnier component, a
Laplacian Pyramid component, and a Gradual Frequency component. We tested GFL
on the Super Resolution and the Denoising tasks. We used three different
datasets and three different architectures for each of them. We found that the
GFL loss improved the PSNR metric in most implemented experiments. Also, it
improved the training of the Super Resolution models in both SwinIR and SRGAN.
In addition, the utility of the GFL loss increased better on constrained data
due to the less stochasticity in the high frequencies' components among
samples