8 research outputs found
Learning Frequency Domain Priors for Image Demoireing.
Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin
Learning Image Demoireing from Unpaired Real Data
This paper focuses on addressing the issue of image demoireing. Unlike the
large volume of existing studies that rely on learning from paired real data,
we attempt to learn a demoireing model from unpaired real data, i.e., moire
images associated with irrelevant clean images. The proposed method, referred
to as Unpaired Demoireing (UnDeM), synthesizes pseudo moire images from
unpaired datasets, generating pairs with clean images for training demoireing
models. To achieve this, we divide real moire images into patches and group
them in compliance with their moire complexity. We introduce a novel moire
generation framework to synthesize moire images with diverse moire features,
resembling real moire patches, and details akin to real moire-free images.
Additionally, we introduce an adaptive denoise method to eliminate the
low-quality pseudo moire images that adversely impact the learning of
demoireing models. We conduct extensive experiments on the commonly-used FHDMi
and UHDM datasets. Results manifest that our UnDeM performs better than
existing methods when using existing demoireing models such as MBCNN and
ESDNet-L. Code: https://github.com/zysxmu/UnDeMComment: AAAI202
FPANet: Frequency-based Video Demoireing using Frame-level Post Alignment
Interference between overlapping gird patterns creates moire patterns,
degrading the visual quality of an image that captures a screen of a digital
display device by an ordinary digital camera. Removing such moire patterns is
challenging due to their complex patterns of diverse sizes and color
distortions. Existing approaches mainly focus on filtering out in the spatial
domain, failing to remove a large-scale moire pattern. In this paper, we
propose a novel model called FPANet that learns filters in both frequency and
spatial domains, improving the restoration quality by removing various sizes of
moire patterns. To further enhance, our model takes multiple consecutive
frames, learning to extract frame-invariant content features and outputting
better quality temporally consistent images. We demonstrate the effectiveness
of our proposed method with a publicly available large-scale dataset, observing
that ours outperforms the state-of-the-art approaches, including ESDNet,
VDmoire, MBCNN, WDNet, UNet, and DMCNN, in terms of the image and video quality
metrics, such as PSNR, SSIM, LPIPS, FVD, and FSIM
Improving Dynamic HDR Imaging with Fusion Transformer
Reconstructing a High Dynamic Range (HDR) image from several Low Dynamic Range (LDR) images with different exposures is a challenging task, especially in the presence of camera and object motion. Though existing models using convolutional neural networks (CNNs) have made great progress, challenges still exist, e.g., ghosting artifacts. Transformers, originating from the field of natural language processing, have shown success in computer vision tasks, due to their ability to address a large receptive field even within a single layer. In this paper, we propose a transformer model for HDR imaging. Our pipeline includes three steps: alignment, fusion, and reconstruction. The key component is the HDR transformer module. Through experiments and ablation studies, we demonstrate that our model outperforms the state-of-the-art by large margins on several popular public datasets