276 research outputs found
Event-guided Multi-patch Network with Self-supervision for Non-uniform Motion Deblurring
Contemporary deep learning multi-scale deblurring models suffer from many
issues: 1) They perform poorly on non-uniformly blurred images/videos; 2)
Simply increasing the model depth with finer-scale levels cannot improve
deblurring; 3) Individual RGB frames contain a limited motion information for
deblurring; 4) Previous models have a limited robustness to spatial
transformations and noise. Below, we extend the DMPHN model by several
mechanisms to address the above issues: I) We present a novel self-supervised
event-guided deep hierarchical Multi-patch Network (MPN) to deal with blurry
images and videos via fine-to-coarse hierarchical localized representations;
II) We propose a novel stacked pipeline, StackMPN, to improve the deblurring
performance under the increased network depth; III) We propose an event-guided
architecture to exploit motion cues contained in videos to tackle complex blur
in videos; IV) We propose a novel self-supervised step to expose the model to
random transformations (rotations, scale changes), and make it robust to
Gaussian noises. Our MPN achieves the state of the art on the GoPro and
VideoDeblur datasets with a 40x faster runtime compared to current multi-scale
methods. With 30ms to process an image at 1280x720 resolution, it is the first
real-time deep motion deblurring model for 720p images at 30fps. For StackMPN,
we obtain significant improvements over 1.2dB on the GoPro dataset by
increasing the network depth. Utilizing the event information and
self-supervision further boost results to 33.83dB.Comment: International Journal of Computer Vision. arXiv admin note:
substantial text overlap with arXiv:1904.0346
Fast Deep Multi-patch Hierarchical Network for Nonhomogeneous Image Dehazing
Recently, CNN based end-to-end deep learning methods achieve superiority in
Image Dehazing but they tend to fail drastically in Non-homogeneous dehazing.
Apart from that, existing popular Multi-scale approaches are runtime intensive
and memory inefficient. In this context, we proposed a fast Deep Multi-patch
Hierarchical Network to restore Non-homogeneous hazed images by aggregating
features from multiple image patches from different spatial sections of the
hazed image with fewer number of network parameters. Our proposed method is
quite robust for different environments with various density of the haze or fog
in the scene and very lightweight as the total size of the model is around 21.7
MB. It also provides faster runtime compared to current multi-scale methods
with an average runtime of 0.0145s to process 1200x1600 HD quality image.
Finally, we show the superiority of this network on Dense Haze Removal to other
state-of-the-art models.Comment: CVPR Workshops Proceedings 202
Take a Prior from Other Tasks for Severe Blur Removal
Recovering clear structures from severely blurry inputs is a challenging
problem due to the large movements between the camera and the scene. Although
some works apply segmentation maps on human face images for deblurring, they
cannot handle natural scenes because objects and degradation are more complex,
and inaccurate segmentation maps lead to a loss of details. For general scene
deblurring, the feature space of the blurry image and corresponding sharp image
under the high-level vision task is closer, which inspires us to rely on other
tasks (e.g. classification) to learn a comprehensive prior in severe blur
removal cases. We propose a cross-level feature learning strategy based on
knowledge distillation to learn the priors, which include global contexts and
sharp local structures for recovering potential details. In addition, we
propose a semantic prior embedding layer with multi-level aggregation and
semantic attention transformation to integrate the priors effectively. We
introduce the proposed priors to various models, including the UNet and other
mainstream deblurring baselines, leading to better performance on severe blur
removal. Extensive experiments on natural image deblurring benchmarks and
real-world images, such as GoPro and RealBlur datasets, demonstrate our
method's effectiveness and generalization ability
BANet: Blur-aware Attention Networks for Dynamic Scene Deblurring
Image motion blur usually results from moving objects or camera shakes. Such
blur is generally directional and non-uniform. Previous research efforts
attempt to solve non-uniform blur by using self-recurrent multi-scale or
multi-patch architectures accompanying with self-attention. However, using
self-recurrent frameworks typically leads to a longer inference time, while
inter-pixel or inter-channel self-attention may cause excessive memory usage.
This paper proposes blur-aware attention networks (BANet) that accomplish
accurate and efficient deblurring via a single forward pass. Our BANet utilizes
region-based self-attention with multi-kernel strip pooling to disentangle blur
patterns of different degrees and with cascaded parallel dilated convolution to
aggregate multi-scale content features. Extensive experimental results on the
GoPro and HIDE benchmarks demonstrate that the proposed BANet performs
favorably against the state-of-the-art in blurred image restoration and can
provide deblurred results in real-time
MC-Blur: A Comprehensive Benchmark for Image Deblurring
Blur artifacts can seriously degrade the visual quality of images, and
numerous deblurring methods have been proposed for specific scenarios. However,
in most real-world images, blur is caused by different factors, e.g., motion
and defocus. In this paper, we address how different deblurring methods perform
in the case of multiple types of blur. For in-depth performance evaluation, we
construct a new large-scale multi-cause image deblurring dataset (called
MC-Blur), including real-world and synthesized blurry images with mixed factors
of blurs. The images in the proposed MC-Blur dataset are collected using
different techniques: averaging sharp images captured by a 1000-fps high-speed
camera, convolving Ultra-High-Definition (UHD) sharp images with large-size
kernels, adding defocus to images, and real-world blurry images captured by
various camera models. Based on the MC-Blur dataset, we conduct extensive
benchmarking studies to compare SOTA methods in different scenarios, analyze
their efficiency, and investigate the built dataset's capacity. These
benchmarking results provide a comprehensive overview of the advantages and
limitations of current deblurring methods, and reveal the advances of our
dataset
- …