64,652 research outputs found
Simple, Accurate, and Robust Nonparametric Blind Super-Resolution
This paper proposes a simple, accurate, and robust approach to single image
nonparametric blind Super-Resolution (SR). This task is formulated as a
functional to be minimized with respect to both an intermediate super-resolved
image and a nonparametric blur-kernel. The proposed approach includes a
convolution consistency constraint which uses a non-blind learning-based SR
result to better guide the estimation process. Another key component is the
unnatural bi-l0-l2-norm regularization imposed on the super-resolved, sharp
image and the blur-kernel, which is shown to be quite beneficial for estimating
the blur-kernel accurately. The numerical optimization is implemented by
coupling the splitting augmented Lagrangian and the conjugate gradient (CG).
Using the pre-estimated blur-kernel, we finally reconstruct the SR image by a
very simple non-blind SR method that uses a natural image prior. The proposed
approach is demonstrated to achieve better performance than the recent method
by Michaeli and Irani [2] in both terms of the kernel estimation accuracy and
image SR quality
Learning from Multi-Perception Features for Real-Word Image Super-resolution
Currently, there are two popular approaches for addressing real-world image
super-resolution problems: degradation-estimation-based and blind-based
methods. However, degradation-estimation-based methods may be inaccurate in
estimating the degradation, making them less applicable to real-world LR
images. On the other hand, blind-based methods are often limited by their fixed
single perception information, which hinders their ability to handle diverse
perceptual characteristics. To overcome this limitation, we propose a novel SR
method called MPF-Net that leverages multiple perceptual features of input
images. Our method incorporates a Multi-Perception Feature Extraction (MPFE)
module to extract diverse perceptual information and a series of newly-designed
Cross-Perception Blocks (CPB) to combine this information for effective
super-resolution reconstruction. Additionally, we introduce a contrastive
regularization term (CR) that improves the model's learning capability by using
newly generated HR and LR images as positive and negative samples for ground
truth HR. Experimental results on challenging real-world SR datasets
demonstrate that our approach significantly outperforms existing
state-of-the-art methods in both qualitative and quantitative measures
Advanced Restoration Techniques for Images and Disparity Maps
With increasing popularity of digital cameras, the field of Computa-
tional Photography emerges as one of the most demanding areas of
research. In this thesis we study and develop novel priors and op-
timization techniques to solve inverse problems, including disparity
estimation and image restoration.
The disparity map estimation method proposed in this thesis incor-
porates multiple frames of a stereo video sequence to ensure temporal
coherency. To enforce smoothness, we use spatio-temporal connec-
tions between the pixels of the disparity map to constrain our solution.
Apart from smoothness, we enforce a consistency constraint for the
disparity assignments by using connections between the left and right
views. These constraints are then formulated in a graphical model,
which we solve using mean-field approximation. We use a filter-based
mean-field optimization that perform efficiently by updating the dis-
parity variables in parallel. The parallel updates scheme, however, is
not guaranteed to converge to a stationary point. To compare and
demonstrate the effectiveness of our approach, we developed a new
optimization technique that uses sequential updates, which runs ef-
ficiently and guarantees convergence. Our empirical results indicate
that with proper initialization, we can employ the parallel update
scheme and efficiently optimize our disparity maps without loss of
quality. Our method ranks amongst the state of the art in common
benchmarks, and significantly reduces the temporal flickering artifacts
in the disparity maps.
In the second part of this thesis, we address several image restora-
tion problems such as image deblurring, demosaicing and super-
resolution. We propose to use denoising autoencoders to learn an
approximation of the true natural image distribution. We parametrize
our denoisers using deep neural networks and show that they learn
the gradient of the smoothed density of natural images. Based on
this analysis, we propose a restoration technique that moves the so-
lution towards the local extrema of this distribution by minimizing
the difference between the input and output of our denoiser. Weii
demonstrate the effectiveness of our approach using a single trained
neural network in several restoration tasks such as deblurring and
super-resolution. In a more general framework, we define a new
Bayes formulation for the restoration problem, which leads to a more
efficient and robust estimator. The proposed framework achieves state
of the art performance in various restoration tasks such as deblurring
and demosaicing, and also for more challenging tasks such as noise-
and kernel-blind image deblurring.
Keywords. disparity map estimation, stereo matching, mean-field
optimization, graphical models, image processing, linear inverse prob-
lems, image restoration, image deblurring, image denoising, single
image super-resolution, image demosaicing, deep neural networks,
denoising autoencoder
Camera-independent learning and image quality assessment for super-resolution
An increasing number of applications require high-resolution images in situations where the access to the sensor and the knowledge of its specifications are limited. In this thesis, the problem of blind super-resolution is addressed, here defined as the estimation of a high-resolution image from one or more low-resolution inputs, under the condition that the degradation model parameters are unknown. The assessment of super-resolved results, using objective measures of image quality, is also addressed.Learning-based methods have been successfully applied to the single frame super-resolution problem in the past. However, sensor characteristics such as the Point Spread Function (PSF) must often be known. In this thesis, a learning-based approach is adapted to work without the knowledge of the PSF thus making the framework camera-independent. However, the goal is not only to super-resolve an image under this limitation, but also to provide an estimation of the best PSF, consisting of a theoretical model with one unknown parameter.In particular, two extensions of a method performing belief propagation on a Markov Random Field are presented. The first method finds the best PSF parameter by performing a search for the minimum mean distance between training examples and patches from the input image. In the second method, the best PSF parameter and the super-resolution result are found simultaneously by providing a range of possible PSF parameters from which the super-resolution algorithm will choose from. For both methods, a first estimate is obtained through blind deconvolution and an uncertainty is calculated in order to restrict the search.Both camera-independent adaptations are compared and analyzed in various experiments, and a set of key parameters are varied to determine their effect on both the super-resolution and the PSF parameter recovery results. The use of quality measures is thus essential to quantify the improvements obtained from the algorithms. A set of measures is chosen that represents different aspects of image quality: the signal fidelity, the perceptual quality and the localization and scale of the edges.Results indicate that both methods improve similarity to the ground truth and can in general refine the initial PSF parameter estimate towards the true value. Furthermore, the similarity measure results show that the chosen learning-based framework consistently improves a measure designed for perceptual quality
Real-World Image Restoration Using Degradation Adaptive Transformer-Based Adversarial Network
Most existing learning-based image restoration methods heavily rely on paired degraded/non-degraded training datasets that are based on simplistic handcrafted degradation assumptions. These assumptions often involve a limited set of degradations, such as Gaussian blurs, noises, and bicubic downsampling. However, when these methods are applied to real-world images, there is a significant decrease in performance due to the discrepancy between synthetic and realistic degradation. Additionally, they lack the flexibility to adapt to unknown degradations in practical scenarios, which limits their generalizability to complex and unconstrained scenes.
To address the absence of image pairs, recent studies have proposed Generative Adversarial Network (GAN)-based unpaired methods. Nevertheless, unpaired learning models based on convolution operations encounter challenges in capturing long-range pixel dependencies in real-world images. This limitation stems from their reliance on convolution operations, which offer local connectivity and translation equivariance but struggle to capture global dependencies due to their limited receptive field.
To address these challenges, this dissertation proposed an innovative unpaired image restoration basic model along with an advanced model. The proposed basic model is the DA-CycleGAN model, which is based on the CycleGAN [1] neural network and specifically designed for blind real-world Single Image Super-Resolution (SISR). The DA-CycleGAN incorporates a degradation adaptive (DA) module to learn various real-world degradations (such as noise and blur patterns) in an unpaired manner, enabling strong flexible adaptation. Additionally, an advanced model called Trans-CycleGAN was designed, which integrated the Transformer architecture into CycleGAN to leverage its global connectivity. This combination allowed for image-to-image translation using CycleGAN [1] while enabling the Transformer to model global connectivity across long-range pixels. Extensive experiments conducted on realistic images demonstrate the superior performance of the proposed method in solving real-world image restoration problems, resulting in clearer and finer details.
Overall, this dissertation presents a novel unpaired image restoration basic model and an advanced model that effectively address the limitations of existing approaches. The proposed approach achieves significant advancements in handling real-world degradations and modeling long-range pixel dependencies, thereby offering substantial improvements in image restoration tasks.
Index Terms— Cross-domain translation, generative adversarial network, image restoration, super-resolution, transformer, unpaired training
Deep Mean-Shift Priors for Image Restoration
In this paper we introduce a natural image prior that directly represents a
Gaussian-smoothed version of the natural image distribution. We include our
prior in a formulation of image restoration as a Bayes estimator that also
allows us to solve noise-blind image restoration problems. We show that the
gradient of our prior corresponds to the mean-shift vector on the natural image
distribution. In addition, we learn the mean-shift vector field using denoising
autoencoders, and use it in a gradient descent approach to perform Bayes risk
minimization. We demonstrate competitive results for noise-blind deblurring,
super-resolution, and demosaicing.Comment: NIPS 201
- …