10,185 research outputs found
Bayesian Image Super-Resolution with Deep Modeling of Image Statistics
Modeling statistics of image priors is useful for image super-resolution, but
little attention has been paid from the massive works of deep learning-based
methods. In this work, we propose a Bayesian image restoration framework, where
natural image statistics are modeled with the combination of smoothness and
sparsity priors. Concretely, firstly we consider an ideal image as the sum of a
smoothness component and a sparsity residual, and model real image degradation
including blurring, downscaling, and noise corruption. Then, we develop a
variational Bayesian approach to infer their posteriors. Finally, we implement
the variational approach for single image super-resolution (SISR) using deep
neural networks, and propose an unsupervised training strategy. The experiments
on three image restoration tasks, \textit{i.e.,} ideal SISR, realistic SISR,
and real-world SISR, demonstrate that our method has superior model
generalizability against varying noise levels and degradation kernels and is
effective in unsupervised SISR. The code and resulting models are released via
\url{https://zmiclab.github.io/projects.html}.Comment: 45 page
Bayesian Convolutional Neural Networks for Compressed Sensing Restoration
Deep Neural Networks (DNNs) have aroused great attention in Compressed
Sensing (CS) restoration. However, the working mechanism of DNNs is not
explainable, thereby it is unclear that how to design an optimal DNNs for CS
restoration. In this paper, we propose a novel statistical framework to explain
DNNs, which proves that the hidden layers of DNNs are equivalent to Gibbs
distributions and interprets DNNs as a Bayesian hierarchical model. The
framework provides a Bayesian perspective to explain the working mechanism of
DNNs, namely some hidden layers learn a prior distribution and other layers
learn a likelihood distribution. Moreover, the framework provides insights into
DNNs and reveals two inherent limitations of DNNs for CS restoration. In
contrast to most previous works designing an end-to-end DNNs for CS
restoration, we propose a novel DNNs to model a prior distribution only, which
can circumvent the limitations of DNNs. Given the prior distribution generated
from the DNNs, we design a Bayesian inference algorithm to realize CS
restoration in the framework of Bayesian Compressed Sensing. Finally, extensive
simulations validate the proposed theory of DNNs and demonstrate that the
proposed algorithm outperforms the state-of-the-art CS restoration methods
Deep Regression Bayesian Network and Its Applications
Deep directed generative models have attracted much attention recently due to
their generative modeling nature and powerful data representation ability. In
this paper, we review different structures of deep directed generative models
and the learning and inference algorithms associated with the structures. We
focus on a specific structure that consists of layers of Bayesian Networks due
to the property of capturing inherent and rich dependencies among latent
variables. The major difficulty of learning and inference with deep directed
models with many latent variables is the intractable inference due to the
dependencies among the latent variables and the exponential number of latent
variable configurations. Current solutions use variational methods often
through an auxiliary network to approximate the posterior probability
inference. In contrast, inference can also be performed directly without using
any auxiliary network to maximally preserve the dependencies among the latent
variables. Specifically, by exploiting the sparse representation with the
latent space, max-max instead of max-sum operation can be used to overcome the
exponential number of latent configurations. Furthermore, the max-max operation
and augmented coordinate ascent are applied to both supervised and unsupervised
learning as well as to various inference. Quantitative evaluations on benchmark
datasets of different models are given for both data representation and feature
learning tasks.Comment: Accepted to IEEE Signal Processing Magazin
Image Restoration from Parametric Transformations using Generative Models
When images are statistically described by a generative model we can use this
information to develop optimum techniques for various image restoration
problems as inpainting, super-resolution, image coloring, generative model
inversion, etc. With the help of the generative model it is possible to
formulate, in a natural way, these restoration problems as Statistical
estimation problems. Our approach, by combining maximum a-posteriori
probability with maximum likelihood estimation, is capable of restoring images
that are distorted by transformations even when the latter contain unknown
parameters. The resulting optimization is completely defined with no parameters
requiring tuning. This must be compared with the current state of the art which
requires exact knowledge of the transformations and contains regularizer terms
with weights that must be properly defined. Finally, we must mention that we
extend our method to accommodate mixtures of multiple images where each image
is described by its own generative model and we are able of successfully
separating each participating image from a single mixture
Unrolled Optimization with Deep Priors
A broad class of problems at the core of computational imaging, sensing, and
low-level computer vision reduces to the inverse problem of extracting latent
images that follow a prior distribution, from measurements taken under a known
physical image formation model. Traditionally, hand-crafted priors along with
iterative optimization methods have been used to solve such problems. In this
paper we present unrolled optimization with deep priors, a principled framework
for infusing knowledge of the image formation into deep networks that solve
inverse problems in imaging, inspired by classical iterative methods. We show
that instances of the framework outperform the state-of-the-art by a
substantial margin for a wide variety of imaging problems, such as denoising,
deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover,
we conduct experiments that explain how the framework is best used and why it
outperforms previous methods.Comment: First two authors contributed equall
The Nishimori line and Bayesian Statistics
``Nishimori line'' is a line or hypersurface in the parameter space of
systems with quenched disorder, where simple expressions of the averages of
physical quantities over the quenched random variables are obtained. It has
been playing an important role in the theoretical studies of the random
frustrated systems since its discovery around 1980. In this paper, a novel
interpretation of the Nishimori line from the viewpoint of statistical
information processing is presented. Our main aim is the reconstruction of the
whole theory of the Nishimori line from the viewpoint of Bayesian statistics,
or, almost equivalently, from the viewpoint of the theory of error-correcting
codes. As a byproduct of our interpretation, counterparts of the Nishimori line
in models without gauge invariance are given. We also discussed the issues on
the ``finite temperature decoding'' of error-correcting codes in connection
with our theme and clarify the role of gauge invariance in this topic.Comment: 16 pages, 1 table, no figures, using Iopart.cls and Iopart10.clo,
submitted to Journal of Physics A (Mathematical and General), this cond-mat
version contains full titles of the reference
Deep Gaussian Conditional Random Field Network: A Model-based Deep Network for Discriminative Denoising
We propose a novel deep network architecture for image\\ denoising based on a
Gaussian Conditional Random Field (GCRF) model. In contrast to the existing
discriminative denoising methods that train a separate model for each noise
level, the proposed deep network explicitly models the input noise variance and
hence is capable of handling a range of noise levels. Our deep network, which
we refer to as deep GCRF network, consists of two sub-networks: (i) a parameter
generation network that generates the pairwise potential parameters based on
the noisy input image, and (ii) an inference network whose layers perform the
computations involved in an iterative GCRF inference procedure.\ We train the
entire deep GCRF network (both parameter generation and inference networks)
discriminatively in an end-to-end fashion by maximizing the peak
signal-to-noise ratio measure. Experiments on Berkeley segmentation and
PASCALVOC datasets show that the proposed deep GCRF network outperforms
state-of-the-art image denoising approaches for several noise levels.Comment: 10 pages, 5 figure
Non-Local Video Denoising by CNN
Non-local patch based methods were until recently state-of-the-art for image
denoising but are now outperformed by CNNs. Yet they are still the
state-of-the-art for video denoising, as video redundancy is a key factor to
attain high denoising performance. The problem is that CNN architectures are
hardly compatible with the search for self-similarities. In this work we
propose a new and efficient way to feed video self-similarities to a CNN. The
non-locality is incorporated into the network via a first non-trainable layer
which finds for each patch in the input image its most similar patches in a
search region. The central values of these patches are then gathered in a
feature vector which is assigned to each image pixel. This information is
presented to a CNN which is trained to predict the clean image. We apply the
proposed architecture to image and video denoising. For the latter patches are
searched for in a 3D spatio-temporal volume. The proposed architecture achieves
state-of-the-art results. To the best of our knowledge, this is the first
successful application of a CNN to video denoising.Comment: A shorter version of this work has been accepted at ICIP 2019 (A
NON-LOCAL CNN FOR VIDEO DENOISING). The results of v2 were improved compared
to v1 and the code was updated accordingly. Code is available at:
https://github.com/axeldavy/vnlne
Toward Convolutional Blind Denoising of Real Photographs
While deep convolutional neural networks (CNNs) have achieved impressive
success in image denoising with additive white Gaussian noise (AWGN), their
performance remains limited on real-world noisy photographs. The main reason is
that their learned models are easy to overfit on the simplified AWGN model
which deviates severely from the complicated real-world noise model. In order
to improve the generalization ability of deep CNN denoisers, we suggest
training a convolutional blind denoising network (CBDNet) with more realistic
noise model and real-world noisy-clean image pairs. On the one hand, both
signal-dependent noise and in-camera signal processing pipeline is considered
to synthesize realistic noisy images. On the other hand, real-world noisy
photographs and their nearly noise-free counterparts are also included to train
our CBDNet. To further provide an interactive strategy to rectify denoising
result conveniently, a noise estimation subnetwork with asymmetric learning to
suppress under-estimation of noise level is embedded into CBDNet. Extensive
experimental results on three datasets of real-world noisy photographs clearly
demonstrate the superior performance of CBDNet over state-of-the-arts in terms
of quantitative metrics and visual quality. The code has been made available at
https://github.com/GuoShi28/CBDNet
Model-blind Video Denoising Via Frame-to-frame Training
Modeling the processing chain that has produced a video is a difficult
reverse engineering task, even when the camera is available. This makes model
based video processing a still more complex task. In this paper we propose a
fully blind video denoising method, with two versions off-line and on-line.
This is achieved by fine-tuning a pre-trained AWGN denoising network to the
video with a novel frame-to-frame training strategy. Our denoiser can be used
without knowledge of the origin of the video or burst and the post processing
steps applied from the camera sensor. The on-line process only requires a
couple of frames before achieving visually-pleasing results for a wide range of
perturbations. It nonetheless reaches state of the art performance for standard
Gaussian noise, and can be used off-line with still better performance.Comment: CVPR 201
- …