127 research outputs found
Filter-Based Probabilistic Markov Random Field Image Priors: Learning, Evaluation, and Image Analysis
Markov random fields (MRF) based on linear filter responses are one of the most popular forms for modeling image priors due to their rigorous probabilistic interpretations and versatility in various applications. In this dissertation, we propose an application-independent method to quantitatively evaluate MRF image priors using model samples. To this end, we developed an efficient auxiliary-variable Gibbs samplers for a general class of MRFs with flexible potentials. We found that the popular pairwise and high-order MRF priors capture image statistics quite roughly and exhibit poor generative properties. We further developed new learning strategies and obtained high-order MRFs that well capture the statistics of the inbuilt features, thus being real maximum-entropy models, and other important statistical properties of natural images, outlining the capabilities of MRFs. We suggest a multi-modal extension of MRF potentials which not only allows to train more expressive priors, but also helps to reveal more insights of MRF variants, based on which we are able to train compact, fully-convolutional restricted Boltzmann machines (RBM) that can model visual repetitive textures even better than more complex and deep models.
The learned high-order MRFs allow us to develop new methods for various real-world image analysis problems. For denoising of natural images and deconvolution of microscopy images, the MRF priors are employed in a pure generative setting. We propose efficient sampling-based methods to infer Bayesian minimum mean squared error (MMSE) estimates, which substantially outperform maximum a-posteriori (MAP) estimates and can compete with state-of-the-art discriminative methods. For non-rigid registration of live cell nuclei in time-lapse microscopy images, we propose a global optical flow-based method. The statistics of noise in fluorescence microscopy images are studied to derive an adaptive weighting scheme for increasing model robustness. High-order MRFs are also employed to train image filters for extracting important features of cell nuclei and the deformation of nuclei are then estimated in the learned feature spaces. The developed method outperforms previous approaches in terms of both registration accuracy and computational efficiency
Virtual Super Resolution of Scale Invariant Textured Images Using Multifractal Stochastic Processes
International audienceWe present a new method of magnification for textured images featuring scale invariance properties. This work is originally motivated by an application to astronomical images. One goal is to propose a method to quantitatively predict statistical and visual properties of images taken by a forthcoming higher resolution telescope from older images at lower resolution. This is done by performing a virtual super resolution using a family of scale invariant stochastic processes, namely compound Poisson cascades, and fractional integration. The procedure preserves the visual aspect as well as the statistical properties of the initial image. An augmentation of information is performed by locally adding random small scale details below the initial pixel size. This extrapolation procedure yields a potentially infinite number of magnified versions of an image. It allows for large magnification factors (virtually infinite) and is physically conservative: zooming out to the initial resolution yields the initial image back. The (virtually) super resolved images can be used to predict the quality of future observations as well as to develop and test compression or denoising techniques
Foundations, Inference, and Deconvolution in Image Restoration
Image restoration is a critical preprocessing step in computer vision,
producing images with reduced noise, blur, and pixel defects.
This enables precise higher-level reasoning as to the scene content in
later stages of the vision pipeline (e.g., object segmentation,
detection, recognition, and tracking).
Restoration techniques have found extensive usage in a broad range of
applications from industry, medicine, astronomy, biology, and
photography.
The recovery of high-grade results requires models of the image
degradation process, giving rise to a class of often heavily
underconstrained, inverse problems.
A further challenge specific to the problem of blur removal is noise
amplification, which may cause strong distortion by ringing artifacts.
This dissertation presents new insights and problem solving procedures
for three areas of image restoration, namely (1) model
foundations, (2) Bayesian inference for high-order Markov
random fields (MRFs), and (3) blind image deblurring
(deconvolution).
As basic research on model foundations, we contribute to reconciling
the perceived differences between probabilistic MRFs on the one hand,
and deterministic variational models on the other.
To do so, we restrict the variational functional to locally supported finite
elements (FE) and integrate over the domain.
This yields a sum of terms depending locally on FE basis coefficients,
and by identifying the latter with pixels, the terms resolve to MRF
potential functions.
In contrast with previous literature, we place special emphasis on robust
regularizers used commonly in contemporary computer vision.
Moreover, we draw samples from the derived models to further
demonstrate the probabilistic connection.
Another focal issue is a class of high-order Field of Experts MRFs
which are learned generatively from natural image data and yield
best quantitative results under Bayesian estimation.
This involves minimizing an integral expression, which has no closed
form solution in general.
However, the MRF class under study has Gaussian mixture potentials,
permitting expansion by indicator variables as a technical measure.
As approximate inference method, we study Gibbs sampling in the
context of non-blind deblurring and obtain excellent results, yet
at the cost of high computing effort.
In reaction to this, we turn to the mean field algorithm, and show
that it scales quadratically in the clique size for a standard
restoration setting with linear degradation model.
An empirical study of mean field over several restoration scenarios
confirms advantageous properties with regard to both image quality and
computational runtime.
This dissertation further examines the problem of blind deconvolution,
beginning with localized blur from fast moving objects in the
scene, or from camera defocus.
Forgoing dedicated hardware or user labels, we rely only on the image
as input and introduce a latent variable model to explain the
non-uniform blur.
The inference procedure estimates freely varying kernels and we
demonstrate its generality by extensive experiments.
We further present a discriminative method for blind removal of camera
shake.
In particular, we interleave discriminative non-blind deconvolution
steps with kernel estimation and leverage the error cancellation
effects of the Regression Tree Field model to attain a deblurring
process with tightly linked sequential stages
Advanced Restoration Techniques for Images and Disparity Maps
With increasing popularity of digital cameras, the field of Computa-
tional Photography emerges as one of the most demanding areas of
research. In this thesis we study and develop novel priors and op-
timization techniques to solve inverse problems, including disparity
estimation and image restoration.
The disparity map estimation method proposed in this thesis incor-
porates multiple frames of a stereo video sequence to ensure temporal
coherency. To enforce smoothness, we use spatio-temporal connec-
tions between the pixels of the disparity map to constrain our solution.
Apart from smoothness, we enforce a consistency constraint for the
disparity assignments by using connections between the left and right
views. These constraints are then formulated in a graphical model,
which we solve using mean-field approximation. We use a filter-based
mean-field optimization that perform efficiently by updating the dis-
parity variables in parallel. The parallel updates scheme, however, is
not guaranteed to converge to a stationary point. To compare and
demonstrate the effectiveness of our approach, we developed a new
optimization technique that uses sequential updates, which runs ef-
ficiently and guarantees convergence. Our empirical results indicate
that with proper initialization, we can employ the parallel update
scheme and efficiently optimize our disparity maps without loss of
quality. Our method ranks amongst the state of the art in common
benchmarks, and significantly reduces the temporal flickering artifacts
in the disparity maps.
In the second part of this thesis, we address several image restora-
tion problems such as image deblurring, demosaicing and super-
resolution. We propose to use denoising autoencoders to learn an
approximation of the true natural image distribution. We parametrize
our denoisers using deep neural networks and show that they learn
the gradient of the smoothed density of natural images. Based on
this analysis, we propose a restoration technique that moves the so-
lution towards the local extrema of this distribution by minimizing
the difference between the input and output of our denoiser. Weii
demonstrate the effectiveness of our approach using a single trained
neural network in several restoration tasks such as deblurring and
super-resolution. In a more general framework, we define a new
Bayes formulation for the restoration problem, which leads to a more
efficient and robust estimator. The proposed framework achieves state
of the art performance in various restoration tasks such as deblurring
and demosaicing, and also for more challenging tasks such as noise-
and kernel-blind image deblurring.
Keywords. disparity map estimation, stereo matching, mean-field
optimization, graphical models, image processing, linear inverse prob-
lems, image restoration, image deblurring, image denoising, single
image super-resolution, image demosaicing, deep neural networks,
denoising autoencoder
- …