Small-kernel image restoration

The goal of image restoration is to remove degradations that are introduced during image acquisition and display. Although image restoration is a difficult task that requires considerable computation, in many applications the processing must be performed significantly faster than is possible with traditional algorithms implemented on conventional serial architectures. as demonstrated in this dissertation, digital image restoration can be efficiently implemented by convolving an image with a small kernel. Small-kernel convolution is a local operation that requires relatively little processing and can be easily implemented in parallel. A small-kernel technique must compromise effectiveness for efficiency, but if the kernel values are well-chosen, small-kernel restoration can be very effective.;This dissertation develops a small-kernel image restoration algorithm that minimizes expected mean-square restoration error. The derivation of the mean-square-optimal small kernel parallels that of the Wiener filter, but accounts for explicit spatial constraints on the kernel. This development is thorough and rigorous, but conceptually straightforward: the mean-square-optimal kernel is conditioned only on a comprehensive end-to-end model of the imaging process and spatial constraints on the kernel. The end-to-end digital imaging system model accounts for the scene, acquisition blur, sampling, noise, and display reconstruction. The determination of kernel values is directly conditioned on the specific size and shape of the kernel. Experiments presented in this dissertation demonstrate that small-kernel image restoration requires significantly less computation than a state-of-the-art implementation of the Wiener filter yet the optimal small-kernel yields comparable restored images.;The mean-square-optimal small-kernel algorithm and most other image restoration algorithms require a characterization of the image acquisition device (i.e., an estimate of the device\u27s point spread function or optical transfer function). This dissertation describes an original method for accurately determining this characterization. The method extends the traditional knife-edge technique to explicitly deal with fundamental sampled system considerations of aliasing and sample/scene phase. Results for both simulated and real imaging systems demonstrate the accuracy of the method

Advanced Restoration Techniques for Images and Disparity Maps

With increasing popularity of digital cameras, the field of Computa- tional Photography emerges as one of the most demanding areas of research. In this thesis we study and develop novel priors and op- timization techniques to solve inverse problems, including disparity estimation and image restoration. The disparity map estimation method proposed in this thesis incor- porates multiple frames of a stereo video sequence to ensure temporal coherency. To enforce smoothness, we use spatio-temporal connec- tions between the pixels of the disparity map to constrain our solution. Apart from smoothness, we enforce a consistency constraint for the disparity assignments by using connections between the left and right views. These constraints are then formulated in a graphical model, which we solve using mean-field approximation. We use a filter-based mean-field optimization that perform efficiently by updating the dis- parity variables in parallel. The parallel updates scheme, however, is not guaranteed to converge to a stationary point. To compare and demonstrate the effectiveness of our approach, we developed a new optimization technique that uses sequential updates, which runs ef- ficiently and guarantees convergence. Our empirical results indicate that with proper initialization, we can employ the parallel update scheme and efficiently optimize our disparity maps without loss of quality. Our method ranks amongst the state of the art in common benchmarks, and significantly reduces the temporal flickering artifacts in the disparity maps. In the second part of this thesis, we address several image restora- tion problems such as image deblurring, demosaicing and super- resolution. We propose to use denoising autoencoders to learn an approximation of the true natural image distribution. We parametrize our denoisers using deep neural networks and show that they learn the gradient of the smoothed density of natural images. Based on this analysis, we propose a restoration technique that moves the so- lution towards the local extrema of this distribution by minimizing the difference between the input and output of our denoiser. Weii demonstrate the effectiveness of our approach using a single trained neural network in several restoration tasks such as deblurring and super-resolution. In a more general framework, we define a new Bayes formulation for the restoration problem, which leads to a more efficient and robust estimator. The proposed framework achieves state of the art performance in various restoration tasks such as deblurring and demosaicing, and also for more challenging tasks such as noise- and kernel-blind image deblurring. Keywords. disparity map estimation, stereo matching, mean-field optimization, graphical models, image processing, linear inverse prob- lems, image restoration, image deblurring, image denoising, single image super-resolution, image demosaicing, deep neural networks, denoising autoencoder

A new bandwidth adaptive non-local kernel regression algorithm for image/video restoration and its GPU realization

This paper presents a new bandwidth adaptive nonlocal kernel regression (BA-NLKR) algorithm for image and video restoration. NLKR is a recent approach for improving the performance of conventional steering kernel regression (SKR) and local polynomial regression (LPR) in image/video processing. Its bandwidth, which controls the amount of smoothing, however is chosen empirically. The proposed algorithm incorporates the intersecting confidence intervals (ICI) bandwidth selection method into the framework of NLKR to facilitate automatic bandwidth selection so as to achieve better performance. A parallel implementation of the proposed algorithm is also introduced to reduce significantly its computation time. The effectiveness of the proposed algorithm is illustrated by experimental results on both single image and videos super resolution and denoising.published_or_final_versio

Fully-automatic defects classification and restoration for STM images.

The Scanning tunneling microscope (STM) is a micro instrument designed for surface morphology with nanometer precision. The restoration of the STM image defects usually needs human judgements and manual positioning because of the diversity of the morphology and the randomness of the defects. This paper provides a new fully-automatic method that combines deep convolutional neural classification network and unique restoration algorithms corresponding to different defects. Aimed at automatically processing compound defects in STM images, the method first predicts what kinds of defects a raw STM image has by a series of parallel binary classification networks, and then decides the process order according to the predicted labels, and finally restores the defects by corresponding global restoration algorithms in order. Experiment results prove the provided method can restore the STM images by self-judging, self-positioning, self-processing without any manual intervention

Fast and easy blind deblurring using an inverse filter and PROBE

PROBE (Progressive Removal of Blur Residual) is a recursive framework for blind deblurring. Using the elementary modified inverse filter at its core, PROBE's experimental performance meets or exceeds the state of the art, both visually and quantitatively. Remarkably, PROBE lends itself to analysis that reveals its convergence properties. PROBE is motivated by recent ideas on progressive blind deblurring, but breaks away from previous research by its simplicity, speed, performance and potential for analysis. PROBE is neither a functional minimization approach, nor an open-loop sequential method (blur kernel estimation followed by non-blind deblurring). PROBE is a feedback scheme, deriving its unique strength from the closed-loop architecture rather than from the accuracy of its algorithmic components

Photographic Image Restoration

Deblurring capabilities would significantly improve the Flight Science Support Office's ability to monitor the effects of lift-off on the shuttle and landing on the orbiter. A deblurring program was written and implemented to extract information from blurred images containing a straight line or edge and to use that information to deblur the image. The program was successfully applied to an image blurred by improper focussing and two blurred by different amounts of blurring. In all cases, the reconstructed modulation transfer function not only had the same zero contours as the Fourier transform of the blurred image but the associated point spread function also had structure not easily described by simple parameterizations. The difficulties posed by the presence of noise in the blurred image necessitated special consideration. An amplitude modification technique was developed for the zero contours of the modulation transfer function at low to moderate frequencies and a smooth filter was used to suppress high frequency noise