Efficient non-uniform deblurring based on generalized additive convolution model

Image with non-uniform blurring caused by camera shake can be modeled as a linear combination of the homographically transformed versions of the latent sharp image during exposure. Although such a geometrically motivated model can well approximate camera motion poses, deblurring methods in this line usually suffer from the problems of heavy computational demanding or extensive memory cost. In this paper, we develop generalized additive convolution (GAC) model to address these issues. In GAC model, a camera motion trajectory can be decomposed into a set of camera poses, i.e., in-plane translations (slice) or roll rotations (fiber), which can both be formulated as convolution operation. Moreover, we suggest a greedy algorithm to decompose a camera motion trajectory into a more compact set of slices and fibers, and together with the efficient convolution computation via fast Fourier transform (FFT), the proposed GAC models concurrently overcome the difficulties of computational cost and memory burden, leading to efficient GAC-based deblurring methods. Besides, by incorporating group sparsity of the pose weight matrix into slice GAC, the non-uniform deblurring method naturally approaches toward the uniform blind deconvolution.Department of Computin

Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake

Modelling camera shake as a space-invariant convolution simplifies the problem of removing camera shake, but often insufficiently models actual motion blur such as those due to camera rotation and movements outside the sensor plane or when objects in the scene have different distances to the camera. In an effort to address these limitations, (i) we introduce a taxonomy of camera shakes, (ii) we build on a recently introduced framework for space-variant filtering by Hirsch et al. and a fast algorithm for single image blind deconvolution for space-invariant filters by Cho and Lee to construct a method for blind deconvolution in the case of space-variant blur, and (iii), we present an experimental setup for evaluation that allows us to take images with real camera shake while at the same time recording the spacevariant point spread function corresponding to that blur. Finally, we demonstrate that our method is able to deblur images degraded by spatially-varying blur originating from real camera shake, even without using additionally motion sensor information.

Deblurring Shaken and Partially Saturated Images

International audienceWe address the problem of deblurring images degraded by camera shake blur and saturated (over-exposed) pixels. Saturated pixels violate the common assumption that the image-formation process is linear, and often cause ringing in deblurred outputs. We provide an analysis of ringing in general, and show that in order to prevent ringing, it is insufficient to simply discard saturated pixels. We show that even when saturated pixels are removed, ringing is caused by attempting to estimate the values of latent pixels that are brighter than the sensor's maximum output. Estimating these latent pixels is likely to cause large errors, and these errors propagate across the rest of the image in the form of ringing. We propose a new deblurring algorithm that locates these error-prone bright pixels in the latent sharp image, and by decoupling them from the remainder of the latent image, greatly reduces ringing. In addition, we propose an approximate forward model for saturated images, which allows us to estimate these error-prone pixels separately without causing artefacts. Results are shown for non-blind deblurring of real photographs containing saturated regions, demonstrating improved deblurred image quality compared to previous work

Recent Progress in Image Deblurring

This paper comprehensively reviews the recent development of image deblurring, including non-blind/blind, spatially invariant/variant deblurring techniques. Indeed, these techniques share the same objective of inferring a latent sharp image from one or several corresponding blurry images, while the blind deblurring techniques are also required to derive an accurate blur kernel. Considering the critical role of image restoration in modern imaging systems to provide high-quality images under complex environments such as motion, undesirable lighting conditions, and imperfect system components, image deblurring has attracted growing attention in recent years. From the viewpoint of how to handle the ill-posedness which is a crucial issue in deblurring tasks, existing methods can be grouped into five categories: Bayesian inference framework, variational methods, sparse representation-based methods, homography-based modeling, and region-based methods. In spite of achieving a certain level of development, image deblurring, especially the blind case, is limited in its success by complex application conditions which make the blur kernel hard to obtain and be spatially variant. We provide a holistic understanding and deep insight into image deblurring in this review. An analysis of the empirical evidence for representative methods, practical issues, as well as a discussion of promising future directions are also presented.Comment: 53 pages, 17 figure

Space Variant Blind Image Restoration

In this report, we are interested in blind restoration of optical images that are degraded by a space-variant (SV) blur and corrupted with Poisson noise. For example, blur variation is due to refractive index mismatch in three dimensional fluorescence microscopy or due to atmospheric turbulence in astrophysical images. In our work, the SV Point Spread Function (PSF) is approximated by a convex combination of a set of space-invariant (SI) blurring functions. The problem is thus reduced to the estimation of the set of SI PSFs and the true image. For that, we rely on a Joint Maximum A Posteriori (JMAP) approach where the image and the PSFs are jointly estimated by minimizing a given criterion including l1 and l2 norms for regularizing the image and the PSFs. Our contribution is to provide a functional for the SV blind restoration problem allowing to simultaneously estimate the PSFs and the image. We show the existence of a minimizer of such a functional in the continuous setting. We describe an algorithm based on an alternate minimization scheme using a fast scaled gradient projection (SGP) algorithm. The efficiency of the proposed method is shown on simulated and real images.Dans ce rapport, nous nous intéressons à la restauration aveugle des images optiques qui sont dégradées par un flou spatialement variant (SV) et corrompu par un bruit de Poisson. Par exemple, la variation du flou est due au changement des indices de réfraction dans la microscopie à fluorescence tridimensionnel ou due à la turbulence atmosphérique dans les images astrophysiques. Dans notre travail, la fonction d'étalement de point SV ("Space Variant Point Spread Function (SV PSF)" en anglais) est approchée par une combinaison convexe d'un ensemble fonctions de flou spatialement invariants (SI). Le problème se réduit alors à l'estimation de l'ensemble de ces fonctions ainsi que l'image nette. Pour ce faire, nous nous appuyons sur une approche par Maximum A Posteriori Joint (MAPJ) où l'image et les PSFs sont estimées conjointement par minimisation d'un critère donné contenant les normes l1 et l2 pour régulariser l'image et les PSFs. Notre contribution consiste à fournir une fonctionnelle pour la restauration aveugle SV permettant d'estimer simultanément les PSFs et l'image. Nous montrons l'existence d'un minimiseur d'une telle fonctionnelle dans un cadre continu. Nous décrivons ensuite un algorithme basé sur un schéma de minimisation alternée, chaque problème de minimisation élémentaire est résolu par une méthode rapide de gradient projeté. L'efficacité de la méthode proposée est montrée sur des images simulées et réelles