Sobolev gradients and image interpolation

We present here a new image inpainting algorithm based on the Sobolev gradient method in conjunction with the Navier-Stokes model. The original model of Bertalmio et al is reformulated as a variational principle based on the minimization of a well chosen functional by a steepest descent method. This provides an alternative of the direct solving of a high-order partial differential equation and, consequently, allows to avoid complicated numerical schemes (min-mod limiters or anisotropic diffusion). We theoretically analyze our algorithm in an infinite dimensional setting using an evolution equation and obtain global existence and uniqueness results as well as the existence of an $\omega$-limit. Using a finite difference implementation, we demonstrate using various examples that the Sobolev gradient flow, due to its smoothing and preconditioning properties, is an effective tool for use in the image inpainting problem

Image Inpainting and Enhancement using Fractional Order Variational Model

The intention of image inpainting is to complete or fill the corrupted or missing zones of an image by considering the knowledge from the source region. A novel fractional order variational image inpainting model in reference to Caputo definition is introduced in this article. First, the fractional differential, and its numerical methods are represented according to Caputo definition. Then, a fractional differential mask is represented in 8-directions. The complex diffusivity function is also defined to preserve the edges. Finally, the missing regions are filled by using variational model with fractional differentials of 8-directions. The simulation results and analysis display that the new model not only inpaints the missing regions, but also heightens the contrast of the image. The inpainted images have better visual quality than other fractional differential filters

Optimising Spatial and Tonal Data for PDE-based Inpainting

Some recent methods for lossy signal and image compression store only a few selected pixels and fill in the missing structures by inpainting with a partial differential equation (PDE). Suitable operators include the Laplacian, the biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The quality of such approaches depends substantially on the selection of the data that is kept. Optimising this data in the domain and codomain gives rise to challenging mathematical problems that shall be addressed in our work. In the 1D case, we prove results that provide insights into the difficulty of this problem, and we give evidence that a splitting into spatial and tonal (i.e. function value) optimisation does hardly deteriorate the results. In the 2D setting, we present generic algorithms that achieve a high reconstruction quality even if the specified data is very sparse. To optimise the spatial data, we use a probabilistic sparsification, followed by a nonlocal pixel exchange that avoids getting trapped in bad local optima. After this spatial optimisation we perform a tonal optimisation that modifies the function values in order to reduce the global reconstruction error. For homogeneous diffusion inpainting, this comes down to a least squares problem for which we prove that it has a unique solution. We demonstrate that it can be found efficiently with a gradient descent approach that is accelerated with fast explicit diffusion (FED) cycles. Our framework allows to specify the desired density of the inpainting mask a priori. Moreover, is more generic than other data optimisation approaches for the sparse inpainting problem, since it can also be extended to nonlinear inpainting operators such as EED. This is exploited to achieve reconstructions with state-of-the-art quality. We also give an extensive literature survey on PDE-based image compression methods

Applications of PDEs inpainting to magnetic particle imaging and corneal topography

In this work we propose a novel application of Partial Differential Equations (PDEs) inpainting techniques to two medical contexts. The first one concerning recovering of concentration maps for superparamagnetic nanoparticles, used as tracers in the framework of Magnetic Particle Imaging. The analysis is carried out by two set of simulations, with and without adding a source of noise, to show that the inpainted images preserve the main properties of the original ones. The second medical application is related to recovering data of corneal elevation maps in ophthalmology. A new procedure consisting in applying the PDEs inpainting techniques to the radial curvature image is proposed. The images of the anterior corneal surface are properly recovered to obtain an approximation error of the required precision. We compare inpainting methods based on second, third and fourth-order PDEs with standard approximation and interpolation techniques

RIBBONS: Rapid Inpainting Based on Browsing of Neighborhood Statistics

Image inpainting refers to filling missing places in images using neighboring pixels. It also has many applications in different tasks of image processing. Most of these applications enhance the image quality by significant unwanted changes or even elimination of some existing pixels. These changes require considerable computational complexities which in turn results in remarkable processing time. In this paper we propose a fast inpainting algorithm called RIBBONS based on selection of patches around each missing pixel. This would accelerate the execution speed and the capability of online frame inpainting in video. The applied cost-function is a combination of statistical and spatial features in all neighboring pixels. We evaluate some candidate patches using the proposed cost function and minimize it to achieve the final patch. Experimental results show the higher speed of 'Ribbons' in comparison with previous methods while being comparable in terms of PSNR and SSIM for the images in MISC dataset

A New Multiscale Representation for Shapes and Its Application to Blood Vessel Recovery

In this paper, we will first introduce a novel multiscale representation (MSR) for shapes. Based on the MSR, we will then design a surface inpainting algorithm to recover 3D geometry of blood vessels. Because of the nature of irregular morphology in vessels and organs, both phantom and real inpainting scenarios were tested using our new algorithm. Successful vessel recoveries are demonstrated with numerical estimation of the degree of arteriosclerosis and vessel occlusion.Comment: 12 pages, 3 figure

DIGITAL INPAINTING ALGORITHMS AND EVALUATION

Digital inpainting is the technique of filling in the missing regions of an image or a video using information from surrounding area. This technique has found widespread use in applications such as restoration, error recovery, multimedia editing, and video privacy protection. This dissertation addresses three significant challenges associated with the existing and emerging inpainting algorithms and applications. The three key areas of impact are 1) Structure completion for image inpainting algorithms, 2) Fast and efficient object based video inpainting framework and 3) Perceptual evaluation of large area image inpainting algorithms. One of the main approach of existing image inpainting algorithms in completing the missing information is to follow a two stage process. A structure completion step, to complete the boundaries of regions in the hole area, followed by texture completion process using advanced texture synthesis methods. While the texture synthesis stage is important, it can be argued that structure completion aspect is a vital component in improving the perceptual image inpainting quality. To this end, we introduce a global structure completion algorithm for completion of missing boundaries using symmetry as the key feature. While existing methods for symmetry completion require a-priori information, our method takes a non-parametric approach by utilizing the invariant nature of curvature to complete missing boundaries. Turning our attention from image to video inpainting, we readily observe that existing video inpainting techniques have evolved as an extension of image inpainting techniques. As a result, they suffer from various shortcoming including, among others, inability to handle large missing spatio-temporal regions, significantly slow execution time making it impractical for interactive use and presence of temporal and spatial artifacts. To address these major challenges, we propose a fundamentally different method based on object based framework for improving the performance of video inpainting algorithms. We introduce a modular inpainting scheme in which we first segment the video into constituent objects by using acquired background models followed by inpainting of static background regions and dynamic foreground regions. For static background region inpainting, we use a simple background replacement and occasional image inpainting. To inpaint dynamic moving foreground regions, we introduce a novel sliding-window based dissimilarity measure in a dynamic programming framework. This technique can effectively inpaint large regions of occlusions, inpaint objects that are completely missing for several frames, change in size and pose and has minimal blurring and motion artifacts. Finally we direct our focus on experimental studies related to perceptual quality evaluation of large area image inpainting algorithms. The perceptual quality of large area inpainting technique is inherently a subjective process and yet no previous research has been carried out by taking the subjective nature of the Human Visual System (HVS). We perform subjective experiments using eye-tracking device involving 24 subjects to analyze the effect of inpainting on human gaze. We experimentally show that the presence of inpainting artifacts directly impacts the gaze of an unbiased observer and this in effect has a direct bearing on the subjective rating of the observer. Specifically, we show that the gaze energy in the hole regions of an inpainted image show marked deviations from normal behavior when the inpainting artifacts are readily apparent