Virtual restoration of the Ghent altarpiece using crack detection and inpainting

In this paper, we present a new method for virtual restoration of digitized paintings, with the special focus on the Ghent Altarpiece (1432), one of Belgium's greatest masterpieces. The goal of the work is to remove cracks from the digitized painting thereby approximating how the painting looked like before ageing for nearly 600 years and aiding art historical and palaeographical analysis. For crack detection, we employ a multiscale morphological approach, which can cope with greatly varying thickness of the cracks as well as with their varying intensities (from dark to the light ones). Due to the content of the painting (with extremely many fine details) and complex type of cracks (including inconsistent whitish clouds around them), the available inpainting methods do not provide satisfactory results on many parts of the painting. We show that patch-based methods outperform pixel-based ones, but leaving still much room for improvements in this application. We propose a new method for candidate patch selection, which can be combined with different patch-based inpainting methods to improve their performance in crack removal. The results demonstrate improved performance, with less artefacts and better preserved fine details

Contextual-based Image Inpainting: Infer, Match, and Translate

We study the task of image inpainting, which is to fill in the missing region of an incomplete image with plausible contents. To this end, we propose a learning-based approach to generate visually coherent completion given a high-resolution image with missing components. In order to overcome the difficulty to directly learn the distribution of high-dimensional image data, we divide the task into inference and translation as two separate steps and model each step with a deep neural network. We also use simple heuristics to guide the propagation of local textures from the boundary to the hole. We show that, by using such techniques, inpainting reduces to the problem of learning two image-feature translation functions in much smaller space and hence easier to train. We evaluate our method on several public datasets and show that we generate results of better visual quality than previous state-of-the-art methods.Comment: ECCV 2018 camera read

Topological data analysis to improve exemplar-based inpainting

Image inpainting is the process of filling in the missing region to preserve continuity of its overall content and semantic. In this paper, we present a novel approach to improve an existing scheme, called exemplar-based inpainting algorithm, using Topological Data Analysis (TDA). TDA is a mathematical approach concern studying shapes or objects to gain information about connectivity and closeness property of those objects. The challenge in using exemplar-based inpainting is that missing regions neighborhood area needs to have a relatively simple texture and structure. We studied the topological properties (e.g. number of connected components) of missing regions surrounding the missing area by building a sequence of simplicial complexes (known as persistent homology) based on a selected group of uniform Local binary Pattern LBP. Connected components of image regions generated by certain landmark pixels, at different thresholds, automatically quantify the texture nature of the missing regions surrounding areas. Such quantification help determine the appropriate size of patch propagation. We have modified the patch propagation priority function using geometrical properties of curvature of isophote and improved the matching criteria of patches by calculating the correlation coefficients from spatial, gradient and Laplacian domain. We use several image quality measures to illustrate the performance of our approach in comparison to similar inpainting algorithms. In particular, we shall illustrate that our proposed scheme outperforms the state-of-the-art exemplar-based inpainting algorithm

Digital image processing of the Ghent altarpiece : supporting the painting's study and conservation treatment

In this article, we show progress in certain image processing techniques that can support the physical restoration of the painting, its art-historical analysis, or both. We show how analysis of the crack patterns could indicate possible areas of overpaint, which may be of great value for the physical restoration campaign, after further validation. Next, we explore how digital image inpainting can serve as a simulation for the restoration of paint losses. Finally, we explore how the statistical analysis of the relatively simple and frequently recurring objects (such as pearls in this masterpiece) may characterize the consistency of the painter’s style and thereby aid both art-historical interpretation and physical restoration campaign