Digital image processing of the Ghent altarpiece : supporting the painting's study and conservation treatment

In this article, we show progress in certain image processing techniques that can support the physical restoration of the painting, its art-historical analysis, or both. We show how analysis of the crack patterns could indicate possible areas of overpaint, which may be of great value for the physical restoration campaign, after further validation. Next, we explore how digital image inpainting can serve as a simulation for the restoration of paint losses. Finally, we explore how the statistical analysis of the relatively simple and frequently recurring objects (such as pearls in this masterpiece) may characterize the consistency of the painter’s style and thereby aid both art-historical interpretation and physical restoration campaign

A texture based approach to reconstruction of archaeological finds

Reconstruction of archaeological finds from fragments, is a tedious task requiring many hours of work from the archaeologists and restoration personnel. In this paper we present a framework for the full reconstruction of the original objects using texture and surface design information on the sherd. The texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. The confidence of this process is also defined. Feature values are derived from these original and predicted images of pieces. A combination of the feature and confidence values is used to generate an affinity measure of corresponding pieces. The optimization of total affinity gives the best assembly of the piece. Experimental results are presented on real and artificial data

A novel image inpainting framework based on multilevel image pyramids

Image inpainting is the art of manipulating an image so that it is visually unrecognizable way. A considerable amount of research has been done in this area over the last few years. However, the state of art techniques does suffer from computational complexities and plausible results. This paper proposes a multi-level image pyramid-based image inpainting algorithm. The image inpainting algorithm starts with the coarsest level of the image pyramid and overpainting information is transferred to the subsequent levels until the bottom level gets inpainted. The search strategy used in the algorithm is based on hashing the coherent information in an image which makes the search fast and accurate. Also, the search space is constrained based on the propagated information thereby reducing the complexity of the algorithm. Compared to other inpainting methods; the proposed algorithm inpaints the target region with better plausibility and human vision conformation. Experimental results show that the proposed algorithm achieves better results as compared to other inpainting techniques

Structure Preserving Large Imagery Reconstruction

With the explosive growth of web-based cameras and mobile devices, billions of photographs are uploaded to the internet. We can trivially collect a huge number of photo streams for various goals, such as image clustering, 3D scene reconstruction, and other big data applications. However, such tasks are not easy due to the fact the retrieved photos can have large variations in their view perspectives, resolutions, lighting, noises, and distortions. Fur-thermore, with the occlusion of unexpected objects like people, vehicles, it is even more challenging to find feature correspondences and reconstruct re-alistic scenes. In this paper, we propose a structure-based image completion algorithm for object removal that produces visually plausible content with consistent structure and scene texture. We use an edge matching technique to infer the potential structure of the unknown region. Driven by the estimated structure, texture synthesis is performed automatically along the estimated curves. We evaluate the proposed method on different types of images: from highly structured indoor environment to natural scenes. Our experimental results demonstrate satisfactory performance that can be potentially used for subsequent big data processing, such as image localization, object retrieval, and scene reconstruction. Our experiments show that this approach achieves favorable results that outperform existing state-of-the-art techniques

Pareto Optimized Large Mask Approach for Efficient and Background Humanoid Shape Removal

The purpose of automated video object removal is to not only detect and remove the object of interest automatically, but also to utilize background context to inpaint the foreground area. Video inpainting requires to fill spatiotemporal gaps in a video with convincing material, necessitating both temporal and spatial consistency; the inpainted part must seamlessly integrate into the background in a variety of scenes, and it must maintain a consistent appearance in subsequent frames even if its surroundings change noticeably. We introduce deep learning-based methodology for removing unwanted human-like shapes in videos. The method uses Pareto-optimized Generative Adversarial Networks (GANs) technology, which is a novel contribution. The system automatically selects the Region of Interest (ROI) for each humanoid shape and uses a skeleton detection module to determine which humanoid shape to retain. The semantic masks of human like shapes are created using a semantic-aware occlusion-robust model that has four primary components: feature extraction, and local, global, and semantic branches. The global branch encodes occlusion-aware information to make the extracted features resistant to occlusion, while the local branch retrieves fine-grained local characteristics. A modified big mask inpainting approach is employed to eliminate a person from the image, leveraging Fast Fourier convolutions and utilizing polygonal chains and rectangles with unpredictable aspect ratios. The inpainter network takes the input image and the mask to create an output image excluding the background humanoid shapes. The generator uses an encoder-decoder structure with included skip connections to recover spatial information and dilated convolution and squeeze and excitation blocks to make the regions behind the humanoid shapes consistent with their surroundings. The discriminator avoids dissimilar structure at the patch scale, and the refiner network catches features around the boundaries of each background humanoid shape. The efficiency was assessed using the Structural Learned Perceptual Image Patch Similarity, Frechet Inception Distance, and Similarity Index Measure metrics and showed promising results in fully automated background person removal task. The method is evaluated on two video object segmentation datasets (DAVIS indicating respective values of 0.02, FID of 5.01 and SSIM of 0.79 and YouTube-VOS, resulting in 0.03, 6.22, 0.78 respectively) as well a database of 66 distinct video sequences of people behind a desk in an office environment (0.02, 4.01, and 0.78 respectively).publishedVersio