Craquelure as a Graph: Application of Image Processing and Graph Neural Networks to the Description of Fracture Patterns

Cracks on a painting is not a defect but an inimitable signature of an artwork which can be used for origin examination, aging monitoring, damage identification, and even forgery detection. This work presents the development of a new methodology and corresponding toolbox for the extraction and characterization of information from an image of a craquelure pattern. The proposed approach processes craquelure network as a graph. The graph representation captures the network structure via mutual organization of junctions and fractures. Furthermore, it is invariant to any geometrical distortions. At the same time, our tool extracts the properties of each node and edge individually, which allows to characterize the pattern statistically. We illustrate benefits from the graph representation and statistical features individually using novel Graph Neural Network and hand-crafted descriptors correspondingly. However, we also show that the best performance is achieved when both techniques are merged into one framework. We perform experiments on the dataset for paintings' origin classification and demonstrate that our approach outperforms existing techniques by a large margin.Comment: Published in ICCV 2019 Workshop

Crack detection in paintings using convolutional neural networks

The accurate detection of cracks in paintings, which generally portray rich and varying content, is a challenging task. Traditional crack detection methods are often lacking on recent acquisitions of paintings as they are poorly adapted to high-resolutions and do not make use of the other imaging modalities often at hand. Furthermore, many paintings portray a complex or cluttered composition, significantly complicating a precise detection of cracks when using only photographic material. In this paper, we propose a fast crack detection algorithm based on deep convolutional neural networks (CNN) that is capable of combining several imaging modalities, such as regular photographs, infrared photography and X-Ray images. Moreover, we propose an efficient solution to improve the CNN-based localization of the actual crack boundaries and extend the CNN architecture such that areas where it makes little sense to run expensive learning models are ignored. This allows us to process large resolution scans of paintings more efficiently. The proposed on-line method is capable of continuously learning from newly acquired visual data, thus further improving classification results as more data becomes available. A case study on multimodal acquisitions of the Ghent Altarpiece, taken during the currently ongoing conservation-restoration treatment, shows improvements over the state-of-the-art in crack detection methods and demonstrates the potential of our proposed method in assisting art conservators

Advances in multispectral and hyperspectral imaging for archaeology and art conservation

Multispectral imaging has been applied to the field of art conservation and art history since the early 1990s. It is attractive as a noninvasive imaging technique because it is fast and hence capable of imaging large areas of an object giving both spatial and spectral information. This paper gives an overview of the different instrumental designs, image processing techniques and various applications of multispectral and hyperspectral imaging to art conservation, art history and archaeology. Recent advances in the development of remote and versatile multispectral and hyperspectral imaging as well as techniques in pigment identification will be presented. Future prospects including combination of spectral imaging with other noninvasive imaging and analytical techniques will be discussed

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)

The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist1

Image Separation with Side Information: A Connected Auto-Encoders Based Approach

X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist’s techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon ’connected’ auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece , painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications

Combination of the Photogrammetric and Microwave Remote Sensing for Cultural Heritage Documentation and Preservation– Preliminary Results

Nowadays, cultural heritage is an integral part of modern societies and it is necessary to preserve the tangible and intangible evidences of the past. Cultural heritage objects and sites are being devastated by natural causes and human acts. There are many existing techniques for structural health monitoring, but in the cultural heritage area, there is a need for a non-destructive measurements. The main advantages and drawbacks of some of these technologies have been presented for both laser scanner and photogrammetry techniques. The aim of this article is to present the integration of existing methods for spatial documentation (classical surveying, laser scanner, photogrammetry) and structures health monitoring using electromagnetic spectroscopy. The multi-sensor platform was developed to characterise and analyse various building materials such as marble, sandstone and bricks. The spatial techniques were used for generation of architectural documentation and geolocalisation of the multi-sensor platform. The preliminary results demonstrated that the proposed technology enables commonly used image and rage-based surface measurements as well as measurements through the entire structure for more detailed analysi

A deep learning approach to crack detection on road surfaces

Currently, modern achievements in the field of deep learning are increasingly being applied in practice. One of the practical uses of deep learning is to detect cracks on the surface of the roadway. The destruction of the roadway is the result of various factors: for example, the use of low-quality material, non-compliance with the standards of laying asphalt, external physical impact, etc. Detection of these damages in automatic mode with high speed and accuracy is an important and complex task. An effective solution to this problem can reduce the time of services that carry out the detection of damage and also increase the safety of road users. The main challenge for automatically detecting such damage, in most cases, is the complex structure of the roadway. To accurately detect this damage, we use U-Net. After that we improve the binary map with localized cracks from the U-Net neural network, using the morphological filtering. This solution allows localizing cracks with higher accuracy in comparison with traditional methods crack detection, as well as modern methods of deep learning. All experiments were performed using the publicly available CRACK500 dataset with examples of cracks and their binary maps

Comparative study of microlaser excitation thermography and microultrasonic excitation thermography for submillimeter porosity in carbon fiber reinforced polymer composites

Stitching is used to reduce incomplete infusion of T-joint core (dry-core) and reinforce T-joint structure. However, it may cause new types of flaws, especially submillimeter flaws. Thermographic approaches including microvibrothermography, microlaser line thermography, and microlaser spot thermography on the basis of pulsed and lock-in techniques were proposed. These techniques are used to detect the submillimeter porosities in a stitched T-joint carbon fiber reinforced polymer composite specimen. X-ray microcomputed tomography was used to validate the thermographic results. Finally an experimental comparison of microlaser excitation thermography and microultrasonic excitation thermography was conducte

Unsupervised hyperspectral image segmentation of films: a hierarchical clustering-based approach

Hyperspectral imaging (HSI) has been drastically applied in recent years to cultural heritage (CH) analysis, conservation, and also digital restoration. However, the efficient processing of the large datasets registered remains challenging and still in development. In this paper, we propose to use the hierarchical clustering algorithm (HCA) as an alternative machine learning approach to the most common practices, such as principal component analysis(PCA). HCA has shown its potential in the past decades for spectral data classification and segmentation in many other fields, maximizing the information to be extracted from the high-dimensional spectral dataset via the formation of the agglomerative hierarchical tree. However, to date, there has been very limited implementation of HCA in the field of cultural heritage. Data used in this experiment were acquired on real historic film samples with various degradation degrees, using a custom-made push-broom VNIR hyperspectral camera (380–780nm). With the proposed HCA workflow, multiple samples in the entire dataset were processed simultaneously and the degradation areas with distinctive characteristics were successfully segmented into clusters with various hierarchies. A range of algorithmic parameters was tested, including the grid sizes, metrics, and agglomeration methods, and the best combinations were proposed at the end. This novel application of the semi-automating and unsupervised HCA could provide a basis for future digital unfading, and show the potential to solve other CH problems such as pigment mapping

Virtual Cleaning of Works of Art Using Deep Learning Based Approaches

Virtual cleaning of art is a key process that conservators apply to see the likely appearance of the work of art they have aimed to clean, before the process of cleaning. There have been many different approaches to virtually clean artworks but having to physically clean the artwork at a few specific places of specific colors, the need to have pure black and white paint on the painting and their low accuracy are only a few of their shortcomings prompting us to propose deep learning based approaches in this research. First we report the work we have done in this field focusing on the color estimation of the artwork virtual cleaning and then we describe our methods for the spectral reflectance estimation of artwork in virtual cleaning. In the color estimation part, a deep convolutional neural network (CNN) and a deep generative network (DGN) are suggested, which estimate the RGB image of the cleaned artwork from an RGB image of the uncleaned artwork. Applying the networks to the images of the well-known artworks (such as the Mona Lisa and The Virgin and Child with Saint Anne) and Macbeth ColorChecker and comparing the results to the only physics-based model (which is the first model that has approached the issue of virtual cleaning from the physics-point of view, hence our reference to compare our models with) shows that our methods outperform that model and have great potentials of being applied to the real situations in which there might not be much information available on the painting, and all we have is an RGB image of the uncleaned artwork. Nonetheless, the methods proposed in the first part, cannot provide us with the spectral reflectance information of the artwork, therefore, the second part of the dissertation is proposed. This part focuses on the spectral estimation of the artwork virtual cleaning. Two deep learning-based approaches are also proposed here; the first one is deep generative network. This method receives a cube of the hyperspectral image of the uncleaned artwork and tries to output another cube which is the virtually cleaned hyperspectral image of the artwork. The second approach is 1D Convolutional Autoencoder (1DCA), which is based on 1D convolutional neural network and tries to find the spectra of the virtually cleaned artwork using the spectra of the physically cleaned artworks and their corresponding uncleaned spectra. The approaches are applied to hyperspectral images of Macbeth ColorChecker (simulated in the forms of cleaned and uncleaned hyperspectral images) and the \u27Haymakers\u27 (real hyperspectral images of both cleaned and uncleaned states). The results, in terms of Euclidean distance and spectral angle between the virtually cleaned artwork and the physically cleaned one, show that the proposed approaches have outperformed the physics-based model, with DGN outperforming the 1DCA. Methods proposed herein do not rely on finding a specific type of paint and color on the painting first and take advantage of the high accuracy offered by deep learning-based approaches and they are also applicable to other paintings