Thread Counting in Plain Weave for Old Paintings Using Semi-Supervised Regression Deep Learning Models

In this work, the authors develop regression approaches based on deep learning to perform thread density estimation for plain weave canvas analysis. Previous approaches were based on Fourier analysis, which is quite robust for some scenarios but fails in some others, in machine learning tools, that involve pre-labeling of the painting at hand, or the segmentation of thread crossing points, that provides good estimations in all scenarios with no need of pre-labeling. The segmentation approach is time-consuming as the estimation of the densities is performed after locating the crossing points. In this novel proposal, we avoid this step by computing the density of threads directly from the image with a regression deep learning model. We also incorporate some improvements in the initial preprocessing of the input image with an impact on the final error. Several models are proposed and analyzed to retain the best one. Furthermore, we further reduce the density estimation error by introducing a semi-supervised approach. The performance of our novel algorithm is analyzed with works by Ribera, Vel\'azquez, and Poussin where we compare our results to the ones of previous approaches. Finally, the method is put into practice to support the change of authorship or a masterpiece at the Museo del Prado.Comment: 21 page

Crossing points detection in plain weave for old paintings with deep learning

This is an open access article under the CC BY-NC-ND licenseIn the forensic studies of painting masterpieces, the analysis of the support is of major importance. For plain weave fabrics, the densities of vertical and horizontal threads are used as main features, while angle deviations from the vertical and horizontal axis are also of help. These features can be studied locally through the canvas. In this work, deep learning is proposed as a tool to perform these local densities and angle studies. We trained the model with samples from 36 paintings by Velázquez, Rubens or Ribera, among others. The data preparation and augmentation are dealt with at a first stage of the pipeline. We then focus on the supervised segmentation of crossing points between threads. The U-Net with inception and Dice loss are presented as good choices for this task. Densities and angles are then estimated based on the segmented crossing points. We report test results of the analysis of a few canvases and a comparison with methods in the frequency domain, widely used in this problem. We concluded that this new approach successes in some cases where the frequency analysis tools fail, while improves the results in others. Besides, our proposal does not need the labeling of part of the to be processed image. As case studies, we apply this novel algorithm to the analysis of two pairs of canvases by Velázquez and Murillo, to conclude that the fabrics used came from the same roll.Consejería de Transformación Económica, Industria, Conocimiento y Universidades, Junta de Andalucía y la Unión Europea P20_01216 PID2021-123182OB-I00 216 PID2021-127871OB-I00Ministerio de Ciencia e Innovación de España MCIN/AEI/10.13039/50110001103

Artificial intelligence for art investigation: Meeting the challenge of separating x-ray images of the Ghent Altarpiece

X-ray images of polyptych wings, or other artworks painted on both sides of their support, contain in one image content from both paintings, making them difficult for experts to “read.” To improve the utility of these x-ray images in studying these artworks, it is desirable to separate the content into two images, each pertaining to only one side. This is a difficult task for which previous approaches have been only partially successful. Deep neural network algorithms have recently achieved remarkable progress in a wide range of image analysis and other challenging tasks. We, therefore, propose a new self-supervised approach to this x-ray separation, leveraging an available convolutional neural network architecture; results obtained for details from the Adam and Eve panels of the Ghent Altarpiece spectacularly improve on previous attempts

Mixed X-Ray Image Separation for Artworks with Concealed Designs

In this paper, we focus on X-ray images (Xradiographs) of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which therefore include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the Xray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the X-ray image of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Dona Isabel de Porcel ˜ by Francisco de Goya, to show its effectiveness

Nineteen Figures and Counting: Contextualization and Conservation Treatment of a Jacob Spoel Painting

This study focuses on the research, technical analysis, and treatment of an 1852 Jacob Spoel painting (Untitled, acc.62.28, 80cm H x 105cm W x 1.75cm D) owned by the Memorial Art Gallery and described as a ‘family gathering.’ When received by the department, the painting was not in a fit state for display; it was not structurally sound and had a disfiguring varnish reducing the readability of the composition. Technical research, multimodal imaging, radiography, and instrumental analysis, including x-ray fluorescence spectroscopy, Raman spectroscopy, cross-sectional analysis, and Fourier transform infrared spectroscopy, were carried out to understand the materials and techniques used by the artist. Results played a part in shaping the treatment strategy, and the painting was successfully stabilized and returned to a suitable condition for exhibition

Image Separation with Side Information: A Connected Auto-Encoders Based Approach

X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist’s techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon ’connected’ auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece , painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications

A Learning Based Approach to Separate Mixed X-Ray Images Associated with Artwork with Concealed Designs

X-ray images are widely used in the study of paintings. When a painting has hidden sub-surface features (e.g., reuse of the canvas or revision of a composition by the artist), the resulting X-ray images can be hard to interpret as they include contributions from both the surface painting and the hidden design. In this paper we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings (‘mixed X-ray images’) to separate them into two hypothetical X-ray images, one containing information related to the visible painting only and the other containing the hidden features. The proposed approach involves two steps: (1) separation of the mixed X-ray image into two images, guided by the combined use of a reconstruction and an exclusion loss; (2) even allocation of the error map into the two individual, separated X-ray images, yielding separation results that have an appearance that is more familiar in relation to Xray images. The proposed method was demonstrated on a real painting with hidden content, Doña Isabel de Porcel by Francisco de Goya, to show its effectiveness

Mixed X-Ray Image Separation for Artworks with Concealed Designs

In this paper, we focus on X-ray images of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the X-ray of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Do\~na Isabel de Porcel by Francisco de Goya, to show its effectiveness