X-ray image separation via coupled dictionary learning

In support of art investigation, we propose a new source sepa- ration method that unmixes a single X-ray scan acquired from double-sided paintings. Unlike prior source separation meth- ods, which are based on statistical or structural incoherence of the sources, we use visual images taken from the front- and back-side of the panel to drive the separation process. The coupling of the two imaging modalities is achieved via a new multi-scale dictionary learning method. Experimental results demonstrate that our method succeeds in the discrimination of the sources, while state-of-the-art methods fail to do so.Comment: To be presented at the IEEE International Conference on Image Processing (ICIP), 201

Image Processing for Art Investigation

Recent advances in digital image acquisition methods and the wide range of imaging modalities currently available have triggered museums to digitize their painting collections. Not only is this crucial for archival or dissemination purposes but it also enabled the digital analysis of the painting through its digital image counterpart. It also set in motion a cross-disciplinary collaboration between image analysis specialists, mathematicians, statisticians and art historians that have the common goal to develop algorithms and build a digital toolbox in support of art scholarship. Computer processing of digital images of paintings has become a fast growing and challenging field of research during the last few years. Our contribution to this research domain consists of a set of tools that are based on dimensionality reduction methods, sparse representations and dictionary learning techniques. These tools are used to assist in art related matters such as restoration, conservation, art history, material and structure characterization, authentication, dating and even style analysis. Since paintings are complex structures the analysis of all pictorial layers and the support requires a multimodal set of high-resolution image acquisitions. The presented research can broadly be subdivided into three main fields. The first one is the digital enhancement of painting acquisitions in order to assist the art specialist in his professional assessment of the painting. The second main field of research is the automated detection of cracks within the Ghent Altarpiece, which is meant to help in the delicate matter of the conservation of this exceptional masterpiece but also as guidance during its current campaign of restoration. The last field consists of a set of methods that can be deployed in art forensics. These methods consist of the characterization of canvas, the analysis of multispectral imagery of a painting and even the objective quantification of the style of a particular artist.

Multi-modal dictionary learning for image separation with application in art investigation

In support of art investigation, we propose a new source separation method that unmixes a single X-ray scan acquired from double-sided paintings. In this problem, the X-ray signals to be separated have similar morphological characteristics, which brings previous source separation methods to their limits. Our solution is to use photographs taken from the front and back-side of the panel to drive the separation process. The crux of our approach relies on the coupling of the two imaging modalities (photographs and X-rays) using a novel coupled dictionary learning framework able to capture both common and disparate features across the modalities using parsimonious representations; the common component models features shared by the multi-modal images, whereas the innovation component captures modality-specific information. As such, our model enables the formulation of appropriately regularized convex optimization procedures that lead to the accurate separation of the X-rays. Our dictionary learning framework can be tailored both to a single- and a multi-scale framework, with the latter leading to a significant performance improvement. Moreover, to improve further on the visual quality of the separated images, we propose to train coupled dictionaries that ignore certain parts of the painting corresponding to craquelure. Experimentation on synthetic and real data - taken from digital acquisition of the Ghent Altarpiece (1432) - confirms the superiority of our method against the state-of-the-art morphological component analysis technique that uses either fixed or trained dictionaries to perform image separation.Comment: submitted to IEEE Transactions on Images Processin

Image Processing for Art Investigation

Advisors: Ann Dooms, Ingrid Daubechies. Date and location of PhD thesis defense: 13 October 2014, Vrije Universiteit BrusselRecent advances in digital image acquisition methods and the wide range of imaging modalities currently available have triggered museums to digitize their painting collections. Not only is this crucial for archival or dissemination purposes but it also enabled the digital analysis of the painting through its digital image counterpart. It also set in motion a cross-disciplinary collaboration between image analysis specialists, mathematicians, statisticians and art historians that have the common goal to develop algorithms and build a digital toolbox in support of art scholarship. Computer processing of digital images of paintings has become a fast growing and challenging field of research during the last few years. Our contribution to this research domain consists of a set of tools that are based on dimensionality reduction methods, sparse representations and dictionary learning techniques. These tools are used to assist in art related matters such as restoration, conservation, art history, material and structure characterization, authentication, dating and even style analysis. Since paintings are complex structures the analysis of all pictorial layers and the support requires a multimodal set of high-resolution image acquisitions. The presented research can broadly be subdivided into three main fields. The first one is the digital enhancement of painting acquisitions in order to assist the art specialist in his professional assessment of the painting. The second main field of research is the automated detection of cracks within the Ghent Altarpiece, which is meant to help in the delicate matter of the conservation of this exceptional masterpiece but also as guidance during its current campaign of restoration. The last field consists of a set of methods that can be deployed in art forensics. These methods consist of the characterization of canvas, the analysis of multispectral imagery of a painting and even the objective quantification of the style of a particular artist

Assessment of plastics in the National Trust: a case study at Mr Straw's House

The National Trust is a charity that cares for over 300 publically accessible historic buildings and their contents across England, Wales and Northern Ireland. There have been few previous studies on preservation of plastics within National Trust collections, which form a significant part of the more modern collections of objects. This paper describes the design of an assessment system which was successfully trialled at Mr Straws House, a National Trust property in Worksop, UK. This system can now be used for future plastic surveys at other National Trust properties. In addition, the survey gave valuable information about the state of the collection, demonstrating that the plastics that are deteriorating are those that are known to be vulnerable, namely cellulose nitrate/acetate, PVC and rubber. Verifying this knowledge of the most vulnerable plastics enables us to recommend to properties across National Trust that these types should be seen as a priority for correct storage and in-depth recording

Classification of geometric forms in mosaics using deep neural network

The paper addresses an image processing problem in the field of fine arts. In particular, a deep learning-based technique to classify geometric forms of artworks, such as paintings and mosaics, is presented. We proposed and tested a convolutional neural network (CNN)-based framework that autonomously quantifies the feature map and classifies it. Convolution, pooling and dense layers are three distinct categories of levels that generate attributes from the dataset images by introducing certain specified filters. As a case study, a Roman mosaic is considered, which is digitally reconstructed by close-range photogrammetry based on standard photos. During the digital transformation from a 2D perspective view of the mosaic into an orthophoto, each photo is rectified (i.e., it is an orthogonal projection of the real photo on the plane of the mosaic). Image samples of the geometric forms, e.g., triangles, squares, circles, octagons and leaves, even if they are partially deformed, were extracted from both the original and the rectified photos and originated the dataset for testing the CNN-based approach. The proposed method has proved to be robust enough to analyze the mosaic geometric forms, with an accuracy higher than 97%. Furthermore, the performance of the proposed method was compared with standard deep learning frameworks. Due to the promising results, this method can be applied to many other pattern identification problems related to artworks

Image Separation with Side Information: A Connected Auto-Encoders Based Approach

X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist’s techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon ’connected’ auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece , painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications

Oil painting on copper: characterization of the copper support and the feasibility of using pigmented wax-resin infills for paint loss reintegration

This work resulted in one oral presentation and two publications: Paper accepted for presentation and publication, Colóquio em Investigações em Conservação do Património, “Arte e Ciência: Investigação sobre a técnica e materiais aplicados na pintura sobre cobre”, Daniel Vega, Isabel Pombo Cardoso and Leslie Carlyle. Faculdade de Belas Artes da Universidade de Lisboa (FBAUL), Lisbon. Paper accepted for publication, International Symposium Paintings on copper (and other metal plates). Production, degradation and conservation issues, “Investigation and testing to develop an infill formula suitable for oil paintings on copper”, Daniel Vega, Isabel Pombo Cardoso and Leslie Carlyle. Universitat Politècnica de València (UPV), Valencia, Espanha.The present work is divided in two parts. Part 1 concentrates on the study of the manufacture of copper plates used as a support for oil paintings since to date, there has not been a great deal of information available. The research involved comparing the information gathered from historical treatises on metallurgy and recent studies of paintings on copper and copper archaeometallurgy, with results from a set of thorough scientific analyses undertaken on the copper supports of fifteen European paintings (dating from the 17th and 18th centuries). This comparison revealed interesting insights into the metallurgic processes used to produce the copper ingot from native copper, and the subsequent manufacturing processes undertaken to obtain the copper plates. Copper ore purification was a complex and expensive process. Purification included several steps, all of which were rigorously executed as attested by the high level of purity of the copper produced. Scientific analyses undertaken on the copper supports of the fifteen European paintings revealed that the manufacture of the plates from the ingots involved cycles of cold working alternating with annealing. Hammering took place which would have been aimed to form a plate with adequate hardness, while the intermediate stage of annealing returned malleability so that further intense cold work, necessary to achieve a plate without breaking, could be carried out. Part 2 focus on the characterization of two wax-resins formulations used as infill materials for oil paintings: a formula used by Carlyle in the early 1980s (C-PWR) and Gamblin pigmented wax-resin (G-PWR). and, based on the negative impact on copper of the acidic beeswax in both formulations, an exploration to find a new formulation with a neutral acid value was carried out. Preliminary trials and testing focussed on the development of a new wax-resin formulation suitable for infills on oil paintings with a copper substrate. New options for infill materials on copper supports are particularly important as the range of infill materials currently available are not suitable, for a variety of reasons, for use on this type of support. Although ageing tests are still needed, the characterization of the individual materials, and of the new formulation, KTW5-R1, made of Techniwax 9426 microcrystalline wax with Regalrez 1094, showed that this wax resin mixture with an acid number of 0, is likely to be inert in relation to the copper and chemically stable since it is composed of saturated hydrocarbons only

The conservation of panel paintings and related objects

Until the early 17th century almost all portable paintings were created on wood supports, including masterpieces by famous painters, ranging from Giotto to Dürer to Rembrandt. The structural conservation of these paintings requires specific knowledge and skills as the supports are susceptible to damage caused by unstable environmental conditions. Unfortunately, past structural interventions often caused significant damage due to insufficient knowledge of the behaviour of the wood panels, glue and paint layers. Over the last fifty years, the field has developed treatment strategies based on interdisciplinary collaboration and on the knowledge of specialist conservators. Most current conservation protocols rely on empirical knowledge of conservators and are not necessarily based on a scientific understanding of the nature and behaviour of wood and paint layers. In order to move the field forward, it is imperative to strengthen scientific research into the production methods, ageing and future behaviour of panel paintings, being an intricate interplay between different materials. A deeper understanding of the processes that adversely affect panel paintings over time will contribute to the improved care and conservation of these artworks. The Netherlands Organisation for Scientific Research (NWO) and the Rijksmuseum Amsterdam brought together a group of experts from different disciplines to recommend specific areas in the field that would benefit from systematic research. The experts concluded that targeted interdisciplinary research projects are key to understanding the behaviour of panel paintings and help conservators make better informed decisions. Research into chemical and physical properties of wood, glue and paint layers should be combined with an evaluation of past and current conservation treatments. Research should also consider the history of the object, studio practice, conservation history and thoughts on long-term impact of treatments