Investigation of the complementary use of non-invasive techniques for the holistic analysis of paintings and automatic analysis of large scale spectral imaging data

The analysis of painting materials and methods is acknowledged for providing important information to art history. This study illustrates a detailed examination of the characteristics, advantages and capabilities that the combined application of a variety of non-invasive techniques, ranging from spectral imaging and optical coherence tomography (OCT), to fibre optics reflectance spectroscopy (FORS), X-ray Fluorescence (XRF) and Raman spectroscopy, has to offer. The analysis of painting materials is seen under the prism of a holistic examination of different types of cultural heritage objects. More specifically, the limitations that the individual techniques face and, most importantly, how their complementary use can overcome them are thoroughly investigated through the examination of a large and heterogeneous statistical sample, in a completely novel way. The heterogeneity of the sample is related both to the painting materials (i.e pigments, binding media and substrates) and the degradation level (i.e. paintings stored in storages of museum and murals of caves that are exposed in the environmental conditions of the desert). For the extraction of accurate conclusions about the painting materials and methods applied in a specific period, the examination of large number of artworks of this period is required. PRISMS, the spectral imaging system developed by our group enables the time efficient imaging of large painting surfaces, leading to the acquisition of large scale spectral imaging data, which makes such an analysis faster, more cost-effective and less laborious without diminishing the quality of the results. This study proposes methods based on the statistical analysis for the automatic processing of the spectral imaging data in two directions: the revealing of information that is obvious under visual observation and clustering of the spectral information. With regards to the automatic revealing of hidden information, the potential of principal component analysis (PCA) and independent principal analysis (ICA), two of the most commonly used statistical analysis methods, were examined giving very good results. In addition, the development of a new method for the automatic clustering of large scale spectral information based on the 'Self-organised mapping' (SOM) method is presented. The spectral feature of the analysed areas in the UV-VIS/NIR (400-900 nm) is indicative for its pigment composition, therefore the automatic clustering of the pixel-level spectral information that the PRISMS system provides can classify the areas according to their pigment composition. The application of statistical analysis methods in the preliminary stage of the analysis of large number of artworks (e.g. large painting collections) of large surface painting areas (e.g. murals) is of significant importance; as they highlight the areas that should be examined in detail. The multimodal non-invasive approach was applied on the examination of three artworks of significant importance for East Asian art history; the cave 465 of the Mogao complex in China, the export Chinese watercolor paintings from the collections of the Victoria and Albert (V&A) museum and the Royal Horticulture Society (RHS) and the Selden map. The examination of these three works of art, in addition to providing a wide and heterogeneous sample for the detailed examination of the multi-modal approach, has also helped addressing several historical questions

A holistic multimodal approach to the non-invasive analysis of watercolour paintings

A holistic approach using non-invasive multimodal imaging and spectroscopic techniques to study the materials (pigments, drawing materials and paper) and painting techniques of watercolour paintings is presented. The non-invasive imaging and spectroscopic techniques include VIS-NIR reflectance spectroscopy and multispectral imaging, micro-Raman spectroscopy, X-ray fluorescence spectroscopy (XRF) and optical coherence tomography (OCT). The three spectroscopic techniques complement each other in pigment identification. Multispectral imaging (near infrared bands), OCT and micro-Raman complement each other in the visualisation and identification of the drawing material. OCT probes the microstructure and light scattering properties of the substrate while XRF detects the elemental composition that indicates the sizing methods and the filler content . The multiple techniques were applied in a study of forty six 19th century Chinese export watercolours from the Victoria & Albert Museum (V&A) and the Royal Horticultural Society (RHS) to examine to what extent the non-invasive analysis techniques employed complement each other and how much useful information about the paintings can be extracted to address art conservation and history questions

Optical coherence tomography and non-linear microscopy for paintings – a study of the complementary capabilities and laser degradation effects

This paper examines for the first time the potential complementary imaging capabilities of Optical coherence tomography (OCT) and non-linear microscopy (NLM) for multi-modal 3D examination of paintings following the successful application of OCT to the in situ, non-invasive examination of varnish and paint stratigraphy of historic paintings and the promising initial studies of NLM of varnish samples. OCT provides image contrast through the optical scattering and absorption properties of materials, while NLM provides molecular information through multi-photon fluorescence and higher harmonics generation (second and third harmonic generation). OCT is well-established in the in situ non-invasive imaging of the stratigraphy of varnish and paint layers. While NLM examination of transparent samples such as fresh varnish and some transparent paints showed promising results, the ultimate use of NLM on paintings is limited owing to the laser degradation effects caused by the high peak intensity of the laser source necessary for the generation of non-linear phenomena. The high intensity normally employed in NLM is found to be damaging to all non-transparent painting materials from slightly scattering degraded varnish to slightly absorbing paint at the wavelength of the laser excitation source. The results of this paper are potentially applicable to a wide range of materials given the diversity of the materials encountered in paintings (e.g. minerals, plants, insects, oil, egg, synthetic and natural varnish)

Standoff Raman spectroscopy for architectural interiors from 3-15 m distances

Portable and mobile Raman spectroscopy systems are increasingly being adopted in in situ non-invasive examination of artworks given their high specificity in material identification. However, these systems typically operate within centimeter range working distances, making the examination of large architectural interiors such as wall paintings in churches challenging. We demonstrate the first standoff Raman spectroscopy system for in situ investigation of historic architectural interior at distances > 3 m. The 780 nm continuous wave laser-induced standoff Raman system was successfully deployed for the in situ examination of wall paintings, at distances of 3–15 m, under ambient light. It is able to identify most common pigments while maintaining a very low laser intensity to avoid light induced degradation. It is shown to complement our current method of standoff remote surveys of wall paintings using spectral imaging

A non-invasive investigation of Limoges enamels using both Optical Coherence Tomography (OCT) and spectral imaging: a pilot study

This paper investigates the use of Optical Coherence Tomography (OCT) and Short-wave Infrared (SWIR) spectral imaging to study the deterioration of a Limoges enamel panel. Limoges enamels are formed of glass layers applied on a metal substrate and are prone to ‘glass disease’. However, the level of deterioration in Limoges enamels is generally difficult to assess visually. In this study, SWIR was used to produce a hydration level map of the enamel, which was coupled with virtual OCT cross-sections. The study shows a good correlation between levels of hydration and structural damage over the enamel panel. Hydration mapping allows visualisation of structural damage across the entire enamel in one image

Uncovering the Oppenheimer Siddur: using scientific analysis to reveal the production process of a medieval illuminated Hebrew manuscript

The aim of this research was to use non-invasive scientifc analysis to uncover evidence of the planning process and relationship between pigments used in text copying and artwork production in the Oppenheimer Siddur (Oxford Bodleian Library MS Opp. 776), an illuminated 15th-century Hebrew prayer book. In many medieval Hebrew illuminated manuscripts, the authorship of the artwork is unknown. This manuscript’s colophon states that it was copied by its scribe-owner for personal family use but does not confrm who was responsible for the artwork. Prior deductive analysis suggested that the scribe-owner may also have been the manuscript’s artist, based on common motifs and an apparent shared colour palette appearing in both texts and artwork. Visual examination using high resolution digital images also identifed points of contact between pigments used in the manuscript’s texts and artwork, raising questions about the pigment application sequence, and concurrent versus sequential text copying and artwork production. An in-house developed remote spectral imaging system (PRISMS) with 10 flters spanning the spectral range from 400 to 880 nm was modifed for close-range application to image two of the folios to examine the sequence of production, identify the pigments and compare the materials used for the illumination and the text. Optical microscopy and Fourier Transform Infrared spectroscopy in the attenuated total refection mode (FTIR-ATR) were used directly on the folios to complement the spectral imaging data in binding media and pigment identifcation. The results revealed close matches in refectance spectra for the colorants and inks used in both text copying and illuminations, suggesting that the same mixture of colorants and inks have been used. The spectral imaging in the near infrared bands revealed a hidden underdrawing, indicating a design change during production of the manuscript, and the outlining of letters prior to coloured pigment being applied. The pigment use, the variation in the binder for diferent pigments and some elements of its production were found to be consistent with those described in historical sources. The evidence from this study supports the hypothesis that the scribe applied pigments for the manuscript’s artwork at the same time he did some of the scribal work which has implications for understandings of Jewish medieval visual cultures

Deep learning for the extraction of sketches from spectral images of historical paintings

The sketches of painted cultural objects can be the most indicative of the style of paintings. Extraction of the sketches is an integral process used by conservators and art historians for documentation and for artists to learn historical painting styles through copying and painting. However, at present, sketch extraction is mainly manually drawn, which is not only time-consuming, but also subjective and dependent on experience. Therefore, both accuracy and efficiency need to be improved. In recent years, with the development of machine learning, a series of extraction methods based on edge detection have emerged. However, most of the existing methods can only perform successful extraction if the sketches are well preserved , but for the data with faded sketches or severe conservation issues, the extraction methods need to be improved. It is beneficial to extract the bands that accentuate the sketches while suppressing the effects of the degraded areas and the overlapping paints. We propose a sketch extraction method based on hyperspectral image and deep learning. Firstly, the hyperspectral image data is collected and the bands sensitive to the sketches are extracted by a prior knowledge of the sketches (e.g. near infrared bands will be chosen if the sketches are made of carbon ink), and a dataset including a large number of existing natural images is used to pre-train the bi-directional cascade network (BDCN). The network parameters in the model are then fine-tuned by using the images of painted cultural objects drawn by experts, so as to solve the problem of insufficient sketch dataset of painted cultural objects and enhance the generalization ability of the model. Finally, the U-net network is used to further suppress the noise, i.e. unwanted information, and make the sketch clearer. The experimental results show that the proposed method can not only effectively extract sketch from ideal data, but also extract clear sketches from data with faded sketches and even with noise interference. It is superior to the other six advanced based on edge detection methods in visual and objective comparison, and has a good application prospect. The proposed deep learning method is also compared with an unsupervised clustering method using Self-Organising Map (SOM) which is a ‘shallow learning’ method where pixels of similar spectra are grouped into clusters without the need for data labeling by experts

The origins of the Selden map of China: scientific analysis of the painting materials and techniques using a holistic approach

Since the 'rediscovery' of the Selden map of China, an early seventeenth century map of Asia, in the Bodleian Library in Oxford, the importance of the map in our understanding of globalisation in the early seventeenth century has been recognised. One of the unresolved questions is the origin of the map. This paper addresses the question through material evidence provided by a holistic approach using a suite of complementary analytical techniques. The map was examined in situ and non-invasively by a remote spectral imaging instrument (PRISMS) modified for close range imaging, which was followed by a range of complementary techniques applied to a number of detached fragments, though most of the techniques are non-invasive and can be applied to the map directly in the future. The binding medium was found to be a gum, almost certainly gum Arabic, rather than the animal glue commonly used in Chinese paintings. Some of the pigments and their usage were found to be at odds with the common practice in paintings from China. The detection of gum Arabic, a binding medium used by the Europeans, South and West Asians and the use of a mixture of orpiment and indigo, commonly found in European, South and West Asian paintings gives further evidence on the unusual origins of this map. The likely detection of a basic copper chloride, such as atacamite, in the green areas suggests an influence from the South and West Asian rather than the European tradition. Detailed analysis of the various spectral bands of the spectral image cube along with visual inspection of the large scale colour image showed that the map was not fully planned at the beginning but rather painted in stages, at times by trial and error and that it was unfinished. A new hypothesis for the origin of the Selden map in Aceh Sumatra is proposed based on the new evidences