Multispectral RTI Analysis of Heterogeneous Artworks

We propose a novel multi-spectral reflectance transformation imaging (MS-RTI) framework for the acquisition and direct analysis of the reflectance behavior of heterogeneous artworks. Starting from free-form acquisitions, we compute per-pixel calibrated multi-spectral appearance profiles, which associate a reflectance value to each sampled light direction and frequency. Visualization, relighting, and feature extraction is performed directly on appearance profile data, applying scattered data interpolation based on Radial Basis Functions to estimate per-pixel reflectance from novel lighting directions. We demonstrate how the proposed solution can convey more insights on the object materials and geometric details compared to classical multi-light methods that rely on low-frequency analytical model fitting eventually mixed with a separate handling of high-frequency components, hence requiring constraining priors on material behavior. The flexibility of our approach is illustrated on two heterogeneous case studies, a painting and a dark shiny metallic sculpture, that showcase feature extraction, visualization, and analysis of high-frequency properties of artworks using multi-light, multi-spectral (Visible, UV and IR) acquisitions.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091the DSURF (PRIN 2015) project funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa

Surface analysis and visualization from multi-light image collections

Multi-Light Image Collections (MLICs) are stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination that provides large amounts of visual and geometric information. Over the last decades, a wide variety of methods have been devised to extract information from MLICs and have shown its use in different application domains to support daily activities. In this thesis, we present methods that leverage a MLICs for surface analysis and visualization. First, we provide background information: acquisition setup, light calibration and application areas where MLICs have been successfully used for the research of daily analysis work. Following, we discuss the use of MLIC for surface visualization and analysis and available tools used to support the analysis. Here, we discuss methods that strive to support the direct exploration of the captured MLIC, methods that generate relightable models from MLIC, non-photorealistic visualization methods that rely on MLIC, methods that estimate normal map from MLIC and we point out visualization tools used to do MLIC analysis. In chapter 3 we propose novel benchmark datasets (RealRTI, SynthRTI and SynthPS) that can be used to evaluate algorithms that rely on MLIC and discusses available benchmark for validation of photometric algorithms that can be also used to validate other MLIC-based algorithms. In chapter 4, we evaluate the performance of different photometric stereo algorithms using SynthPS for cultural heritage applications. RealRTI and SynthRTI have been used to evaluate the performance of (Neural)RTI method. Then, in chapter 5, we present a neural network-based RTI method, aka NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. In this method using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, particularly in the case of challenging glossy materials. Finally, in chapter 6, we present a method for the detection of crack on the surface of paintings from multi-light image acquisitions and that can be used as well on single images and conclude our presentation

On-the-Go Reflectance Transformation Imaging with Ordinary Smartphones

Reflectance Transformation Imaging (RTI) is a popular technique that allows the recovery of per-pixel reflectance information by capturing an object under different light conditions. This can be later used to reveal surface details and interactively relight the subject. Such process, however, typically requires dedicated hardware setups to recover the light direction from multiple locations, making the process tedious when performed outside the lab. We propose a novel RTI method that can be carried out by recording videos with two ordinary smartphones. The flash led-light of one device is used to illuminate the subject while the other captures the reflectance. Since the led is mounted close to the camera lenses, we can infer the light direction for thousands of images by freely moving the illuminating device while observing a fiducial marker surrounding the subject. To deal with such amount of data, we propose a neural relighting model that reconstructs object appearance for arbitrary light directions from extremely compact reflectance distribution data compressed via Principal Components Analysis (PCA). Experiments shows that the proposed technique can be easily performed on the field with a resulting RTI model that can outperform state-of-the-art approaches involving dedicated hardware setups

Surface Metrology Based on Scanning Conoscopic Holography for In Situ and In-Process Monitoring of Microtexture in Paintings

In the field of engineering, surface metrology is a valuable tool codified by international standards that enables the quantitative study of small-scale surface features. However, it is not recognized as a resource in the field of cultural heritage. Motivated by this fact, in this work, we demonstrate the use and the usefulness of surface metrology based on scanning conoscopic holography for monitoring treatments on the Venetian masterpiece by Tintoretto St. Martial in Glory with the Saints Peter and Paul. We carried out in situ and in-process monitoring of the painting microtexture during an experimental, innovative laser–chemical treatment, and we performed a statistical analysis based on ISO areal field parameters. A wide and in-band roughness analysis through the complementary use of amplitude, spatial, and hybrid parameters confirmed the noninvasive nature of the whole treatment on the painting surface topography, giving us the chance to review and critically discuss the use of these parameters in a real case in heritage science

Digital Techniques for Documenting and Preserving Cultural Heritage

In this unique collection the authors present a wide range of interdisciplinary methods to study, document, and conserve material cultural heritage. The methods used serve as exemplars of best practice with a wide variety of cultural heritage objects having been recorded, examined, and visualised. The objects range in date, scale, materials, and state of preservation and so pose different research questions and challenges for digitization, conservation, and ontological representation of knowledge. Heritage science and specialist digital technologies are presented in a way approachable to non-scientists, while a separate technical section provides details of methods and techniques, alongside examples of notable applications of spatial and spectral documentation of material cultural heritage, with selected literature and identification of future research. This book is an outcome of interdisciplinary research and debates conducted by the participants of the COST Action TD1201, Colour and Space in Cultural Heritage, 2012–16 and is an Open Access publication available under a CC BY-NC-ND licence.https://scholarworks.wmich.edu/mip_arc_cdh/1000/thumbnail.jp

Digital Techniques for Documenting and Preserving Cultural Heritage

This book presents interdisciplinary approaches to the examination and documentation of material cultural heritage, using non-invasive spatial and spectral optical technologies

TECHNART 2017. Non-destructive and microanalytical techniques in art and cultural heritage. Book of abstracts

440 p.TECHNART2017 is the international biannual congress on the application of Analytical Techniques in Art and Cultural Heritage. The aim of this European conference is to provide a scientific forum to present and promote the use of analytical spectroscopic techniques in cultural heritage on a worldwide scale to stimulate contacts and exchange experiences, making a bridge between science and art. This conference builds on the momentum of the previous TECHNART editions of Lisbon, Athens, Berlin, Amsterdam and Catania, offering an outstanding and unique opportunity for exchanging knowledge on leading edge developments. Cultural heritage studies are interpreted in a broad sense, including pigments, stones, metal, glass, ceramics, chemometrics on artwork studies, resins, fibers, forensic applications in art, history, archaeology and conservation science. The meeting is focused in different aspects: - X-ray analysis (XRF, PIXE, XRD, SEM-EDX). - Confocal X-ray microscopy (3D Micro-XRF, 3D Micro-PIXE). - Synchrotron, ion beam and neutron based techniques/instrumentation. - FT-IR and Raman spectroscopy. - UV-Vis and NIR absorption/reflectance and fluorescence. - Laser-based analytical techniques (LIBS, etc.). - Magnetic resonance techniques. - Chromatography (GC, HPLC) and mass spectrometry. - Optical imaging and coherence techniques. - Mobile spectrometry and remote sensing

Aiding the conservation of two wooden Buddhist sculptures with 3D imaging and spectroscopic techniques

The conservation of Buddhist sculptures that were transferred to Europe at some point during their lifetime raises numerous questions: while these objects historically served a religious, devotional purpose, many of them currently belong to museums or private collections, where they are detached from their original context and often adapted to western taste. A scientific study was carried out to address questions from Museo d'Arte Orientale of Turin curators in terms of whether these artifacts might be forgeries or replicas, and how they may have transformed over time. Several analytical techniques were used for materials identification and to study the production technique, ultimately aiming to discriminate the original materials from those added within later interventions

Transfer of learning between screen-based and gallery-based content:an initial study

We ran a participatory design and evaluation of a paper prototype mobile application, called Digital Islam, to engage visitors and provide additional information in the British Museum’s Islamic Gallery. An evolutionary paper prototyping exercise involving 28 visitors aged 18-60 was run for 3 weeks. As visitors interacted with the paper prototype, we manipulated the prototype, observed the visitors as they interacted with it and took notes. We asked thevisitors to “think aloud” while interacting with the prototype. The prototype rapidly evolved as visitor feedback was fed back into the design. Visitors usually do not receive explicit training in the use of museum applications, instead relying on tacit training that may have positive or negative effects depending on what learning is transferred. Our study appeared to show negative transfer between visitors’ interactions with content in the gallery and content in theapplication. Visitors were asked to perform 3 tasks, finding (A) content in the gallery; (B) textual content on screen; and (C) video content on screen. Redesigns of the interface had little impact on users’ performance. The order of tasks and consequent transfer of learning between them seemed to be more important. Visitors found Task B particularly challenging when preceded by Task A. When we introduced Task C between A and B, performance on task B immediately improved: users found the on screen content more easily and faster and nolonger looked fruitlessly for it in the gallery. The study suggests that introducing additional content in mobile applications intended to improve the visitor experience can harm that experience without careful consideration of the tacit training and learning effects when combining content in the gallery and in the application