A DICOM-Inspired Metadata Architecture for Managing Multimodal Acquisitions in Cultural Heritage

Quantitative and qualitative analyses of cultural heritage (CH) assets need to interconnect individual pieces of information, including a variety of multimodal acquisitions, to form a holistic compounded view of studied objects. The need for joint acquisition brings with it the requirement for defining a protocol to store, structure and support the interoperability of the multisource data. In our work, we are performing multiple imaging studies in order to analyze the material, to monitor the behavior and to diagnose the status of CH objects. In particular, we employ, in addition to coarse 3D scanning, two high-resolution surface data capture techniques: reflectance transformation imaging and microprofilometry. Given this multivariate input, we have defined a hierarchical data organization, similar to the one used in the medical field by the Digital Imaging and Communications in Medicine (DICOM) protocol, that supports pre-alignment of local patches with respect to a global model. Furthermore, we have developed two supporting tools for multi-modal data handling: one for metadata annotation and another one for image registration. In this work, we illustrate our approach and discuss its practical application in a case study on a real CH object - a bronze bas-relief.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509

Aging Prediction of Cultural Heritage Samples Based on Surface Microgeometry

A critical and challenging aspect for the study of Cultural Heritage (CH) assets is related to the characterization of the materials that compose them and to the variation of these materials with time. In this paper, we exploit a realistic dataset of artificially aged metallic samples treated with different coatings commonly used for artworks' protection in order to evaluate different approaches to extract material features from high-resolution depth maps. In particular, we estimated, on microprofilometric surface acquisitions of the samples, performed at different aging steps, standard roughness descriptors used in materials science as well as classical and recent image texture descriptors. We analyzed the ability of the features to discriminate different aging steps and performed supervised classification tests showing the feasibility of a texture-based aging analysis and the effectiveness of coatings in reducing the surfaces' change with time.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509

Techniques for seamless color registration and mapping on dense 3D models

Today’s most widely used 3D digitization approach is a combination of active geometric sensing, mainly using laser scanning, with active or passive color sensing, mostly using digital photography. Producing a seamless colored object, starting from a geometric representation and a set of photographs, is a data fusion problem requiring effective solutions for image-to-geometry registration, and color mapping and blending. This chapter provides a brief survey of the state-of-the-art solutions, ranging from manual approaches to fully scalable automated methods.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509

Mobile Graphics

The increased availability and performance of mobile graphics terminals, including smartphones and tablets with high resolution screens and powerful GPUs, combined with the increased availability of high-speed mobile data connections, is opening the door to a variety of networked graphics applications. In this world, native apps or mobile sites coexist to reach the goal of providing us access to a wealth of multimedia information while we are on the move. This half-day tutorial provides a technical introduction to the mobile graphics world spanning the hardware-software spectrum, and explores the state of the art and key advances in specific application domains, including capture and acquisition, real-time high-quality 3D rendering and interactive exploration.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091Sardinian Regional Authorities under projects VIGEC and VIDEOLABKing Abdullah University of Science and Technolog

A Novel Framework for Highlight Reflectance Transformation Imaging

We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091DSURF project (PRIN 2015) funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa

Multispectral RTI Analysis of Heterogeneous Artworks

We propose a novel multi-spectral reflectance transformation imaging (MS-RTI) framework for the acquisition and direct analysis of the reflectance behavior of heterogeneous artworks. Starting from free-form acquisitions, we compute per-pixel calibrated multi-spectral appearance profiles, which associate a reflectance value to each sampled light direction and frequency. Visualization, relighting, and feature extraction is performed directly on appearance profile data, applying scattered data interpolation based on Radial Basis Functions to estimate per-pixel reflectance from novel lighting directions. We demonstrate how the proposed solution can convey more insights on the object materials and geometric details compared to classical multi-light methods that rely on low-frequency analytical model fitting eventually mixed with a separate handling of high-frequency components, hence requiring constraining priors on material behavior. The flexibility of our approach is illustrated on two heterogeneous case studies, a painting and a dark shiny metallic sculpture, that showcase feature extraction, visualization, and analysis of high-frequency properties of artworks using multi-light, multi-spectral (Visible, UV and IR) acquisitions.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091the DSURF (PRIN 2015) project funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa

Automatic Algorithms for Medieval Manuscript Analysis

Massive digital acquisition and preservation of deteriorating historical and artistic documents is of particular importance due to their value and fragile condition. The study and browsing of such digital libraries is invaluable for scholars in the Cultural Heritage field, but requires automatic tools for analyzing and indexing these datasets. We will describe a set of completely automatic solutions to estimate per-page text leading, to extract text lines, blocks and other layout elements, and to perform query-by-example word-spotting on medieval manuscripts. Those techniques have been evaluated on a huge heterogeneous corpus of illuminated medieval manuscripts of different writing styles, languages, image resolutions, amount of illumination and ornamentation, and levels of conservation, with various problematic issues such as holes, spots, ink bleed-through, ornamentation, and background noise. We also present a quantitative analysis to better assess the quality of the proposed algorithms. By not requiring any human intervention to produce a large amount of annotated training data, the developed methods provide Computer Vision researchers and Cultural Heritage practitioners with a compact and efficient system for document analysis.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509

A Practical Reflectance Transformation Imaging Pipeline for Surface Characterization in Cultural Heritage

We present a practical acquisition and processing pipeline to characterize the surface structure of cultural heritage objects. Using a free-form Reflectance Transformation Imaging (RTI) approach, we acquire multiple digital photographs of the studied object shot from a stationary camera. In each photograph, a light is freely positioned around the object in order to cover a wide variety of illumination directions. Multiple reflective spheres and white Lambertian surfaces are added to the scene to automatically recover light positions and to compensate for non-uniform illumination. An estimation of geometry and reflectance parameters (e.g., albedo, normals, polynomial texture maps coefficients) is then performed to locally characterize surface properties. The resulting object description is stable and representative enough of surface features to reliably provide a characterization of measured surfaces. We validate our approach by comparing RTI-acquired data with data acquired with a high-resolution microprofilometer.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509