Macroscale multimodal imaging reveals ancient painting production technology and the vogue in Greco-Roman Egypt.

Macroscale multimodal chemical imaging combining hyperspectral diffuse reflectance (400-2500 nm), luminescence (400-1000 nm), and X-ray fluorescence (XRF, 2 to 25 keV) data, is uniquely equipped for noninvasive characterization of heterogeneous complex systems such as paintings. Here we present the first application of multimodal chemical imaging to analyze the production technology of an 1,800-year-old painting and one of the oldest surviving encaustic ("burned in") paintings in the world. Co-registration of the data cubes from these three hyperspectral imaging modalities enabled the comparison of reflectance, luminescence, and XRF spectra at each pixel in the image for the entire painting. By comparing the molecular and elemental spectral signatures at each pixel, this fusion of the data allowed for a more thorough identification and mapping of the painting's constituent organic and inorganic materials, revealing key information on the selection of raw materials, production sequence and the fashion aesthetics and chemical arts practiced in Egypt in the second century AD

SEMANTIC AND ABSTRACTION CONTENT OF ART IMAGES

In this paper the semantic and abstraction content of art images is studied. Different techniques for search in art image repositories are analyzed and new ones are proposed. The content-based retrieval process integrates the search on different components, linked in XML structures. Some experiments over 200 paintings of six Israel contemporary artists are done and analyzed

KNOWLEDGE AND DOCUMENTATION OF RENAISSANCE WORKS OF ART: THE REPLICA OF THE “ANNUNCIATION” BY BEATO ANGELICO

The Annunciation by Guido di Pietro from Mugello, known as Beato Angelico, is a wide tempera painting with some fine gold foil placed on a wooden support, today hosted at the Museum of the Basilica of Santa Maria delle Grazie, in San Giovanni Valdarno. On the occasion of the exhibition “Masaccio e Angelico. Dialogo sulla verità nella pittura”, the museum asked to the Department of Architecture at the University of Bologna to develop a digital high-resolution surrogate to favour deep investigations, to plan restoration and to simply tell the stories behind the artwork. Two tasks were accomplished: to let visitors discover the secrets in the painting and to let scholars study the artwork, to better understand the masterpiece. This paper introduces the outcomes of the research developed to digitize the Annunciation, following a dedicated pipeline developed to improve the fruition of its digital replica, originated from different input sources, and surrogating the user experience on the real object. This work presents a method for the 3D reconstruction of the surfaces based on different techniques for elements with different depth resolutions (i.e., the painting and the wooden frame) which combine photogrammetry and photometric stereo exploiting both procedures and pushing forward the boundaries of Gigapixel Imaging and photogrammetric-based 3D model representation

Visual complexity modelling based on image features fusion of multiple kernels

[Abstract] Humans’ perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind. For these reasons, creating accurate computational models of visual complexity is a demanding task. Building upon on previous work in the field (Forsythe et al., 2011; Machado et al., 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity. For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf’s law. In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed. Subsequently, we conduct an exhaustive outlier analysis. We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple Kernel Learning obtains the best results, yielding an average correlation to humans’ perception of complexity of 0.71 with only twenty-two features. These results outperform the current state-of-the-art, showing the potential of this technique for regression.Xunta de Galicia; GRC2014/049Portuguese Foundation for Science and Technology; SBIRC; PTDC/EIA EIA/115667/2009Xunta de Galicia; Ref. XUGA-PGIDIT-10TIC105008-PRMinisterio de Ciencia y Tecnología; TIN2008-06562/TINMinisterio de Ecnomía y Competitividad; FJCI-2015-2607

Image and Video Processing for Cultural Heritage

Charvillat V., Tonazzini A., Van Gool L., Nikolaidis N., ''Editorial: Image and video processing for cultural heritage'', EURASIP journal on image and video processing, vol. 2009, Article ID 163064, 3 pp., 2010.status: publishe

Reflectance Transformation Imaging (RTI) System for Ancient Documentary Artefacts

This tutorial summarises our uses of reflectance transformation imaging in archaeological contexts. It introduces the UK AHRC funded project reflectance Transformation Imaging for Anciant Documentary Artefacts and demonstrates imaging methodologies

Virtual Cleaning of Works of Art Using Deep Learning Based Approaches

Virtual cleaning of art is a key process that conservators apply to see the likely appearance of the work of art they have aimed to clean, before the process of cleaning. There have been many different approaches to virtually clean artworks but having to physically clean the artwork at a few specific places of specific colors, the need to have pure black and white paint on the painting and their low accuracy are only a few of their shortcomings prompting us to propose deep learning based approaches in this research. First we report the work we have done in this field focusing on the color estimation of the artwork virtual cleaning and then we describe our methods for the spectral reflectance estimation of artwork in virtual cleaning. In the color estimation part, a deep convolutional neural network (CNN) and a deep generative network (DGN) are suggested, which estimate the RGB image of the cleaned artwork from an RGB image of the uncleaned artwork. Applying the networks to the images of the well-known artworks (such as the Mona Lisa and The Virgin and Child with Saint Anne) and Macbeth ColorChecker and comparing the results to the only physics-based model (which is the first model that has approached the issue of virtual cleaning from the physics-point of view, hence our reference to compare our models with) shows that our methods outperform that model and have great potentials of being applied to the real situations in which there might not be much information available on the painting, and all we have is an RGB image of the uncleaned artwork. Nonetheless, the methods proposed in the first part, cannot provide us with the spectral reflectance information of the artwork, therefore, the second part of the dissertation is proposed. This part focuses on the spectral estimation of the artwork virtual cleaning. Two deep learning-based approaches are also proposed here; the first one is deep generative network. This method receives a cube of the hyperspectral image of the uncleaned artwork and tries to output another cube which is the virtually cleaned hyperspectral image of the artwork. The second approach is 1D Convolutional Autoencoder (1DCA), which is based on 1D convolutional neural network and tries to find the spectra of the virtually cleaned artwork using the spectra of the physically cleaned artworks and their corresponding uncleaned spectra. The approaches are applied to hyperspectral images of Macbeth ColorChecker (simulated in the forms of cleaned and uncleaned hyperspectral images) and the \u27Haymakers\u27 (real hyperspectral images of both cleaned and uncleaned states). The results, in terms of Euclidean distance and spectral angle between the virtually cleaned artwork and the physically cleaned one, show that the proposed approaches have outperformed the physics-based model, with DGN outperforming the 1DCA. Methods proposed herein do not rely on finding a specific type of paint and color on the painting first and take advantage of the high accuracy offered by deep learning-based approaches and they are also applicable to other paintings

Reconstructing Van Gogh’s palette to determine the optical characteristics of his paints

The colors of Field with Irises near Arles, painted by Van Gogh in Arles in 1888, have changed considerably. To get an idea of how this painting, as well as other works by Van Gogh, looked shortly after their production, the Revigo (Re-assessing Vincent van Gogh’s colors) research project was initiated. The aim of this project was to digitally visualize the original colors of paintings and drawings by Vincent van Gogh, using scientific methods backed by expert judgement where required. We adopted an experimental art technological approach and physically reconstructed Van Gogh’s full palette of oil paints, closely matching those he used to paint Field with Irises near Arles. Sixteen different paints were reconstructed, among which the most light-sensitive pigments and linseed oil, which is prone to yellowing, were produced according to 19th century practice. The resulting pigments and oils were chemically analyzed and compared to those used by Van Gogh. The ones that resembled his paints the most were used in the paint reconstructions. Other pigments were either obtained from the Cultural Heritage Agency’s collection of historical pigments, or purchased from Kremer Pigmente. The reconstructed paints were subsequently used to calculate the absorption K and scattering S parameters of the individual paints. Using Kubelka–Munk theory, these optical parameters could in turn be used to determine the color of paint mixtures. We applied this method successfully to digitally visualize the original colors of Field with Irises near Arles. Moreover, the set of optical parameters presented here can similarly be applied to calculate digital visualizations of other paintings by Van Gogh and his contemporaries