Neural Networks for Hyperspectral Imaging of Historical Paintings: A Practical Review

Hyperspectral imaging (HSI) has become widely used in cultural heritage (CH). This very efficient method for artwork analysis is connected with the generation of large amounts of spectral data. The effective processing of such heavy spectral datasets remains an active research area. Along with the firmly established statistical and multivariate analysis methods, neural networks (NNs) represent a promising alternative in the field of CH. Over the last five years, the application of NNs for pigment identification and classification based on HSI datasets has drastically expanded due to the flexibility of the types of data they can process, and their superior ability to extract structures contained in the raw spectral data. This review provides an exhaustive analysis of the literature related to NNs applied for HSI data in the CH field. We outline the existing data processing workflows and propose a comprehensive comparison of the applications and limitations of the various input dataset preparation methods and NN architectures. By leveraging NN strategies in CH, the paper contributes to a wider and more systematic application of this novel data analysis method

3D printing of oil paintings based on material jetting and its reduction of staircase effect

Material jetting is a high-precision and fast 3D printing technique for color 3D objects reproduction, but it also suffers from color accuracy and jagged issues. The UV inks jetting processes based on the polymer jetting principle have been studied from printing materials regarding the parameters in the default layer order, which is prone to staircase effects. In this work, utilizing the Mimaki UV inks jetting system with a variable layer thickness, a new framework to print a photogrammetry-based oil painting 3D model has been proposed with the tunable coloring layer sequence to improve the jagged challenge between adjacent layers. Based on contour tracking, a height-rendering image of the oil painting model is generated, which is further segmented and pasted to the corresponding slicing layers to control the overall printing sequence of coloring layers and white layers. The final results show that photogrammetric models of oil paintings can be printed vividly by UV-curable color polymers, and that the proposed reverse-sequence printing method can significantly improve the staircase effect based on visual assessment and color difference. Finally, the case of polymer-based oil painting 3D printing provides new insights for optimizing color 3D printing processes based on other substrates and print accuracy to improve the corresponding staircase effect

Prototype software for colorant formulation using Gamblin conservation colors

When selecting pigments from a large set for restorative inpainting, it can often be challenging to create a mixture that will provide an exact match to the original artwork under a range of viewing and illumination conditions. In this research, a prototype computer program was developed that will aid the user by providing a color match and paint recipe that exhibits minimal metamerism when compared to the original artwork. The Gamblin Conservation Colors, a set of 43 colorants specially formulated for inpainting, were characterized in terms of their optical properties, absorption and scattering, according to Kubelka-Munk turbid media theory. Formulations were made using traditional spectrophotometric measurements and image-based measurements. The multispectral imaging system consisted of a trichromatic CFA camera coupled with two absorption filters; spectral reflectance data for each pixel location was estimated with a transformation based on calibration target images. Three targets were used for testing formulation accuracy: a target consisting of mixtures of Gamblin Conservation Colors, and two oil paintings. Pigment selection was reasonably successful, and good predictions resulted from both measurement techniques, but for more complex tasks such as pigment identification, a more rigorous colorant characterization approach may be needed. Predictions from image-based measurements were generally less accurate, and improvements in the camera model would likely remedy this. It is expected that this software will be of assistance to conservators by simplifying the process of selecting from a large set of available pigments, as well as reducing the possibility of damage to painted surfaces in cases where direct measurements are impractical. The open source nature of the software provides the opportunity for changes and addition of features in the future

Image segmentation and pigment mapping of cultural heritage based on spectral imaging

The goal of the work reported in this dissertation is to develop methods for image segmentation and pigment mapping of paintings based on spectral imaging. To reach this goal it is necessary to achieve sufficient spectral and colorimetric accuracies of both the spectral imaging system and pigment mapping. The output is a series of spatial distributions of pigments (or pigment maps) composing a painting. With these pigment maps, the change of the color appearance of the painting can be simulated when the optical properties of one or more pigments are altered. These pigment maps will also be beneficial for enriching the historical knowledge of the painting and aiding conservators in determining the best course for retouching damaged areas of the painting when metamerism is a factor. First, a new spectral reconstruction algorithm was developed based on Wyszecki’s hypothesis and the matrix R theory developed by Cohen and Kappauf. The method achieved both high spectral and colorimetric accuracies for a certain combination of illuminant and observer. The method was successfully tested with a practical spectral imaging system that included a traditional color-filter-array camera coupled with two optimized filters, developed in the Munsell Color Science Laboratory. The spectral imaging system was used to image test paintings, and the method was used to retrieve spectral reflectance factors for these paintings. Next, pigment mapping methods were brought forth, and these methods were based on Kubelka-Munk (K-M) turbid media theory that can predict spectral reflectance factor for a specimen from the optical properties of the specimen’s constituent pigments. The K-M theory has achieved practical success for opaque materials by reduction in mathematical complexity and elimination of controlling thickness. The use of the general K-M theory for the translucent samples was extensively studied, including determination of optical properties of pigments as functions of film thickness, and prediction of spectral reflectance factor of a specimen by selecting the right pigment combination. After that, an investigation was carried out to evaluate the impact of opacity and layer configuration of a specimen on pigment mapping. The conclusions were drawn from the comparisons of prediction accuracies of pigment mapping between opaque and translucent assumption, and between single and bi-layer assumptions. Finally, spectral imaging and pigment mapping were applied to three paintings. Large images were first partitioned into several small images, and each small image was segmented into different clusters based on either an unsupervised or supervised classification method. For each cluster, pigment mapping was done pixel-wise with a limited number of pigments, or with a limited number of pixels and then extended to other pixels based on a similarity calculation. For the masterpiece The Starry Night, these pigment maps can provide historical knowledge about the painting, aid conservators for inpainting damaged areas, and digitally rejuvenate the original color appearance of the painting (e.g. when the lead white was not noticeably darkened)

Artificial Intelligence as a Substitute for Human Creativity

Creativity has always been perceived as a human trait, even though the exact neural mechanisms remain unknown, it has been the subject of research and debate for a long time. The recent development of AI technologies and increased interest in AI has led to many projects capable of performing tasks that have been previously regarded as impossible without human creativity. Music composition, visual arts, literature, and science represent areas in which these technologies have started to both help and replace the creative human, with the question of whether AI can be creative and capable of creation more realistic than ever. This review aims to provide an extensive perspective over several state-of-the art technologies and applications based on AI which are currently being implemented into areas of interest closely correlated to human creativity, as well as the economic impact the development of such technologies might have on those domains

The development of multi-channel inkjet printing methodologies for fine art applications

This thesis contributes to the defence of the practitioner perspective as a means of undertaking problems addressed predominantly in the field of colour science. Whilst artists have been exploring the use of colour for centuries through their personal practice and education, the rise of industrialised printing processes has generated a shift in focus away from these creative pursuits and into the computational field of colour research. It is argued here that the disposition and knowledge generated by creative practice has significant value to offer developing technologies. While creative practice has limited influence in the development of colour printing, practitioners and users of technology actively engage with the process in ways that extend beyond its intended uses in order to overcome recognised shortcomings. Here consideration is given to this creative engagement as motivation to develop bespoke printing parameters that demonstrate the effects of colour mixing through methods alternative to standard workflows. The research is undertaken incorporating both qualitative and quantitative analysis, collecting data from visual assessments and by examining spectral measurements taken from printed output. Action research is employed to directly access and act upon the constant developments in the art and science disciplines related to inkjet printing, observing and engaging with current methods and techniques employed by practitioners and developers. This method of research has strongly informed the empirical testing that has formed this thesis’s contribution to fine art inkjet printing practice. The research follows a practitioner led approach to designing and testing alternative printing methods and is aimed at expanding the number of discernible colours an inkjet printer can reproduce. The application of this methodology is evidenced through demonstrative prints and a reproduction study undertaken at the National Gallery, London. The experimentation undertaken in partnership with the National Gallery has proven the ability to increase accuracy between colour measured from the original target and reproduction, beyond the capabilities of current inkjet printing workflows

Aiding the conservation of two wooden Buddhist sculptures with 3D imaging and spectroscopic techniques

The conservation of Buddhist sculptures that were transferred to Europe at some point during their lifetime raises numerous questions: while these objects historically served a religious, devotional purpose, many of them currently belong to museums or private collections, where they are detached from their original context and often adapted to western taste. A scientific study was carried out to address questions from Museo d'Arte Orientale of Turin curators in terms of whether these artifacts might be forgeries or replicas, and how they may have transformed over time. Several analytical techniques were used for materials identification and to study the production technique, ultimately aiming to discriminate the original materials from those added within later interventions

TECHNART 2017. Non-destructive and microanalytical techniques in art and cultural heritage. Book of abstracts

440 p.TECHNART2017 is the international biannual congress on the application of Analytical Techniques in Art and Cultural Heritage. The aim of this European conference is to provide a scientific forum to present and promote the use of analytical spectroscopic techniques in cultural heritage on a worldwide scale to stimulate contacts and exchange experiences, making a bridge between science and art. This conference builds on the momentum of the previous TECHNART editions of Lisbon, Athens, Berlin, Amsterdam and Catania, offering an outstanding and unique opportunity for exchanging knowledge on leading edge developments. Cultural heritage studies are interpreted in a broad sense, including pigments, stones, metal, glass, ceramics, chemometrics on artwork studies, resins, fibers, forensic applications in art, history, archaeology and conservation science. The meeting is focused in different aspects: - X-ray analysis (XRF, PIXE, XRD, SEM-EDX). - Confocal X-ray microscopy (3D Micro-XRF, 3D Micro-PIXE). - Synchrotron, ion beam and neutron based techniques/instrumentation. - FT-IR and Raman spectroscopy. - UV-Vis and NIR absorption/reflectance and fluorescence. - Laser-based analytical techniques (LIBS, etc.). - Magnetic resonance techniques. - Chromatography (GC, HPLC) and mass spectrometry. - Optical imaging and coherence techniques. - Mobile spectrometry and remote sensing

Geometry-Aware Scattering Compensation for 3D Printing

Commercially available full-color 3D printing allows for detailed control of material deposition in a volume, but an exact reproduction of a target surface appearance is hampered by the strong subsurface scattering that causes nontrivial volumetric cross-talk at the print surface. Previous work showed how an iterative optimization scheme based on accumulating absorptive materials at the surface can be used to find a volumetric distribution of print materials that closely approximates a given target appearance. // In this work, we first revisit the assumption that pushing the absorptive materials to the surface results in minimal volumetric cross-talk. We design a full-fledged optimization on a small domain for this task and confirm this previously reported heuristic. Then, we extend the above approach that is critically limited to color reproduction on planar surfaces, to arbitrary 3D shapes. Our proposed method enables high-fidelity color texture reproduction on 3D prints by effectively compensating for internal light scattering within arbitrarily shaped objects. In addition, we propose a content-aware gamut mapping that significantly improves color reproduction for the pathological case of thin geometric features. Using a wide range of sample objects with complex textures and geometries, we demonstrate color reproduction whose fidelity is superior to state-of-the-art drivers for color 3D printers

Modeling and Halftoning for Multichannel Printers: A Spectral Approach

Printing has been has been the major communication medium for many centuries. In the last twenty years, multichannel printing has brought new opportunities and challenges. Beside of extended colour gamut of the multichannel printer, the opportunity was presented to use a multichannel printer for ‘spectral printing’. The aim of spectral printing is typically the same as for colour printing; that is, to match input signal with printing specific ink combinations. In order to control printers so that the combination or mixture of inks results in specific colour or spectra requires a spectral reflectance printer model that estimates reflectance spectra from nominal dot coverage. The printer models have one of the key roles in accurate communication of colour to the printed media. Accordingly, this has been one of the most active research areas in printing. The research direction was toward improvement of the model accuracy, model simplicity and toward minimal resources used by the model in terms of computational power and usage of material. The contribution of the work included in the thesis is also directed toward improvement of the printer models but for the multichannel printing. The thesis is focused primarily on improving existing spectral printer models and developing a new model. In addition, the aim was to develop and implement a multichannel halftoning method which should provide with high image quality. Therefore, the research goals of the thesis were: maximal accuracy of printer models, optimal resource usage and maximal image quality of halftoning and whole spectral reproduction system. Maximal colour accuracy of a model but with the least resources used is achieved by optimizing printer model calibration process. First, estimation of the physical and optical dot gain is performed with newly proposed method and model. Second, a custom training target is estimated using the proposed new method. These two proposed methods and one proposed model were at the same time the means of optimal resource usage, both in computational time and material. The third goal was satisfied with newly proposed halftoning method for multichannel printing. This method also satisfies the goal of optimal computational time but with maintaining high image quality. When applied in spectral reproduction workflow, this halftoning reduces noise induced in an inversion of the printer model. Finally, a case study was conducted on the practical use of multichannel printers and spectral reproduction workflow. In addition to a gamut comparison in colour space, it is shown that otherwise limited reach of spectral printing could potentially be used to simulate spectra and colour of textile fabrics