Craquelure as a Graph: Application of Image Processing and Graph Neural Networks to the Description of Fracture Patterns

Cracks on a painting is not a defect but an inimitable signature of an artwork which can be used for origin examination, aging monitoring, damage identification, and even forgery detection. This work presents the development of a new methodology and corresponding toolbox for the extraction and characterization of information from an image of a craquelure pattern. The proposed approach processes craquelure network as a graph. The graph representation captures the network structure via mutual organization of junctions and fractures. Furthermore, it is invariant to any geometrical distortions. At the same time, our tool extracts the properties of each node and edge individually, which allows to characterize the pattern statistically. We illustrate benefits from the graph representation and statistical features individually using novel Graph Neural Network and hand-crafted descriptors correspondingly. However, we also show that the best performance is achieved when both techniques are merged into one framework. We perform experiments on the dataset for paintings' origin classification and demonstrate that our approach outperforms existing techniques by a large margin.Comment: Published in ICCV 2019 Workshop

Authentication of Amadeo de Souza-Cardoso Paintings and Drawings With Deep Learning

Art forgery has a long-standing history that can be traced back to the Roman period and has become more rampant as the art market continues prospering. Reports disclosed that uncountable artworks circulating on the art market could be fake. Even some principal art museums and galleries could be exhibiting a good percentage of fake artworks. It is therefore substantially important to conserve cultural heritage, safeguard the interest of both the art market and the artists, as well as the integrity of artists’ legacies. As a result, art authentication has been one of the most researched and well-documented fields due to the ever-growing commercial art market in the past decades. Over the past years, the employment of computer science in the art world has flourished as it continues to stimulate interest in both the art world and the artificial intelligence arena. In particular, the implementation of Artificial Intelligence, namely Deep Learning algorithms and Neural Networks, has proved to be of significance for specialised image analysis. This research encompassed multidisciplinary studies on chemistry, physics, art and computer science. More specifically, the work presents a solution to the problem of authentication of heritage artwork by Amadeo de Souza-Cardoso, namely paintings, through the use of artificial intelligence algorithms. First, an authenticity estimation is obtained based on processing of images through a deep learning model that analyses the brushstroke features of a painting. Iterative, multi-scale analysis of the images is used to cover the entire painting and produce an overall indication of authenticity. Second, a mixed input, deep learning model is proposed to analyse pigments in a painting. This solves the image colour segmentation and pigment classification problem using hyperspectral imagery. The result is used to provide an indication of authenticity based on pigment classification and correlation with chemical data obtained via XRF analysis. Further algorithms developed include a deep learning model that tackles the pigment unmixing problem based on hyperspectral data. Another algorithm is a deep learning model that estimates hyperspectral images from sRGB images. Based on the established algorithms and results obtained, two applications were developed. First, an Augmented Reality mobile application specifically for the visualisation of pigments in the artworks by Amadeo. The mobile application targets the general public, i.e., art enthusiasts, museum visitors, art lovers or art experts. And second, a desktop application with multiple purposes, such as the visualisation of pigments and hyperspectral data. This application is designed for art specialists, i.e., conservators and restorers. Due to the special circumstances of the pandemic, trials on the usage of these applications were only performed within the Department of Conservation and Restoration at NOVA University Lisbon, where both applications received positive feedback.A falsificação de arte tem uma história de longa data que remonta ao período romano e tornou-se mais desenfreada à medida que o mercado de arte continua a prosperar. Relatórios revelaram que inúmeras obras de arte que circulam no mercado de arte podem ser falsas. Mesmo alguns dos principais museus e galerias de arte poderiam estar exibindo uma boa porcentagem de obras de arte falsas. Por conseguinte, é extremamente importante conservar o património cultural, salvaguardar os interesses do mercado da arte e dos artis- tas, bem como a integridade dos legados dos artistas. Como resultado, a autenticação de arte tem sido um dos campos mais pesquisados e bem documentados devido ao crescente mercado de arte comercial nas últimas décadas.Nos últimos anos, o emprego da ciência da computação no mundo da arte floresceu à medida que continua a estimular o interesse no mundo da arte e na arena da inteligência artificial. Em particular, a implementação da Inteligência Artificial, nomeadamente algoritmos de aprendizagem profunda (ou Deep Learning) e Redes Neuronais, tem-se revelado importante para a análise especializada de imagens.Esta investigação abrangeu estudos multidisciplinares em química, física, arte e informática. Mais especificamente, o trabalho apresenta uma solução para o problema da autenticação de obras de arte patrimoniais de Amadeo de Souza-Cardoso, nomeadamente pinturas, através da utilização de algoritmos de inteligência artificial. Primeiro, uma esti- mativa de autenticidade é obtida com base no processamento de imagens através de um modelo de aprendizagem profunda que analisa as características de pincelada de uma pintura. A análise iterativa e multiescala das imagens é usada para cobrir toda a pintura e produzir uma indicação geral de autenticidade. Em segundo lugar, um modelo misto de entrada e aprendizagem profunda é proposto para analisar pigmentos em uma pintura. Isso resolve o problema de segmentação de cores de imagem e classificação de pigmentos usando imagens hiperespectrais. O resultado é usado para fornecer uma indicação de autenticidade com base na classificação do pigmento e correlação com dados químicos obtidos através da análise XRF. Outros algoritmos desenvolvidos incluem um modelo de aprendizagem profunda que aborda o problema da desmistura de pigmentos com base em dados hiperespectrais. Outro algoritmo é um modelo de aprendizagem profunda estabelecidos e nos resultados obtidos, foram desenvolvidas duas aplicações. Primeiro, uma aplicação móvel de Realidade Aumentada especificamente para a visualização de pigmentos nas obras de Amadeo. A aplicação móvel destina-se ao público em geral, ou seja, entusiastas da arte, visitantes de museus, amantes da arte ou especialistas em arte. E, em segundo lugar, uma aplicação de ambiente de trabalho com múltiplas finalidades, como a visualização de pigmentos e dados hiperespectrais. Esta aplicação é projetada para especialistas em arte, ou seja, conservadores e restauradores. Devido às circunstâncias especiais da pandemia, os ensaios sobre a utilização destas aplicações só foram realizados no âmbito do Departamento de Conservação e Restauro da Universidade NOVA de Lisboa, onde ambas as candidaturas receberam feedback positivo

The brushstroke and materials of Amadeo de Souza-Cardoso combined in an authentication tool

Nowadays, authentication studies for paintings require a multidisciplinary approach, based on the contribution of visual features analysis but also on characterizations of materials and techniques. Moreover, it is important that the assessment of the authorship of a painting is supported by technical studies of a selected number of original artworks that cover the entire career of an artist. This dissertation is concerned about the work of modernist painter Amadeo de Souza-Cardoso. It is divided in three parts. In the first part, we propose a tool based on image processing that combines information obtained by brushstroke and materials analysis. The resulting tool provides qualitative and quantitative evaluation of the authorship of the paintings; the quantitative element is particularly relevant, as it could be crucial in solving authorship controversies, such as judicial disputes. The brushstroke analysis was performed by combining two algorithms for feature detection, namely Gabor filter and Scale Invariant Feature Transform. Thanks to this combination (and to the use of the Bag-of-Features model), the proposed method shows an accuracy higher than 90% in distinguishing between images of Amadeo’s paintings and images of artworks by other contemporary artists. For the molecular analysis, we implemented a semi-automatic system that uses hyperspectral imaging and elemental analysis. The system provides as output an image that depicts the mapping of the pigments present, together with the areas made using materials not coherent with Amadeo’s palette, if any. This visual output is a simple and effective way of assessing the results of the system. The tool proposed based on the combination of brushstroke and molecular information was tested in twelve paintings obtaining promising results. The second part of the thesis presents a systematic study of four selected paintings made by Amadeo in 1917. Although untitled, three of these paintings are commonly known as BRUT, Entrada and Coty; they are considered as his most successful and genuine works. The materials and techniques of these artworks have never been studied before. The paintings were studied with a multi-analytical approach using micro-Energy Dispersive X-ray Fluorescence spectroscopy, micro-Infrared and Raman Spectroscopy, micro-Spectrofluorimetry and Scanning Electron Microscopy. The characterization of Amadeo’s materials and techniques used on his last paintings, as well as the investigation of some of the conservation problems that affect these paintings, is essential to enrich the knowledge on this artist. Moreover, the study of the materials in the four paintings reveals commonalities between the paintings BRUT and Entrada. This observation is supported also by the analysis of the elements present in a photograph of a collage (conserved at the Art Library of the Calouste Gulbenkian Foundation), the only remaining evidence of a supposed maquete of these paintings. The final part of the thesis describes the application of the image processing tools developed in the first part of the thesis on a set of case studies; this experience demonstrates the potential of the tool to support painting analysis and authentication studies. The brushstroke analysis was used as additional analysis on the evaluation process of four paintings attributed to Amadeo, and the system based on hyperspectral analysis was applied on the painting dated 1917. The case studies therefore serve as a bridge between the first two parts of the dissertation

Detecting Forgery: Forensic Investigation of Documents

Detecting Forgery reveals the complete arsenal of forensic techniques used to detect forged handwriting and alterations in documents and to identify the authorship of disputed writings. Joe Nickell looks at famous cases such as Clifford Irving\u27s autobiography of Howard Hughes and the Mormon papers of document dealer Mark Hoffman, as well as cases involving works of art. Detecting Forgery is a fascinating introduction to the growing field of forensic document examination and forgery detection. Seldom does a book about forgery come along containing depth of subject matter in addition to presenting clear and understandable information. This book has both, plus a readability that is accessible to those studying questioned documents as well as seasoned experts. -- Journal of Forensic Identification The author\u27s expertise in historical documents is unmistakably evident throughout the book. Once I began reading, I found it hard to put down. -- Journal of Questioned Document Examination Guides the reader through various methods and techniques of identifying fakes and phone manuscripts. -- Manchester (KY) Enterprisehttps://uknowledge.uky.edu/upk_legal_studies/1000/thumbnail.jp

Alligatoring: An investigation into paint failure and loss of image integrity in 19th century oil paintings

"Alligatoring" or "Bitumen cracking" are terms used to describe extreme paint defects found in 19th century oil paintings. This paint failure in the form of severely disfiguring cracking and surface distor- tions often results in a loss of image integrity. This problem has been associated with the use of as- phalt/bitumen paint, with no clear understanding of the materials and mechanisms which contribute to the phenomenon. This thesis investigates this phenomenon from a multi-disciplinary approach that aims to con- tribute knowledge to the study of oil painting suffering from alligatoring. Part 1 focuses on the perception of the problem through a literature survey, and introduces the 19th century painting used in the case study. A thorough review of the literature showed the existing bias created by the perceived connection between appearance and cause, resulting in an association of paint film defects in brown paint with the use of asphalt/bitumen. Because of this, the overall focus of previous research has been the detection of asphalt, considering it in isolation and as the primary factor in the paint defect. This singular view has had significant implications in the study and analyses of paintings with alligatoring. It inhibited wider investigations and overlooked other materials present in the paintings that may be acting in combination or be more predominant in the deterioration mech- anism. The visual and chemical study of the oil painting O Cardeal D. Henrique recebendo a notícia da morte de D. Sebastião, by Marciano Henriques da Silva (1831-1873), painted in Rome in 1861, which exibits extreme alligatoring, offered specific challenges due to its complex and highly disrupted paint layer stratigraphy coupled with the uncertainty introduced by analytical detection limits. For that rea- son, a multi-analytical approach was carried out using Optical Microscopy, micro Raman Spectroscopy, Scanning Electron Microscopy with Energy Dispersive X-ray Spectrometry, X-ray Fluorescence, micro Fourier Transform Infrared Spectroscopy, Fourier Transform Raman Spectroscopy, Attenuated Total Reflection Fourier Transform Infrared Spectroscopy Microscopy, Pyrolysis-Tetramethylammonium Hy- droxide-Gas Chromatography/Mass Spectrometry and Time-of-Flight Secondary Ion Mass Spectrome- try. This research indicates that the deterioration of the painting might be related to the oil binder which has a high degree of oxidation, rich in polar fatty acids and diacids, combined with a lack of the xiv pigments that are capable of stabilizing the paint. The presence of the translucent lead azelate layer above the ground underlines the complexity of the paint layering, materials present, and possible mechanisms for degradation. Part 2 reports on new research carried out using the British colourman Winsor & Newton's 19th- century Archive Database to analyse their production records for “Bitumen” brown for oil tube paints. While there are a substantial number of published recipes in 19th century artist’s manuals for the use of asphalt/bitumen brown, for the first time W&N’s records offer a unique source of detailed infor- mation on the commercially prepared product which differs substantially in ingredients used and method of preparation. The critical analysis of W&N's production records for "Bitumen" revealed that their formulation became standardised in the 1850s and that their product was prepared in two sep- arate steps at two separate locations. Despite standarisation of the ingredients this research revealed that the company still found it necessary to make adjustments for each production run in order to achieve a uniform product. A production record from 1858 was selected and reconstructed, using where possible, histori- cally appropriate materials. W&N’s formulation for bitumen brown oil paint involved a complex mix- ture of ingredients, some of which were other proprietary products sold by the company (these were reconstructed individually using their production records and included drying oil, double mastic var- nish, lead acetate, purple lake and the gelled painter's Medium, megilp). The reconstructions were analysed using thermally assisted methylation with tetrame- thylammonium hydroxide, and pyrolysis comprehensive two-dimensional gas chromatography and compared with the starting material, Trinidad Lake asphalt, to determine how detectable this asphalt is after heat processing in lead treated linseed oil. Results show that asphalt markers identified in the Trinidad Lake asphalt disappear in the first stage of reconstructing W&N's "Bitumen" oil paint. This important finding offers an explanation for the paucity of analytical evidence in previous attempts to identify asphalt/bitumen in paintings where this material was believed to have been used. In addition to clarifying the analytical results obtained from the investigation of the painting, O Cardeal D. Henrique…, the reference samples produced from the W&N reconstruction illustrate the strengths and weaknesses of organic analysis of highly processed complex mixtures."Alligatoring" ou "Bitumen cracking" são termos utilizados para descrever defeitos extremos encon- trados em pinturas a óleo do século XIX. Esta patologia sob a forma de fissuras e distorções severas da superfície pictórica resulta frequentemente numa perda de integridade da imagem. Este problema tem sido associado à utilização de tintas de asfalto/betume, sem no entanto existir uma compreensão clara dos materiais e mecanismos que contribuem para este fenómeno. Esta tese investiga este fenómeno a partir de uma abordagem multidisciplinar que visa contri- buir para o conhecimento de pinturas a óleo afetadas por este defeito extremo. A parte 1 centra-se na perceção que existe sobre este problema através de uma pesquisa bibli- ográfica, e introduz a pintura do século XIX utilizada como caso de estudo. A revisão rigorosa da lite- ratura revelou o preconceito existente criado pela perceção da ligação entre a aparência e a causa, resultando numa associação de defeitos em filmes de tinta castanha com a utilização de tintas de as- falto/betume. Devido a isto, o foco geral de estudos anteriores foi a deteção do asfalto, considerando- o isoladamente e como a principal causa nos defeitos de tinta. Esta visão simplista tem tido implicações significativas no estudo e análise de pinturas com “alligatoring” (efeito de pele de crocodilo), inibindo investigações mais abrangentes e subvalorizando outros materiais presentes nas pinturas que podem estar a atuar em combinação ou ser mais predominantes no mecanismo de deterioração. O estudo visual e químico da pintura a óleo O Cardeal D. Henrique recebendo a notícia da morte de D. Sebastião, de Marciano Henriques da Silva (1831-1873), pintado em Roma em 1861, que exibe “alligatoring” extremo, ofereceu desafios específicos devido à sua estratigrafia complexa e muito alte- rada, aliada à incerteza introduzida pelos limites de deteção analítica. Por essa razão, foi realizada uma abordagem multi-analítica utilizando Microscopia Óptica, Micro-espectroscopia de Raman, Microsco- pia Electrónica de Varrimento com Espectrometria Dispersiva de Raios-X, Micro-espectrometria por Fluorescência de Raios X Dispersiva de Energias, Micro-espectroscopia de Infravermelho com Trans- formada de Fourier, Espectroscopia Raman com Transformada de Fourier, Espectroscopia de Infraver- melho com Transformada de Fourier por Reflexão Total Atenuada acoplada a Microscopia Óptica, Pi- rólise-Cromatografia Gasosa acoplada à Espectrometria de Massa com derivatização usando Hidróxido de Tetrametilamónio e Espectrometria de Massa de iões secundários. Esta investigação indica que a deterioração da pintura pode estar relacionada com o aglutinante de óleo altamente hidrolisado, rico em ácidos gordos polares e diácidos, combinado com a falta de pigmentos capazes de estabilizar a tinta. A presença da camada translúcida de azelato de chumbo sobre a camada de preparação sublinha a complexidade da estratigrafia, dos materiais presentes, e dos possíveis mecanismos de degradação. A Parte 2 descreve a analise dos registos de produção de "Betume" castanho para tintas em tubos de óleo do arquivo e base de dados do século XIX da Winsor & Newton (fabricante de materiais para artistas britânico). Embora exista um número substancial de receitas de tintas castanhas de as- falto/betume publicadas em manuais para artistas do século XIX, os registos da W&N são uma fonte de informação detalhada e única sobre o produto preparado comercialmente que difere substancial- mente nos ingredientes utilizados e no método de preparação. A análise crítica dos registos de produ- ção da W&N para "Betume" revelou que a sua formulação foi normalizada na década de 1850 e que era preparado em duas etapas distintas e em dois locais distintos. Apesar da padronização dos ingre- dientes, esta investigação revelou que a empresa comsiderava necessário fazer ajustes em cada etapa de produção, a fim de obter um produto uniforme. Um registo de produção de 1858 foi selecionado e reconstruído, utilizando, sempre que possí- vel, materiais historicamente apropriados. A formulação da W&N para a tinta castanha de betume a óleo envolveu uma mistura complexa de ingredientes, alguns dos quais eram produtos produzidos e vendidos pela W&N. Estes foram reconstruídos individualmente, utilizando os seus próprios registos de produção, e incluíram óleo de secagem, verniz mástique duplo, acetato de chumbo, pigmento laca roxo, e o gel tixotrópico Megilp). As reconstruções foram analisadas utilizando Pirólise-Cromatografia Gasosa acoplada à Espec- trometria de Massa com derivatização usando Hidróxido de Tetrametilamónio, e Pirólise por Croma- tografia Gasosa Bidimensional acoplada à Espectrometria de Massa. As reconstruções foram compa- radas com a materia prima utilizada, o asfalto do Lago Trinidad, para determinar o grau de deteção deste asfalto após o processamento térmico em óleo de linhaça tratado com chumbo. Os resultados mostram que os marcadores asfálticos identificados no asfalto do Lago de Trinidad desaparecem na primeira fase da reconstrução da tinta a óleo "Betume" da W&N. Esta descoberta importante oferece uma explicação para a escassez de provas analíticas em tentativas anteriores de identificação do asfalto/betume em pinturas em que se acreditava ter sido utilizado este material. Para além de clarificar os resultados analíticos obtidos a partir da investigação da pintura O Cardeal D. Henrique..., as amostras de referência produzidas a partir da reconstrução da W&N ilustram os pontos fortes e fracos da análise orgânica de misturas complexas altamente proces- sadas

Ambrose Bierce is Missing: And Other Historical Mysteries

What constitutes historical truth is often subject to change. Through ingenious detection, the accepted wisdom of one generation may become the discredited legend of another—or vice versa. In this wide- ranging study of historical investigation, former detective Joe Nickell allows the reader to look over his shoulder as he demonstrates the use of varied techniques in solving some of the world\u27s most perplexing mysteries. All the major categories of historical mystery are here—ancient riddles, biographical enigmas, hidden identity, “fakelore,” questioned artifacts, suspect documents, lost texts, obscured sources, and scientific challenges. Each is then illustrated by a complete case from the author\u27s own files. Nickell’s investigation of the giant Nazca drawings in Peru, for example—thought by some to provide proof of ancient extraterrestrial visitations—uses innovative techniques to reveal a very different origin. Other cases concern the 1913 disappearance of writer and journalist Ambrose Bierce, the authenticity of the Shroud of Turin, the truth about the identity of John Demjanjuk (“Ivan the Terrible” to Polish death camp victims), the fate of a lost colonial American text, the authenticity of Abraham Lincoln\u27s celebrated Bixby letter, and the apparent real-life model for a mysterious character in a novel by Nathaniel Hawthorne. In reaching his solutions, Nickell demonstrates a wide variety of investigative techniques—chemical and instrumental analyses, physical experimentation, a “psychological autopsy,” forensic identification, archival research, linguistic analysis, folklore study, and many others. His highly readable book will intrigue the scholar and the history buff no less than the mystery lover. Joe Nickell, a former investigator for a world-famous detective agency, teaches at the University of Kentucky and is author of several books, including Pen, Ink, and Evidence.https://uknowledge.uky.edu/upk_history_in_general/1002/thumbnail.jp

Digital 3D Technologies for Humanities Research and Education: An Overview

Digital 3D modelling and visualization technologies have been widely applied to support research in the humanities since the 1980s. Since technological backgrounds, project opportunities, and methodological considerations for application are widely discussed in the literature, one of the next tasks is to validate these techniques within a wider scientific community and establish them in the culture of academic disciplines. This article resulted from a postdoctoral thesis and is intended to provide a comprehensive overview on the use of digital 3D technologies in the humanities with regards to (1) scenarios, user communities, and epistemic challenges; (2) technologies, UX design, and workflows; and (3) framework conditions as legislation, infrastructures, and teaching programs. Although the results are of relevance for 3D modelling in all humanities disciplines, the focus of our studies is on modelling of past architectural and cultural landscape objects via interpretative 3D reconstruction methods

Fine Art Pattern Extraction and Recognition

This is a reprint of articles from the Special Issue published online in the open access journal Journal of Imaging (ISSN 2313-433X) (available at: https://www.mdpi.com/journal/jimaging/special issues/faper2020)