CULTURAL HERITAGE RECONSTRUCTION FROM HISTORICAL PHOTOGRAPHS AND VIDEOS

Historical archives save invaluable treasures and play a critical role in the conservation of Cultural Heritage. Old photographs and videos, which have survived over time and stored in these archives, preserve traces of architecture and urban transformation and, in many cases, are the only evidence of buildings that no longer exist. They are a precious source of enormous informative potential in Cultural Heritage documentation and save invaluable treasures. Thanks to photogrammetric techniques it is possible to extract metric information from these sources useful for 3D virtual reconstructions of monuments and historic buildings. This paper explores the ways to search for, classify and group historical data by considering their possible use in metric documentation and aims to provide an overview of criticality and open issues of the methodologies that could be used to process these data. A practical example is described and presented as a case study. The video "Torino 1928", an old movie dating from the 1930s, was processed for reconstructing the temporary pavilions of the "Exposition" held in Turin in 1928. Despite the initial concerns relating to processing this kind of data, the experimental methodology used in this research has allowed to reach a quality of results of acceptable standard

A COMPARISON BETWEEN 3D RECONSTRUCTION USING NERF NEURAL NETWORKS AND MVS ALGORITHMS ON CULTURAL HERITAGE IMAGES

In this research, an innovative comparison between 3D reconstructions obtained by means of Artificial Intelligence, in particular NeRF Neural Networks, and by Structure-from-Motion (SfM) and Multi-View-Stereo (MVS) open-source algorithms is proposed. The 3D reconstruction comparison is performed on two test cases, one of cultural interest, one useful only for technical discussion. It is known that the approaches are traditionally used with different objectives and in different contexts but they can however also be used with similar purpose, i.e., 3D reconstruction. In particular, we were interested in evaluating how NeRF reconstructions are accurate from a metric point of view and how the models obtained from the application of NeRF differ from the model obtained from the classical photogrammetry. By analyzing the results in the considered test cases, we show how NeRF networks, although computationally demanding, can be an interesting alternative or complementary methodology, especially in cases where classical photogrammetric techniques do not allow satisfactory results to be achieved. It is therefore suggested to expand efforts in this direction by exploiting, for example, the numerous improvement proposals of the original NeRF network

A COMPARISON BETWEEN 3D RECONSTRUCTION USING NERF NEURAL NETWORKS AND MVS ALGORITHMS ON CULTURAL HERITAGE IMAGES

In this research, an innovative comparison between 3D reconstructions obtained by means of Artificial Intelligence, in particular NeRF Neural Networks, and by Structure-from-Motion (SfM) and Multi-View-Stereo (MVS) open-source algorithms is proposed. The 3D reconstruction comparison is performed on two test cases, one of cultural interest, one useful only for technical discussion. It is known that the approaches are traditionally used with different objectives and in different contexts but they can however also be used with similar purpose, i.e., 3D reconstruction. In particular, we were interested in evaluating how NeRF reconstructions are accurate from a metric point of view and how the models obtained from the application of NeRF differ from the model obtained from the classical photogrammetry. By analyzing the results in the considered test cases, we show how NeRF networks, although computationally demanding, can be an interesting alternative or complementary methodology, especially in cases where classical photogrammetric techniques do not allow satisfactory results to be achieved. It is therefore suggested to expand efforts in this direction by exploiting, for example, the numerous improvement proposals of the original NeRF network

PHOTOGRAMMETRY AND MEDIEVAL ARCHITECTURE. USING BLACK AND WHITE ANALOGIC PHOTOGRAPHS FOR RECONSTRUCTING THE FOUNDATIONS OF THE LOST ROOD SCREEN AT SANTA CROCE, FLORENCE

In this research paper photogrammetric techniques have been successfully applied to historic black and white analogic photographs to convey previously inaccessible architectural and archaeological information. The chosen case study for this paper is the Franciscan Basilica of Santa Croce in Florence, Italy. A photogrammetric algorithm has been implemented over a series of b/w negatives portraying the archaeological excavations carried out in the years 1967–1969, after the traumatic flood of the river Arno in 1966 that severely damaged the city centre of Florence and, particularly, the Santa Croce monumental site. The final aim of this operation is to provide solid evidence for the virtual reconstruction of the lost rood screen of the basilica of Santa Croce, the current subject of the PhD research of one of the Authors (Giovanni Pescarmona) at the University of Florence. The foundations that were uncovered during the archaeological excavation in the ‘60s are one of the most important hints towards a convincing retro-planning of the structure. Using advanced photogrammetric techniques, and combining them with LIDAR scanning, it is possible to uncover new datasets that were previously inaccessible for scholars, opening new paths of research. This interdisciplinary approach, combining traditional art-historical research methods and state-of-the-art computational tools, tries to bridge the gap between areas of research that still do not communicate enough with each other, defining new frameworks in the field of Digital Art History

ARCHITECTURAL HERITAGE RECOGNITION IN HISTORICAL FILM FOOTAGE USING NEURAL NETWORKS

Researching historical archives for material suitable for photogrammetry is essential for the documentation and 3D reconstruction of Cultural Heritage, especially when this heritage has been lost or transformed over time. This research presents an innovative workflow which combines the photogrammetric procedure with Machine Learning for the processing of historical film footage. A Neural Network is trained to automatically detect frames in which architectural heritage appears. These frames are subsequently processed using photogrammetry and finally the resulting model is assessed for metric quality. This paper proposes best practises in training and validation on a Cultural Heritage asset. The algorithm was tested through a case study of the Tour Saint Jacques in Paris for which an entirely new dataset was created. The findings are encouraging both in terms of saving human effort and of improvement of the photogrammetric survey pipeline. This new tool can help researchers to better manage and organize historical information

improving performance of feature extraction in sfm algorithms for 3d sparse point cloud

Abstract. The use of Structure-from-Motion algorithms is a common practice to obtain a rapid photogrammetric reconstruction. However, the performance of these algorithms is limited by the fact that in some conditions the resulting point clouds present low density. This is the case when processing materials from historical archives, such as photographs and videos, which generates only sparse point clouds due to the lack of necessary information in the photogrammetric reconstruction. This paper explores ways to improve the performance of open source SfM algorithms in order to guarantee the presence of strategic feature points in the resulting point cloud, even if sparse. To reach this objective, a photogrammetric workflow is proposed to process historical images. The first part of the workflow presents a method that allows the manual selection of feature points during the photogrammetric process. The second part evaluates the metric quality of the reconstruction on the basis of a comparison with a point cloud that has a different density from the sparse point cloud. The workflow was applied to two different case studies. Transformations of wall paintings of the Karanlık church in Cappadocia were analysed thanks to the comparison of 3D model resulting from archive photographs and a recent survey. Then a comparison was performed between the state of the Komise building in Japan, before and after restoration. The findings show that the method applied allows the metric scale and evaluation of the model also in bad condition and when only low-density point clouds are available. Moreover, this tool should be of great use for both art and architecture historians and geomatics experts, to study the evolution of Cultural Heritage

BENCHMARK OF METRIC QUALITY ASSESSMENT IN PHOTOGRAMMETRIC RECONSTRUCTION FOR HISTORICAL FILM FOOTAGE

Quality assessment in photogrammetric processing is fundamental to obtain metric information and to reconstruct 3D models of Cultural Heritage, especially when it has been lost or changed over time. The determination of metric precision is technically challenging when dealing with historical films and videos that in many cases represent the only remaining traces of this heritage, which is useful for architectural, archaeological and restoration studies. This paper examines the suitability of existing photogrammetric software to evaluate the maximum possible metric accuracy for processing videos shot with fixed camera motions. In order to evaluate the metric quality obtained processing historical film footage with photogrammetric techniques, a benchmark was created on a new video dataset with the aim of reproducing the camera motions in which old video were shot. Three different camera motions were considered: Up/Down Motion-Tilting, Left/Right Motion-Trucking and Rolling Motion-Panning. The methodology was experimented on Valentino Castle in Turin, a monument inscribed in the UNESCO World Heritage List. Data were processed with the implementation of open source Structure-from-Motion algorithms and the results were analysed for the evaluation of metric quality. Results show the different maximum precision assessments according to the different typologies of camera motion. This research provides fundamental support to historical studies on Cultural Heritage, creating a sharing standard with zero-cost data and tools useful for both geomatics and restorers

3D DIGITALIZATION AND VISUALIZATION OF ARCHAEOLOGICAL ARTIFACTS WITH THE USE OF PHOTOGRAMMETRY AND VIRTUAL REALITY SYSTEM

Digital technologies are increasingly being used in the field of archaeology to provide three-dimensional metric information to scholars and help them in the process of understanding and interpreting the site under investigation. Among different surveying methods certainly photogrammetry has many advantages being a low cost, reliable and fast technique, but most importantly it allows the creation of realistic and interactive 3D models that can be viewed and interpreted by a wider audience. This certainly makes the enjoyment of sites easier and makes data accessible from anywhere. This study shows three particularly significant case studies in the archaeological field where photogrammetry has served as support. These case studies are representative of specific situations in which archaeology requires digitization of artefacts. The first one concerns the Temple of Apollo in Gortyn (Crete, Greece), the second one is the ancient city of Nora (Sardinia, Italy) and the third one is the Museo Civico of Eremitani in Padua (Italy). The paper explains how 3D metric surveying has served for the representation and analysis of stratigraphic sections of buildings in the case of the Gortyn site, for the creation of virtual tours of archaeological sites in the case of Nora, and for the documentation and visualization of small artifacts in the case of the Museo Civico of Eremitani by highlighting potentials and criticalities of the method