MIXED REALITY CONTENT ALIGNMENT IN MONUMENTAL ENVIRONMENTS

Mixed reality provides on-the-spot and real-time data access capabilities by making virtual models and information more intuitive and accessible. Furthermore, allowing the operator to perceive 3D as holograms would allow for a more natural and straightforward manipulation of the perceived 3D content by permitting the augmentation of real objects with various levels of data. This can be accomplished by appropriately registering and superimposing the presented 3D models with the surrounding environment. This work aims to provide a quantitative evaluation of HoloLens 2 capabilities in registering virtual content inside monumental spaces. Two different methodologies are evaluated: Vuforia image targets and Microsoft World Locking Tools (WLTs). Tests have been performed inside Milan Cathedral's monumental spaces. Here, ambience dimensions, single architectural element repetition and non-uniform lighting conditions push out-of-the-box methods to their limits. Results show that WLTs with their space pins API can correctly reference virtual content keeping deviations in the order of 15 cm coping with the scale error produced from sensors' drifts

3D SURVEY POINT CLOUD DATA AS DIRECT RENDERING ASSETS FOR VISUALISING COMPLEX HERITAGE IN VIRTUAL APPLICATIONS

Digital technology provides methods to record and preserve cultural heritage, support conservation and restoration efforts, and share our collective past with a worldwide audience. Between 2011 and 2017, the 3D Survey Group from Politecnico di Milano operated an annual workshop in the medieval village of Ghesc in which photogrammetry and laser-scanner surveys were carried out. The point cloud data acquired in these activities has become “time slices” documenting different stages of the preservation interventions in Ghesc and the evolution of advanced survey techniques. The main objective of this research is to streamline the workflow of delivering immersive and interactive experiences for complex heritage by directly utilising the 3D survey point cloud data, whether derived from a photogrammetric survey, static laser scanner, or mobile mapping. A point cloud-based multiplatform application is designed and delivered with versatile functions. It runs on PC and VR devices to provide virtual access to the village and narrate its revitalisation story. Additionally, it operates on mobile devices with an AR feature that brings vibrancy to the on-site experience. This application integrates high-fidelity point cloud models, detailed information on vernacular architecture in the Ossola Valley, and information on the preservation project with gamified learning experiences. The unconventional approach of using points as rendering primitives in virtual applications offers a practical solution for visualising complex heritage, enabling an efficient transition from the data collection stage to the data sharing stage without the need for 3D reconstruction and intricate BIM modelling

BIM SYSTEM FOR THE CONSERVATION AND PRESERVATION OF THE MOSAICS OF SAN MARCO IN VENICE

The Basilica of San Marco in Venice is a well-known masterpiece of World Heritage. It is a real multi-faceted architecture. The management of the church and its construction site is very complicated, and requires an efficient system to collect and manage different kinds of data. The BIM approach appeared to be the most suitable to collect multi-source data, to monitor activities and guarantee the well-timed operations inside the church. The purpose of this research was to build a BIM of the Basilica, considering all aspects that characterize it and that require particular care.Many problems affected the phase of the acquisition of data, and forced the team to establish a clear working pipeline that allowed the survey simultaneously, hand in hand, with all the usual activities of the church. The fundamental principle for the organization of the whole work was the subdivision of the entire complex in smaller parts, which could be managed independently, both in the acquisition and the modelling stage. This subdivision also reflects the method used for the photogrammetric acquisition. The complexity of some elements, as capitals and statues, was acquired with different Level of Detail (LoD) using various photogrammetric acquisitions: from the most general ones to describe the space, to the most detailed one 1:1 scale renderings. In this way, different LoD point clouds correspond to different areas or details.As evident, this pipeline allows to work in a more efficient way during the survey stage, but it involves more difficulties in the modelling stage. Because of the complexity of the church and the presence of sculptural elements represented by a mesh, from the beginning the problem of the amount of data was evident: it is nonsense to manage all models in a single file.The challenging aspect of the research job was the precise requirement of the Procuratoria di San Marco: to obtain the 1:1 representation of all the mosaics of the Basilica. This requirement significantly increased the effort in the acquisition stage, because it was necessary to reach a submillimetre resolution in the photographic images sufficient to distinguish perfectly each single tessera, also in the highest domes (28 meters). Furthermore, it introduced a new problem about the management of the gigapixel - orthophotos.The BIM approach presented in this paper tries to offer a solution to all these problems. The BIM application is based not on commercial software, but on a self-implemented system, which was previously tested on the Main Spire of Milano Cathedral. The multi-scale and multi-area approach have also been maintained in the BIM construction phase.In the case of Basilica di San Marco, the most important requirement was the management of the orthophotos of each single element. It was necessary to give the user the possibility to recover, for each item, not only the geometric model, but also the raster representation -orthophoto- of its surface: in order to do it, the BIM model acts as a three-dimensional catalogue

Underwater calibration of dome port pressure housings.

Underwater photogrammetry using consumer grade photographic equipment can be feasible for different applications, e.g. archaeology, biology, industrial inspections, etc. The use of a camera underwater can be very different from its terrestrial use due to the optical phenomena involved. The presence of the water and camera pressure housing in front of the camera act as additional optical elements. Spherical dome ports are difficult to manufacture and consequently expensive but at the same time they are the most useful for underwater photogrammetry as they keep the main geometric characteristics of the lens unchanged. Nevertheless, the manufacturing and alignment of dome port pressure housing components can be the source of unexpected changes of radial and decentering distortion, source of systematic errors that can influence the final 3D measurements. The paper provides a brief introduction of underwater optical phenomena involved in underwater photography, then presents the main differences between flat and dome ports to finally discuss the effect of manufacturing on 3D measurements in two case studies

PORTABLE MULTI-CAMERA SYSTEM: FROM FAST TUNNEL MAPPING TO SEMI-AUTOMATIC SPACE DECOMPOSITION AND CROSS-SECTION EXTRACTION

The paper outlines the first steps of a research project focused on the digitalization of underground tunnels for the mining industry. The aim is to solve the problem of rapidly, semi-automatically, efficiently, and reliably digitizing complex and meandering tunnels. A handheld multi-camera photogrammetric tool is used for the survey phase, which allows for the rapid acquisition of the image dataset needed to produce the 3D data. Moreover, since often, automatic, and fast acquisitions are not supported by easy-to-use tools to access and use the data at an operational level, a second aim of the research is to define a method able to arrange and organise the gathered data so that it would be easily accessible. The proposed approach is to compute the 3D skeleton of the surveyed environment by employing tools developed for the analysis of vascular networks in medical imagery. From the computed skeletonization of the underground tunnels, a method is proposed to automatically extrapolate valuable information such as cross-sections, decomposed portions of the tunnel, and the referenced images from the photogrammetric survey. The long-term research goal is to create an effective workflow, both at the hardware and software level, that can reduce computation times, process large amounts of data, and reduce dependency on high levels of experience

SURVEY OF HISTORICAL GARDENS: MULTI-CAMERA PHOTOGRAMMETRY VS MOBILE LASER SCANNING

This paper presents an investigation into the characterization of historical gardens by comparing two 3D survey methodologies. In this context, approaches employing terrestrial laser scanning are considered the most accurate, while Mobile Mapping Systems (MMSs) are considered promising due to their extreme productivity. Less common is the use of close-range photogrammetry. This paper compares two approaches based on the use of a wearable MMS and the use of an in-house built photogrammetric multi-camera prototype. The comparison aims to assess the applicability of the two techniques in this field, evaluating their advantages and disadvantages in surveying a historical garden and extracting information for tree inventory, such as the DBH (Diameter at Breast Height) and canopy footprint. We compared the practicality of surveying and processing operations; and the quality and characteristics of the point clouds obtained. Both systems produced a dense representation of the terrain. The multi-camera survey resulted to be more defined due to the lower noise of the point cloud but incomplete in the definition of tree canopies. DBH of tree trunks can be extracted with both systems, except for thinner and finer diameter trunks detected by the MMS approach but not always by the multi-camera. The MMS approach proved more effective thanks to a shorter survey time required to cover an equal area and the fact that the MMS survey alone is sufficient for the geometric description of trees. In contrast, the multi-camera approach cannot avoid integration with an aerial survey for canopy reconstructio

V-SLAM-AIDED PHOTOGRAMMETRY TO PROCESS FISHEYE MULTI-CAMERA SYSTEMS SEQUENCES

The advent of mobile mapping systems (MMSs) and computer vision algorithms has enriched a wide range of navigation and mapping tasks such as localisation, 3D motion estimation and 3D mapping. This study focuses on Visual Simultaneous Localisation and Mapping (V-SLAM) in the context of two in-houses MMSs: Ant3D, a patented five-fisheye multi-camera rig and GeoRizon, a high-resolution stereo fisheye rig. The aim is to leverage V-SLAM to enhance the systems performance in near-real-time and non-real-time 3D reconstruction applications. The research investigates both Monocular and Stereo V-SLAM applied to both MMSs and tackles the challenge of combining the V-SLAM estimated trajectory of one or a pair of cameras with known multi-camera relative orientation. We propose a state-of-the-art code that serves as a flexible and extensible platform for MMSs image acquisition and processing, along with an adapted version of the well-established ORB-SLAM3.0. Evaluation is performed in a cultural heritage challenging setup: the Minguzzi spiral staircase in the Duomo di Milano Cathedral. Performed tests highlight that introducing V-SLAM trajectories as well as pre-calibrated interior orientation and multi-camera constraints improve speed, applicability and accuracy of 3D surveys

PHOTOGRAMMETRIC SURVEY OF NARROW SPACES IN CULTURAL HERITAGE: COMPARISON OF TWO MULTI-CAMERA APPROACHES

Multi-camera devices are increasingly popular in various metrological applications, including cultural heritage digitalisation, where these devices are adopted as low-cost alternatives to more traditional methods or mobile mapping systems. They can be of two types: panoramic and non-panoramic configurations, with the former usually more compact and ready-made off-The-shelves and the latter usually custom-developed for metrological applications. In the paper, we compare the accuracy and reliability performance of two types of multi-camera: The spherical camera INSTA 360 Pro2 and the custom multi-camera rig Ant3D. The case study is a challenging spiral staircase environment, typical in many cultural heritage survey projects. The processed image datasets were evaluated in the most common constrain scenario (GCPs at both ends of the staircase) and the worst-case scenario (open-ended path, GCPs at the start). The datasets were processed with precalibrated IO and various degrees of multi-camera constraints up to precalibrated relative orientations. The results highlight that the nominal scale 1:50 can be achieved, e.g. an accuracy of <2 cm plus complete and precise point clouds and mesh results

SURVEY OF HISTORICAL GARDENS: MULTI-CAMERA PHOTOGRAMMETRY VS MOBILE LASER SCANNING

This paper presents an investigation into the characterization of historical gardens by comparing two 3D survey methodologies. In this context, approaches employing terrestrial laser scanning are considered the most accurate, while Mobile Mapping Systems (MMSs) are considered promising due to their extreme productivity. Less common is the use of close-range photogrammetry. This paper compares two approaches based on the use of a wearable MMS and the use of an in-house built photogrammetric multi-camera prototype. The comparison aims to assess the applicability of the two techniques in this field, evaluating their advantages and disadvantages in surveying a historical garden and extracting information for tree inventory, such as the DBH (Diameter at Breast Height) and canopy footprint. We compared the practicality of surveying and processing operations; and the quality and characteristics of the point clouds obtained. Both systems produced a dense representation of the terrain. The multi-camera survey resulted to be more defined due to the lower noise of the point cloud but incomplete in the definition of tree canopies. DBH of tree trunks can be extracted with both systems, except for thinner and finer diameter trunks detected by the MMS approach but not always by the multi-camera. The MMS approach proved more effective thanks to a shorter survey time required to cover an equal area and the fact that the MMS survey alone is sufficient for the geometric description of trees. In contrast, the multi-camera approach cannot avoid integration with an aerial survey for canopy reconstruction