Close-Range Sensing and Data Fusion for Built Heritage Inspection and Monitoring - A Review

Built cultural heritage is under constant threat due to environmental pressures, anthropogenic damages, and interventions. Understanding the preservation state of monuments and historical structures, and the factors that alter their architectural and structural characteristics through time, is crucial for ensuring their protection. Therefore, inspection and monitoring techniques are essential for heritage preservation, as they enable knowledge about the altering factors that put built cultural heritage at risk, by recording their immediate effects on monuments and historic structures. Nondestructive evaluations with close-range sensing techniques play a crucial role in monitoring. However, data recorded by different sensors are frequently processed separately, which hinders integrated use, visualization, and interpretation. This article’s aim is twofold: i) to present an overview of close-range sensing techniques frequently applied to evaluate built heritage conditions, and ii) to review the progress made regarding the fusion of multi-sensor data recorded by them. Particular emphasis is given to the integration of data from metric surveying and from recording techniques that are traditionally non-metric. The article attempts to shed light on the problems of the individual and integrated use of image-based modeling, laser scanning, thermography, multispectral imaging, ground penetrating radar, and ultrasonic testing, giving heritage practitioners a point of reference for the successful implementation of multidisciplinary approaches for built cultural heritage scientific investigations

SkiMap: An Efficient Mapping Framework for Robot Navigation

We present a novel mapping framework for robot navigation which features a multi-level querying system capable to obtain rapidly representations as diverse as a 3D voxel grid, a 2.5D height map and a 2D occupancy grid. These are inherently embedded into a memory and time efficient core data structure organized as a Tree of SkipLists. Compared to the well-known Octree representation, our approach exhibits a better time efficiency, thanks to its simple and highly parallelizable computational structure, and a similar memory footprint when mapping large workspaces. Peculiarly within the realm of mapping for robot navigation, our framework supports realtime erosion and re-integration of measurements upon reception of optimized poses from the sensor tracker, so as to improve continuously the accuracy of the map.Comment: Accepted by International Conference on Robotics and Automation (ICRA) 2017. This is the submitted version. The final published version may be slightly differen

Uncertainty Minimization in Robotic 3D Mapping Systems Operating in Dynamic Large-Scale Environments

This dissertation research is motivated by the potential and promise of 3D sensing technologies in safety and security applications. With specific focus on unmanned robotic mapping to aid clean-up of hazardous environments, under-vehicle inspection, automatic runway/pavement inspection and modeling of urban environments, we develop modular, multi-sensor, multi-modality robotic 3D imaging prototypes using localization/navigation hardware, laser range scanners and video cameras. While deploying our multi-modality complementary approach to pose and structure recovery in dynamic real-world operating conditions, we observe several data fusion issues that state-of-the-art methodologies are not able to handle. Different bounds on the noise model of heterogeneous sensors, the dynamism of the operating conditions and the interaction of the sensing mechanisms with the environment introduce situations where sensors can intermittently degenerate to accuracy levels lower than their design specification. This observation necessitates the derivation of methods to integrate multi-sensor data considering sensor conflict, performance degradation and potential failure during operation. Our work in this dissertation contributes the derivation of a fault-diagnosis framework inspired by information complexity theory to the data fusion literature. We implement the framework as opportunistic sensing intelligence that is able to evolve a belief policy on the sensors within the multi-agent 3D mapping systems to survive and counter concerns of failure in challenging operating conditions. The implementation of the information-theoretic framework, in addition to eliminating failed/non-functional sensors and avoiding catastrophic fusion, is able to minimize uncertainty during autonomous operation by adaptively deciding to fuse or choose believable sensors. We demonstrate our framework through experiments in multi-sensor robot state localization in large scale dynamic environments and vision-based 3D inference. Our modular hardware and software design of robotic imaging prototypes along with the opportunistic sensing intelligence provides significant improvements towards autonomous accurate photo-realistic 3D mapping and remote visualization of scenes for the motivating applications

Machine Learning and Pattern Recognition Methods for Remote Sensing Image Registration and Fusion

In the last decade, the remote sensing world has dramatically evolved. New types of sensor, each one collecting data with possibly different modalities, have been designed, developed, and deployed. Moreover, new missions have been planned and launched, aimed not only at collecting data of the Earth's surface, but also at acquiring planetary data in support of the study of the whole Solar system. Indeed, such a variety of technologies highlights the need for automatic methods able to effectively exploit all the available information. In the last years, lot of effort has been put in the design and development of advanced data fusion methods able to extract and make use of all the information available from as many complementary information sources as possible. Indeed, the goal of this thesis is to present novel machine learning and pattern recognition methodologies designed to support the exploitation of diverse sources of information, such as multisensor, multimodal, or multiresolution imagery. In this context, image registration plays a major role as is allows bringing two or more digital images into precise alignment for analysis and comparison. Here, image registration is tackled using both feature-based and area-based strategies. In the former case, the features of interest are extracted using a stochastic geometry model based on marked point processes, while, in the latter case, information theoretic functionals and the domain adaptation capabilities of generative adversarial networks are exploited. In addition, multisensor image registration is also applied in a large scale scenario by introducing a tiling-based strategy aimed at minimizing the computational burden, which is usually heavy in the multisensor case due to the need for information theoretic similarity measures. Moreover, automatic change detection with multiresolution and multimodality imagery is addressed via a novel Markovian framework based on a linear mixture model and on an ad-hoc multimodal energy function minimized using graph cuts or belied propagation methods. The statistics of the data at the various spatial scales is modelled through appropriate generalized Gaussian distributions and by iteratively estimating a set of virtual images, at the finest resolution, representing the data that would have been collected in case all the sensors worked at that resolution. All such methodologies have been experimentally evaluated with respect to different datasets, and with particular focus on the trade-off between the achievable performances and the demands in terms of computational resources. Moreover, such methods are also compared with state-of-the-art solutions, and are analyzed in terms of future developments, giving insights to possible future lines of research in this field

3D scanning, modelling and printing of ultra-thin nacreous shells from Jericho: a case study of small finds documentation in archaeology

This paper springs out from a collaborative project jointly carried out by the FabLab Saperi&Co and the Museum of Near East, Egypt and Mediterranean of Sapienza University of Rome focused at producing replicas of ultra-thin archeological finds with a sub-millimetric precision. The main technological challenge of this project was to produce models through 3D optical scanning (photogrammetry) and to print faithful replicas with additive manufacturing. The objects chosen for the trial were five extremely fragile and ultra-thin nacreous shells retrieved in Tell es-Sultan/ancient Jericho by the Italian-Palestinian Expedition in spring 2017, temporarily on exhibit in the Museum. The experiment proved to be successful, and the scanning, modeling and printing of the shells also allowed some observations on their possible uses in research and museum activities

A robust coregistration method for in vivo studies using a first generation simultaneous PET/MR scanner

Purpose: Hybrid positron emission tomography (PET)/magnetic resonance (MR) imaging systems have recently been built that allow functional and anatomical information obtained from PET and MR to be acquired simultaneously. The authors have developed a robust coregistration scheme for a first generation small animal PET/MR imaging system and illustrated the potential of this system to study intratumoral heterogeneity in a mouse model. Methods: An alignment strategy to fuse simultaneously acquired PET and MR data, using the MR imaging gradient coordinate system as the reference basis, was developed. The fidelity of the alignment was evaluated over multiple study sessions. In order to explore its robustness in vivo, the alignment strategy was applied to explore the heterogeneity of glucose metabolism in a xenograft tumor model, using ^(18)F-FDG-PET to guide the acquisition of localized ^1H MR spectra within a single imaging session. Results: The alignment method consistently fused the PET/MR data sets with subvoxel accuracy (registration error mean=0.55 voxels, <0.28 mm); this was independent of location within the field of view. When the system was used to study intratumoral heterogeneity within xenograft tumors, a correlation of high ^(18)F-FDG-PET signal with high choline/creatine ratio was observed. Conclusions: The authors present an implementation of an efficient and robust coregistration scheme for multimodal noninvasive imaging using PET and MR. This setup allows time-sensitive, multimodal studies of physiology to be conducted in an efficient manner

4D Reconstruction and Visualization of Cultural Heritage: Analysing our Legacy Through Time

Temporal analyses and multi-temporal 3D reconstruction are fundamental for the preservation and maintenance of all forms of Cultural Heritage (CH) and are the basis for decisions related to interventions and promotion. Introducing the fourth dimension of time into three-dimensional geometric modelling of real data allows the creation of a multi-temporal representation of a site. In this way, scholars from various disciplines (surveyors, geologists, archaeologists, architects, philologists, etc.) are provided with a new set of tools and working methods to support the study of the evolution of heritage sites, both to develop hypotheses about the past and to model likely future developments. The capacity to “see” the dynamic evolution of CH assets across different spatial scales (e.g. building, site, city or territory) compressed in diachronic model, affords the possibility to better understand the present status of CH according to its history. However, there are numerous challenges in order to carry out 4D modelling and the requisite multi-data source integration. It is necessary to identify the specifications, needs and requirements of the CH community to understand the required levels of 4D model information. In this way, it is possible to determine the optimum material and technologies to be utilised at different CH scales, as well as the data management and visualization requirements. This manuscript aims to provide a comprehensive approach for CH time-varying representations, analysis and visualization across different working scales and environments: rural landscape, urban landscape and architectural scales. Within this aim, the different available metric data sources are systemized and evaluated in terms of their suitability