New instruments and technologies for Cultural Heritage survey: full integration between point clouds and digital photogrammetry

In the last years the Geomatic Research Group of the Politecnico di Torino faced some new research topics about new instruments for point cloud generation (e.g. Time of Flight cameras) and strong integration between multi-image matching techniques and 3D Point Cloud information in order to solve the ambiguities of the already known matching algorithms. ToF cameras can be a good low cost alternative to LiDAR instruments for the generation of precise and accurate point clouds: up to now the application range is still limited but in a near future they will be able to satisfy the most part of the Cultural Heritage metric survey requirements. On the other hand multi-image matching techniques with a correct and deep integration of the point cloud information can give the correct solution for an "intelligent" survey of the geometric object break-lines, which are the correct starting point for a complete survey. These two research topics are strictly connected to a modern Cultural Heritage 3D survey approach. In this paper after a short analysis of the achieved results, an alternative possible scenario for the development of the metric survey approach inside the wider topic of Cultural Heritage Documentation is reporte

Self-supervised monocular depth estimation from oblique UAV videos

UAVs have become an essential photogrammetric measurement as they are affordable, easily accessible and versatile. Aerial images captured from UAVs have applications in small and large scale texture mapping, 3D modelling, object detection tasks, DTM and DSM generation etc. Photogrammetric techniques are routinely used for 3D reconstruction from UAV images where multiple images of the same scene are acquired. Developments in computer vision and deep learning techniques have made Single Image Depth Estimation (SIDE) a field of intense research. Using SIDE techniques on UAV images can overcome the need for multiple images for 3D reconstruction. This paper aims to estimate depth from a single UAV aerial image using deep learning. We follow a self-supervised learning approach, Self-Supervised Monocular Depth Estimation (SMDE), which does not need ground truth depth or any extra information other than images for learning to estimate depth. Monocular video frames are used for training the deep learning model which learns depth and pose information jointly through two different networks, one each for depth and pose. The predicted depth and pose are used to reconstruct one image from the viewpoint of another image utilising the temporal information from videos. We propose a novel architecture with two 2D CNN encoders and a 3D CNN decoder for extracting information from consecutive temporal frames. A contrastive loss term is introduced for improving the quality of image generation. Our experiments are carried out on the public UAVid video dataset. The experimental results demonstrate that our model outperforms the state-of-the-art methods in estimating the depths.Comment: Submitted to ISPRS Journal of Photogrammetry and Remote Sensin

Integration of range and image data for building reconstruction

The extraction of information from image and range data is one of the main research topics. In literature, several papers dealing with this topic has been already presented. In particular, several authors have suggested an integrated use of both range and image information in order to increase the reliability and the completeness of the results exploiting their complementary nature. In this paper, an integration between range and image data for the geometric reconstruction of man-made object is presented. The focus is on the edge extraction procedure performed in an integrated way exploiting both the from range and image data. Both terrestrial and aerial applications have been analysed for the faade extraction in terrestrial acquisitions and the roof outline extraction from aerial data. The algorithm and the achieved results will be described and discussed in detail

Integration between calibrated time-of-flight camera data and multi-image matching approach for architectural survey

In this work, the integration between data provided by Time-of-Flight cameras and a multi-image matching technique for metric surveys of architectural elements is presented. The main advantage is given by the quickness in the data acquisition (few minutes) and the reduced cost of the instruments. The goal of this approach is the automatic extraction of the object breaklines in a 3D environment using a photogrammetric process, which is helpful for the final user exigencies for the reduction of the time needed for the drawing production. The results of the performed tests on some architectural elements will be reported in this paper

Structural Building Damage Detection with Deep Learning: Assessment of a State-of-the-Art CNN in Operational Conditions

Remotely sensed data can provide the basis for timely and efficient building damage maps that are of fundamental importance to support the response activities following disaster events. However, the generation of these maps continues to be mainly based on the manual extraction of relevant information in operational frameworks. Considering the identification of visible structural damages caused by earthquakes and explosions, several recent works have shown that Convolutional Neural Networks (CNN) outperform traditional methods. However, the limited availability of publicly available image datasets depicting structural disaster damages, and the wide variety of sensors and spatial resolution used for these acquisitions (from space, aerial and UAV platforms), have limited the clarity of how these networks can effectively serve First Responder needs and emergency mapping service requirements. In this paper, an advanced CNN for visible structural damage detection is tested to shed some light on what deep learning networks can currently deliver, and its adoption in realistic operational conditions after earthquakes and explosions is critically discussed. The heterogeneous and large datasets collected by the authors covering different locations, spatial resolutions and platforms were used to assess the network performances in terms of transfer learning with specific regard to geographical transferability of the trained network to imagery acquired in different locations. The computational time needed to deliver these maps is also assessed. Results show that quality metrics are influenced by the composition of training samples used in the network. To promote their wider use, three pre-trained networks—optimized for satellite, airborne and UAV image spatial resolutions and viewing angles—are made freely available to the scientific community

Towards Fully Autonomous UAV:Damaged Building-Opening Detection for Outdoor-Indoor Transition in Urban Search and Rescue

Autonomous unmanned aerial vehicle (UAV) technology is a promising technology for minimizing human involvement in dangerous activities like urban search and rescue missions (USAR), both in indoor and outdoor. Automated navigation from outdoor to indoor environments is not trivial, as it encompasses the ability of a UAV to automatically map and locate the openings in a damaged building. This study focuses on developing a deep learning model for the detection of damaged building openings in real time. A novel damaged building-opening dataset containing images and mask annotations, as well as a comparison between single and multi-task learning-based detectors are given. The deep learning-based detector used in this study is based on YOLOv5. First, this study compared the different versions of YOLOv5 (i.e., small, medium, and large) capacity to perform damaged building-opening detections. Second, a multitask learning YOLOv5 was trained on the same dataset and compared with the single-task detector. The multitask learning (MTL) was developed based on the YOLOv5 object detection architecture, adding a segmentation branch jointly with the detection head. This study found that the MTL-based YOLOv5 can improve detection performance by combining detection and segmentation losses. The YOLOv5s-MTL trained on the damaged building-opening dataset obtained 0.648 mAP, an increase of 0.167 from the single-task-based network, while its inference speed was 73 frames per second on the tested platform.</p

Real-time Semantic Segmentation with Context Aggregation Network

With the increasing demand of autonomous systems, pixelwise semantic segmentation for visual scene understanding needs to be not only accurate but also efficient for potential real-time applications. In this paper, we propose Context Aggregation Network, a dual branch convolutional neural network, with significantly lower computational costs as compared to the state-of-the-art, while maintaining a competitive prediction accuracy. Building upon the existing dual branch architectures for high-speed semantic segmentation, we design a cheap high resolution branch for effective spatial detailing and a context branch with light-weight versions of global aggregation and local distribution blocks, potent to capture both long-range and local contextual dependencies required for accurate semantic segmentation, with low computational overheads. We evaluate our method on two semantic segmentation datasets, namely Cityscapes dataset and UAVid dataset. For Cityscapes test set, our model achieves state-of-the-art results with mIOU of 75.9%, at 76 FPS on an NVIDIA RTX 2080Ti and 8 FPS on a Jetson Xavier NX. With regards to UAVid dataset, our proposed network achieves mIOU score of 63.5% with high execution speed (15 FPS).Comment: extended version of v

Review article: The use of remotely piloted aircraft systems (RPAS) for natural hazards monitoring and management

The number of scientific studies that consider possible applications of Remotely Piloted Aircraft Systems (RPAS) for the management of natural hazards effects and the identification of occurred damages are strongly increased in last decade. Nowadays, in the scientific community, the use of these systems is not a novelty, but a deeper analysis of literature shows a lack of codified complex methodologies that can be used not only for scientific experiments but also for normal codified emergency operations. RPAS can acquire on-demand ultra-high resolution images that can be used for the identification of active processes like landslides or volcanic activities but also for the definition of effects of earthquakes, wildfires and floods. In this paper, we present a review of published literature that describes experimental methodologiesdeveloped for the study and monitoring of natural hazards

Review article: The use of remotely piloted aircraft systems (RPASs) for natural hazards monitoring and management

The number of scientific studies that consider possible applications of remotely piloted aircraft systems (RPASs) for the management of natural hazards effects and the identification of occurred damages strongly increased in the last decade. Nowadays, in the scientific community, the use of these systems is not a novelty, but a deeper analysis of the literature shows a lack of codified complex methodologies that can be used not only for scientific experiments but also for normal codified emergency operations. RPASs can acquire on-demand ultra-high-resolution images that can be used for the identification of active processes such as landslides or volcanic activities but can also define the effects of earthquakes, wildfires and floods. In this paper, we present a review of published literature that describes experimental methodologies developed for the study and monitoring of natural hazard