OBLIQUE MULTI-CAMERA SYSTEMS - ORIENTATION AND DENSE MATCHING ISSUES

International audience3D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK), Trento, Italy <rupnik, franex, remondino>@fbk.eu, http://3dom.fbk.eu Commission III-WG4 ABS TRACT: The use of oblique imagery has become a standard for many civil and mapping applications, thanks to the development of airborne digital multi-camera systems, as proposed by many companies (Blomoblique, IGI, Leica, M idas, Pictometry, Vexcel/M icrosoft, VisionM ap, etc.). The indisputable virtue of oblique photography lies in its simplicity of interpretation and understanding for inexperienced users allowing their use of oblique images in very different applications, such as building detection and reconstruction, building structural damage classification, road land updating and administration services, etc. The paper reports an overview of the actual oblique commercial systems and presents a workflow for the automated orientation and dense matching of large image blocks. Perspectives, potentialities, pitfalls and suggestions for achieving satisfactory results are given. Tests performed on two datasets acquired with two multi-camera systems over urban areas are also reported. Figure 1: Large urban area pictured with an oblique multi-camera system. Once advanced image triangulation methods have retrieved interior and exterior parameters of the cameras, dense point clouds can be deriv ed for 3D city modelling, feature extraction and mapping purposes

MULTI-TEMPORAL ANALYSIS OF LANDSCAPES AND URBAN AREAS

This article presents a 4D modelling approach that employs multi-temporal and historical aerial images to derive spatio-temporal information for scenes and landscapes. Such imagery represent a unique data source, which combined with photo interpretation and reality-based 3D reconstruction techniques, can offer a more complete modelling procedure because it adds the fourth dimension of time to 3D geometrical representation and thus, allows urban planners, historians, and others to identify, describe, and analyse changes in individual scenes and buildings as well as across landscapes. Particularly important to this approach are historical aerial photos, which provide data about the past that can be collected, processed, and then integrated as a database. The proposed methodology employs both historical (1945) and more recent (1973 and 2000s) aerial images from the Trentino region in North-eastern Italy in order to create a multi-temporal database of information to assist researchers in many disciplines such as topographic mapping, geology, geography, architecture, and archaeology as they work to reconstruct building phases and to understand landscape transformations (Fig. 1)

IMAGE ORIENTATION WITH A HYBRID PIPELINE ROBUST TO ROTATIONS AND WIDE-BASELINES

The extraction of reliable and repeatable interest points among images is a fundamental step for automatic image orientation (Structure-From-Motion). Despite recent progresses, open issues in challenging conditions - such as wide baselines and strong light variations - are still present. Over the years, traditional hand-crafted methods have been paired by learning-based approaches, progressively updating the state-of-the-art according to recent benchmarks. Notwithstanding these advancements, learning-based methods are often not suitable for real photogrammetric surveys due to their lack of rotation invariance, a fundamental requirement for these specific applications. This paper proposes a novel hybrid image matching pipeline which employs both hand-crafted and deep-based components, to extract reliable rotational invariant keypoints optimized for wide-baseline scenarios. The proposed hybrid pipeline was compared with other hand-crafted and learning-based state-of-the-art approaches on some photogrammetric datasets using metric ground-truth data. Results show that the proposed hybrid matching pipeline has high accuracy and appeared to be the only method among the evaluated ones able to register images in the most challenging wide-baseline scenarios

Dimensional discoveries: unveiling the potential of 3D heritage point clouds with a robust ontology framework

3D point clouds feature valuable geometric and, often, radiometric and semantic information to support studies, analyses and understanding of the surveyed scene. Due to their generally large size, the use and interpretation of point clouds could be problematic. User-friendly and quick approaches for querying these valuable datasets and retrieving information could surely support end-users, in particular in the heritage sector. This work presents an ontology-based approach to facilitate the query and use of 3D heritage point clouds by means of sets of rules in order to infer properties and characteristics of the surveyed scene. Our approach is focused on linking together 3D spatial data and expert knowledge, in a way that the ontology can elaborate, represent, enrich and query a given point cloud. Results show how different queries can be set-up and how the procedure can be replicated to various queries and datasets

Principali tecniche e strumenti per il rilievo tridimensionale in ambito archeologico

The increase of 3D acquisition and modeling techniques applied to archeology is due principally to (i) their capacity to survey archeological artifacts with high precision and a non-contact approach and (ii) the possibility to create 3D digital models useful for data analysis, simulation and preservation. These benefits in terms of knowledge oblige the contemporary archaeologist to acquire a better understanding of 3D acquisition and modeling principles and practice. This evidence arises from the necessity of adopting a common language for experts in 3D data management and archaeologists with the principal aim being the understanding of each other’s requirements and sharing of the purposes of the project. In this article the authors propose a concise but exhaustive explanation of the working principles of active and passive 3D acquisition techniques. For each one a description of instruments and methodologies is developed, pointing out pros and cons of every technique. In conclusion, a sensor fusion approach is presented as an interesting solution to increase the instrument performances while obtaining at the same time a quality improvement of 3D acquisition and modeling results. A final multi-resolution application about Pompeii Forum 3D modeling follows and closes the article

Principali tecniche e strumenti per il rilievo tridimensionale in ambito archeologico

The increase of 3D acquisition and modeling techniques applied to archeology is due principally to (i) their capacity to survey archeological artifacts with high precision and a non-contact approach and (ii) the possibility to create 3D digital models useful for data analysis, simulation and preservation. These benefits in terms of knowledge oblige the contemporary archaeologist to acquire a better understanding of 3D acquisition and modeling principles and practice. This evidence arises from the necessity of adopting a common language for experts in 3D data management and archaeologists with the principal aim being the understanding of each other’s requirements and sharing of the purposes of the project. In this article the authors propose a concise but exhaustive explanation of the working principles of active and passive 3D acquisition techniques. For each one a description of instruments and methodologies is developed, pointing out pros and cons of every technique. In conclusion, a sensor fusion approach is presented as an interesting solution to increase the instrument performances while obtaining at the same time a quality improvement of 3D acquisition and modeling results. A final multi-resolution application about Pompeii Forum 3D modeling follows and closes the article

Underwater calibration of dome port pressure housings.

Underwater photogrammetry using consumer grade photographic equipment can be feasible for different applications, e.g. archaeology, biology, industrial inspections, etc. The use of a camera underwater can be very different from its terrestrial use due to the optical phenomena involved. The presence of the water and camera pressure housing in front of the camera act as additional optical elements. Spherical dome ports are difficult to manufacture and consequently expensive but at the same time they are the most useful for underwater photogrammetry as they keep the main geometric characteristics of the lens unchanged. Nevertheless, the manufacturing and alignment of dome port pressure housing components can be the source of unexpected changes of radial and decentering distortion, source of systematic errors that can influence the final 3D measurements. The paper provides a brief introduction of underwater optical phenomena involved in underwater photography, then presents the main differences between flat and dome ports to finally discuss the effect of manufacturing on 3D measurements in two case studies

Dimensional discoveries: Unveiling the potential of 3d heritage point clouds with a robust ontology framework

3D point clouds feature valuable geometric and, often, radiometric and semantic information to support studies, analyses and understanding of the surveyed scene. Due to their generally large size, the use and interpretation of point clouds could be problematic. User-friendly and quick approaches for querying these valuable datasets and retrieving information could surely support end-users, in particular in the heritage sector. This work presents an ontology-based approach to facilitate the query and use of 3D heritage point clouds by means of sets of rules in order to infer properties and characteristics of the surveyed scene. Our approach is focused on linking together 3D spatial data and expert knowledge, in a way that the ontology can elaborate, represent, enrich and query a given point cloud. Results show how different queries can be set-up and how the procedure can be replicated to various queries and datasets

JOINT ALIGNMENT OF UNDERWATER AND ABOVE-THE-WATER PHOTOGRAMMETRIC 3D MODELS BY INDEPENDENT MODELS ADJUSTMENT

The surveying and 3D modelling of objects that extend both below and above the water level, such as ships, harbour structures, offshore platforms, are still an open issue. Commonly, a combined and simultaneous survey is the adopted solution, with acoustic/optical sensors respectively in underwater and in air (most common) or optical/optical sensors both below and above the water level. In both cases, the system must be calibrated and a ship is to be used and properly equipped with also a navigation system for the alignment of sequential 3D point clouds. Such a system is usually highly expensive and has been proved to work with still structures. On the other hand for free floating objects it does not provide a very practical solution. In this contribution, a flexible, low-cost alternative for surveying floating objects is presented. The method is essentially based on photogrammetry, employed for surveying and modelling both the emerged and submerged parts of the object. Special targets, named Orientation Devices, are specifically designed and adopted for the successive alignment of the two photogrammetric models (underwater and in air). A typical scenario where the proposed procedure can be particularly suitable and effective is the case of a ship after an accident whose damaged part is underwater and necessitate to be measured (Figure 1). The details of the mathematical procedure are provided in the paper, together with a critical explanation of the results obtained from the adoption of the method for the survey of a small pleasure boat in floating condition

DEEP LEARNING TO SUPPORT 3D MAPPING CAPABILITIES OF A PORTABLE VSLAM-BASED SYSTEM

The use of vision-based localization and mapping techniques, such as visual odometry and SLAM, has become increasingly prevalent in the field of Geomatics, particularly in mobile mapping systems. These methods provide real-time estimation of the 3D scene as well as sensor's position and orientation using images or LiDAR sensors mounted on a moving platform. While visual odometry primarily focuses on the camera's position, SLAM also creates a 3D reconstruction of the environment. Conventional (geometric) and learning-based approaches are used in visual SLAM, with deep learning networks being integrated to perform semantic segmentation, object detection and depth prediction. The goal of this work is to report ongoing developments to extend the GuPho stereo-vision SLAM-based system with deep learning networks for tasks such as crack detection, obstacle detection and depth estimation. Our findings show how a neural network can be coupled to SLAM sequences in order to support 3D mapping application with semantic information