A Classification-Segmentation Framework for the Detection of Individual Trees in Dense MMS Point Cloud Data Acquired in Urban Areas

International audienceIn this paper, we present a novel framework for detecting individual trees in densely sampled 3D point cloud data acquired in urban areas. Given a 3D point cloud, the objective is to assign point-wise labels that are both class-aware and instance-aware, a task that is known as instance-level segmentation. To achieve this, our framework addresses two successive steps. The first step of our framework is given by the use of geometric features for a binary point-wise semantic classification with the objective of assigning semantic class labels to irregularly distributed 3D points, whereby the labels are defined as " tree points " and " other points ". The second step of our framework is given by a semantic segmentation with the objective of separating individual trees within the " tree points ". This is achieved by applying an efficient adaptation of the mean shift algorithm and a subsequent segment-based shape analysis relying on semantic rules to only retain plausible tree segments. We demonstrate the performance of our framework on a publicly available benchmark dataset, which has been acquired with a mobile mapping system in the city of Delft in the Netherlands. This dataset contains 10.13 M labeled 3D points among which 17.6% are labeled as " tree points ". The derived results clearly reveal a semantic classification of high accuracy (up to 90.77%) and an instance-level segmentation of high plausibility, while the simplicity, applicability and efficiency of the involved methods even allow applying the complete framework on a standard laptop computer with a reasonable processing time (less than 2.5 h)

Potential of Mobile Mapping to Create Digital Twins of Forests

Forests are irreplaceable and are being studied extensively. Better forest inventory and understanding necessitate effective mapping, modeling, and automatic analysis. As a result, considerable research effort is being devoted to digitizing forest environments. Recently, digital twins have come to the attention of the geospatial community as a virtual representation of the Earth’s surface linked to its corresponding physical asset. This concept is applicable to forests and has been studied in the literature. This requires initial input data obtained through reality capture. Among mapping techniques, laser scanning has emerged as a state-of-the-art technology for vegetation modeling. In this paper, we look into the potential of mobile laser scanning for forest digital twinning. While most studies concentrate on single tree detection, modeling, and estimation of dendrometric parameters, we also include lower vegetation in our investigations. To accomplish this, we first detect single trees and then investigate different vegetation densities and levels using geometric metrics. We also demonstrate how to model the underlying layers of vegetation in a digital twin. We perform the tests on data from mobile laser scanning (MLS) and compare the results to those from airborne laser scanning (ALS).We show that single tree detection based on crown separation using MLS data works similarly to or slightly better than ALS data. Furthermore, we demonstrate that MLS data allows for more detailed analysis of understory vegetation taking into account different height levels and a multi-level representation, whereas ALS data only allows for rough analysis of the lower parts of forest vegetation

モービルマッピングシステムと航空測量を用いた都市空間高精度３次元モデリング

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 瀬崎 薫, 東京大学教授 江崎 浩, 東京大学教授 苗村 健, 東京大学教授 柴崎 亮介, 東京大学准教授 上條 俊介, 国際電気通信基礎技術研究所 浅見 徹University of Tokyo(東京大学

Update urban basemap by using the LiDAR mobile mapping system : the case of Abu Dhabi municipal system

Basemaps are the main resource used in urban planning and in building and infrastructure asset management. These maps are used by citizens and by private and public stakeholders. Therefore, accurate, up-to-date geoinformation of reference are needed to provide a good service. In general, basemaps have been updated by aerial photogrammetry or field surveying, but these methods are not always possible and alternatives need to be sought. Current limitations and challenges that face traditional field surveys include areas with extreme weather, deserts or artic environments, and flight restrictions due to proximity with other countries if there is not an agreement. In such cases, alternatives for large-scale are required. This thesis proposes the use of a mobile mapping system (MMS) to update urban basemaps. Most urban features can be extracted from point cloud using commercial software or open libraries. However, there are some exceptions: manhole covers, or hidden elements even with captures from defferent perspective, the most common building corners. Therefore, the main objective of this study was to establish a methodology for extracting manholes automatically and for completing hidden corners of buildings, so that urban basemaps can be updated. The algorithm developed to extract manholes is based on time, intensity and shape detection parameters, whereas additional information from satellite images is used to complete buildings. Each municipality knows the materials and dimensions of its manholes. Taking advantage of this knowledge, the point cloud is filtered to classify points according to the set of intensity values associated with the manhole material. From the classified points, the minimum bounding rectangles (MBR) are obtained and finally the shape is adjusted and drawn. We use satellite imagery to automatically digitize the layout of building footprints with automated software tools. Then, the visible corners of buildings from the LiDAR point cloud are imported and a fitting process is performed by comparing them with the corners of the building from the satellite image. Two methods are evaluated to establish which is the most suitable for adjustment in these conditions. In the first method, the differences in X and Y directions are measured in the corners, where LiDAR and satellite data are available, and is often computed as the average of the offsets. In the second method, a Helmert 2D transformation is applied. MMS involves Global Navigation Satellite Systems (GNSS) and Inertial Measurement Units (IMU) to georeference point clouds. Their accuracy depends on the acquisition environment. In this study, the influence of the urban pattern is analysed in three zones with varied urban characteristics: different height buildings, open areas, and areas with a low and high level of urbanization. To evaluate the efficiency of the proposed algorithms, three areas were chosen with varying urban patterns in Abu Dhabi. In these areas, 3D urban elements (light poles, street signs, etc) were automatically extracted using commercial software. The proposed algorithms were applied to the manholes and buildings. The completeness and correctness ratio, and geometric accuracy were calculated for all urban elements in the three areas. The best success rates (>70%) were for light poles, street signs and road curbs, regardless of the height of the buildings. The worst rate was obtained for the same features in peri-urban areas, due to high vegetation. In contrast, the best results for trees were found in theses areas. Our methodology demonstrates the great potential and efficiency of mobile LiDAR technology in updating basemaps; a process that is required to achieve standard accuracy in large scale maps. The cost of the entire process and the time required for the proposed methodology was calculated and compared with the traditional method. It was found that mobile LiDAR could be a standard cost-efficient procedure for updating maps.La cartografía de referencia es la principal herramienta en planificación urbanística, y gestión de infraestructuras y edificios, al servicio de ciudadanos, empresas y administración. Por esta razón, debe estar actualizada y ser lo más precisa posible. Tradicionalmente, la cartografía se actualiza mediante fotogrametría aérea o levantamientos terrestres. No obstante, deben buscarse alternativas válidas para escalas grandes, porque no siempre es posible emplear estas técnicas debido a las limitaciones y retos actuales a los que se enfrenta la medición tradicional en algunas zonas del planeta, con meteorología extrema o restricciones de vuelo por la proximidad a la frontera con otros países. Esta tesis propone el uso del sistema Mobile Mapping System (MMS) para actualizar la cartografía urbana de referencia. La mayoría de los elementos pueden extraerse empleando software comercial o librerías abiertas, excepto los registros de servicios. Los elementos ocultos son otro de los inconvenientes encontrados en el proceso de creación o actualización de la cartografía, incluso si se dispone de capturas desde diferentes puntos de vista. El caso más común es el de las esquinas de edificios. Por ello, el principal objetivo de este estudio es establecer una metodología de extracción automática de los registros y completar las esquinas ocultas de los edificios para actualizar cartografía urbana. El algoritmo desarrollado para la detección y extracción de registros se basa en parámetros como el tiempo, la intensidad de la señal laser y la forma de los registros, mientras que para completar los edificios se emplea información adicional de imágenes satélite. Aprovechando el conocimiento del material y dimensión de los registros, en disposición de los gestores municipales, el algoritmo propuesto filtra y clasifica los puntos de acuerdo a los valores de intensidad. De aquellos clasificados como registros se calcula el mínimo rectángulo que los contiene (Minimum Bounding Rectangle) y finalmente se ajusta la forma y se dibuja. Las imágenes de satélite son empleadas para obtener automáticamente la huella de los edificios. Posteriormente, se importan las esquinas visibles de los edificios obtenidas desde la nube de puntos y se realiza el ajuste comparándolas con las obtenidas desde satélite. Para llevar a cabo este ajuste se han evaluado dos métodos, el primero de ellos considera las diferencias entre las coordenadas XY, desplazándose el promedio. En el segundo, se aplica una transformación Helmert2D. MMS emplea sistemas de navegación global por satélite (Global Navigation Satellite Systems, GNSS) e inerciales (Inertial Measurement Unit, IMU) para georreferenciar la nube de puntos. La precisión de estos sistemas de posicionamiento depende del entorno de adquisición. Por ello, en este estudio se han seleccionado tres áreas con distintas características urbanas (altura de edificios, nivel de urbanización y áreas abiertas) de Abu Dhabi con el fin de analizar su influencia, tanto en la captura, como en la extracción de los elementos. En el caso de farolas, señales viales, árboles y aceras se ha realizado con software comercial, y para registros y edificios con los algoritmos propuestos. Las ratios de corrección y completitud, y la precisión geométrica se han calculado en las diferentes áreas urbanas. Los mejores resultados se han conseguido para las farolas, señales y bordillos, independientemente de la altura de los edificios. La peor ratio se obtuvo para los mismos elementos en áreas peri-urbanas, debido a la vegetación. Resultados opuestos se han conseguido en la detección de árboles. El coste económico y en tiempo de la metodología propuesta resulta inferior al de métodos tradicionales. Lo cual demuestra el gran potencial y eficiencia de la tecnología LiDAR móvil para la actualización cartografía de referenciaPostprint (published version

SPARSE POINT CLOUD FILTERING BASED ON COVARIANCE FEATURES

Abstract. This work presents an extended photogrammetric pipeline aimed to improve 3D reconstruction results. Standard photogrammetric pipelines can produce noisy 3D data, especially when images are acquired with various sensors featuring different properties. In this paper, we propose an automatic filtering procedure based on some geometric features computed on the sparse point cloud created within the bundle adjustment phase. Bad 3D tie points and outliers are detected and removed, relying on micro and macro-clusters analyses. Clusters are built according to the prevalent dimensionality class (1D, 2D, 3D) assigned to low-entropy points, and corresponding to the main linear, planar o scatter local behaviour of the point cloud. While the macro-clusters analysis removes smallsized clusters and high-entropy points, in the micro-clusters investigation covariance features are used to verify the inner coherence of each point to the assigned class. Results on heritage scenarios are presented and discussed.</p

Classification of aerial laser scanning point clouds using machine learning: a comparison between Random Forest and Tensorflow

In this investigation a comparison between two machine learning (ML) models for semantic classification of an aerial laser scanner point cloud is presented. One model is Random Forest (RF), the other is a multi-layer neural network, TensorFlow (TF). Accuracy results were compared over a growing set of training data, using a stratified independent sampling over classes from 5% to 50% of the total dataset. Results show RF to have average F1=0.823 for the 9 classes considered, whereas TF had average F1=0.450. F1 values where higher for RF than TF, due to complexity in the determination of a suitable composition of the hidden layers of the neural network in TF, and this can likely be improved to reach higher accuracy values. Further study in this sense is planned

DETECTION OF BUILDING ROOFS AND FACADES FROM AERIAL LASER SCANNING DATA USING DEEP LEARNING

In this work we test the power of prediction of deep learning for detection of buildings from aerial laser scanner point cloud information. Automatic extraction of built features from remote sensing data is of extreme interest for many applications. In particular latest paradigms of 3D mapping of buildings, such as CityGML and BIM, can benefit from an initial determination of building geometries. In this work we used a LiDAR dataset of urban environment from the ISPRS benchmark on urban object detection. The dataset is labelled with eight classes, two were used for this investigation: roof and facades. The objective is to test how TensorFlow neural network for deep learning can predict these two classes. Results show that for “roof” and “facades” semantic classes respectively, recall is 84% and 76% and precision is 72% and 63%. The number and distribution of correct points well represent the geometry, thus allowing to use them as support for CityGML and BIM modelling. Further tuning of the hidden layers of the DL model will likely improve results and will be tested in future investigations

DETECTION OF BUILDING ROOFS AND FACADES FROM AERIAL LASER SCANNING DATA USING DEEP LEARNING

In this work we test the power of prediction of deep learning for detection of buildings from aerial laser scanner point cloud information. Automatic extraction of built features from remote sensing data is of extreme interest for many applications. In particular latest paradigms of 3D mapping of buildings, such as CityGML and BIM, can benefit from an initial determination of building geometries. In this work we used a LiDAR dataset of urban environment from the ISPRS benchmark on urban object detection. The dataset is labelled with eight classes, two were used for this investigation: roof and facades. The objective is to test how TensorFlow neural network for deep learning can predict these two classes. Results show that for “roof” and “facades” semantic classes respectively, recall is 84% and 76% and precision is 72% and 63%. The number and distribution of correct points well represent the geometry, thus allowing to use them as support for CityGML and BIM modelling. Further tuning of the hidden layers of the DL model will likely improve results and will be tested in future investigations

Mapping of Road Facilities and Information on High Definition Maps using Mobile Mapping System

学位の種別: 修士University of Tokyo(東京大学