Long-Term Urban Vehicle Localization Using Pole Landmarks Extracted from 3-D Lidar Scans

Due to their ubiquity and long-term stability, pole-like objects are well suited to serve as landmarks for vehicle localization in urban environments. In this work, we present a complete mapping and long-term localization system based on pole landmarks extracted from 3-D lidar data. Our approach features a novel pole detector, a mapping module, and an online localization module, each of which are described in detail, and for which we provide an open-source implementation at www.github.com/acschaefer/polex. In extensive experiments, we demonstrate that our method improves on the state of the art with respect to long-term reliability and accuracy: First, we prove reliability by tasking the system with localizing a mobile robot over the course of 15~months in an urban area based on an initial map, confronting it with constantly varying routes, differing weather conditions, seasonal changes, and construction sites. Second, we show that the proposed approach clearly outperforms a recently published method in terms of accuracy.Comment: 9 page

Robust statistical approaches for feature extraction in laser scanning 3D point cloud data

Three dimensional point cloud data acquired from mobile laser scanning system commonly contain outliers and/or noise. The presence of outliers and noise means most of the frequently used methods for feature extraction produce inaccurate and non-robust results. We investigate the problems of outliers and how to accommodate them for automatic robust feature extraction. This thesis develops algorithms for outlier detection, point cloud denoising, robust feature extraction, segmentation and ground surface extraction

Automatic Reconstruction of Urban Objects from Mobile Laser Scanner Data

Aktuelle 3D-Stadtmodelle werden immer wichtiger in verschiedenen städtischen Anwendungsbereichen. Im Moment dienen sie als Grundlage bei der Stadtplanung, virtuellem Tourismus und Navigationssystemen. Mittlerweile ist der Bedarf an 3D-Gebäudemodellen dramatisch gestiegen. Der Grund dafür sind hauptsächlich Navigationssysteme und Onlinedienste wie Google Earth. Die Mehrheit der Untersuchungen zur Rekonstruktion von Gebäudemodellen von Luftaufnahmen konzentriert sich ausschließlich auf Dachmodellierung. Jedoch treiben Anwendungen wie Virtuelle Realität und Navigationssysteme die Nachfrage nach detaillieren Gebäudemodellen, die nicht nur die geometrischen Aspekte sondern auch semantische Informationen beinhalten, stark an. Urbanisierung und Industrialisierung beeinflussen das Wachstum von urbaner Vegetation drastisch, welche als ein wesentlicher Teil des Lebensraums angesehen wird. Aus diesem Grund werden Aufgaben wie der Ökosystemüberwachung, der Verbesserung der Planung und des Managements von urbanen Regionen immer mehr Aufmerksamkeit geschenkt. Gleichermaßen hat die Erkennung und Modellierung von Bäumen im Stadtgebiet sowie die kontinuierliche Überprüfung ihrer Inventurparameter an Bedeutung gewonnen. Die steigende Nachfrage nach 3D-Gebäudemodellen, welche durch Fassadeninformation ergänzt wurden, und Informationen über einzelne Bäume im städtischen Raum erfordern effiziente Extraktions- und Rekonstruktionstechniken, die hochgradig automatisiert sind. In diesem Zusammenhang ist das Wissen über die geometrische Form jedes Objektteils ein wichtiger Aspekt. Heutzutage, wird das Mobile Laser Scanning (MLS) vermehrt eingesetzt um Objekte im städtischen Umfeld zu erfassen und es entwickelt sich zur Hauptquelle von Daten für die Modellierung von urbanen Objekten. Eine Vielzahl von Objekten wurde schon mit Daten von MLS rekonstruiert. Außerdem wurden bereits viele Methoden für die Verarbeitung von MLS-Daten mit dem Ziel urbane Objekte zu erkennen und zu rekonstruieren vorgeschlagen. Die 3D-Punkwolke einer städtischen Szene stellt eine große Menge von Messungen dar, die viele Objekte von verschiedener Größe umfasst, komplexe und unvollständige Strukturen sowie Löcher (Rauschen und Datenlücken) enthält und eine inhomogene Punktverteilung aufweist. Aus diesem Grund ist die Verarbeitung von MLS-Punktwolken im Hinblick auf die Extrahierung und Modellierung von wesentlichen und charakteristischen Fassadenstrukturen sowie Bäumen von großer Bedeutung. In der Arbeit werden zwei neue Methoden für die Rekonstruktion von Gebäudefassaden und die Extraktion von Bäumen aus MLS-Punktwolken vorgestellt, sowie ihre Anwendbarkeit in der städtischen Umgebung analysiert. Die erste Methode zielt auf die Rekonstruktion von Gebäudefassaden mit expliziter semantischer Information, wie beispielsweise Fenster, Türen, und Balkone. Die Rekonstruktion läuft vollautomatisch ab. Zu diesem Zweck werden einige Algorithmen vorgestellt, die auf dem Vorwissen über die geometrische Form und das Arrangement von Fassadenmerkmalen beruhen. Die initiale Klassifikation, mit welcher die Punkte in Objektpunkte und Bodenpunkte unterschieden werden, wird über eine lokale Höhenhistogrammanalyse zusammen mit einer planaren Region-Growing-Methode erzielt. Die Punkte, die als zugehörig zu Objekten klassifiziert werden, werden anschließend in Ebenen segmentiert, welche als Basiselemente der Merkmalserkennung angesehen werden können. Information über die Gebäudestruktur kann in Form von Regeln und Bedingungen erfasst werden, welche die wesentlichen Steuerelemente bei der Erkennung der Fassadenmerkmale und der Rekonstruktion des geometrischen Modells darstellen. Um Merkmale wie Fenster oder Türen zu erkennen, die sich an der Gebäudewand befinden, wurde eine löcherbasierte Methode implementiert. Einige Löcher, die durch Verdeckungen entstanden sind, können anschließend durch einen neuen regelbasierten Algorithmus eliminiert werden. Außenlinien der Merkmalsränder werden durch ein Polygon verbunden, welches das geometrische Modell repräsentiert, indem eine Methode angewendet wird, die auf geometrischen Primitiven basiert. Dabei werden die topologischen Relationen unter Beachtung des Vorwissens über die primitiven Formen analysiert. Mögliche Außenlinien können von den Kantenpunkten bestimmt werden, welche mit einer winkelbasierten Methode detektiert werden können. Wiederkehrende Muster und Ähnlichkeiten werden ausgenutzt um geometrische und topologische Ungenauigkeiten des rekonstruierten Modells zu korrigieren. Neben der Entwicklung des Schemas zur Rekonstruktion des 3D-Fassadenmodells, sind die Segmentierung einzelner Bäume und die Ableitung von Attributen der städtischen Bäume im Fokus der Untersuchung. Die zweite Methode zielt auf die Extraktion von individuellen Bäumen aus den Restpunktwolken. Vorwissen über Bäume, welches speziell auf urbane Regionen zugeschnitten ist, wird im Extraktionsprozess verwendet. Der formbasierte Ansatz zur Extraktion von Einzelbäumen besteht aus einer Reihe von Schritten. In jedem Schritt werden Objekte in Abhängigkeit ihrer geometrischen Merkmale gefunden. Stämme werden unter Ausnutzung der Hauptrichtung der Punktverteilung identifiziert. Dafür werden Punktsegmente gesucht, die einen Teil des Baumstamms repräsentieren. Das Ergebnis des Algorithmus sind segmentierte Bäume, welche genutzt werden können um genaue Informationen über die Größe und Position jedes einzelnen Baumes abzuleiten. Einige Beispiele der Ergebnisse werden in der Arbeit angeführt. Die Zuverlässigkeit der Algorithmen und der Methoden im Allgemeinen wurden unter Verwendung von drei Datensätzen, die mit verschiedenen Laserscannersystemen aufgenommen wurden, verifiziert. Die Untersuchung zeigt auch das Potential sowie die Einschränkungen der entwickelten Methoden wenn sie auf verschiedenen Datensätzen angewendet werden. Die Ergebnisse beider Methoden wurden quantitativ bewertet unter Verwendung einer Menge von Maßen, die die Qualität der Fassadenrekonstruktion und Baumextraktion betreffen wie Vollständigkeit und Genauigkeit. Die Genauigkeit der Fassadenrekonstruktion, der Baumstammdetektion, der Erfassung von Baumkronen, sowie ihre Einschränkungen werden diskutiert. Die Ergebnisse zeigen, dass MLS-Punktwolken geeignet sind um städtische Objekte detailreich zu dokumentieren und dass mit automatischen Rekonstruktionsmethoden genaue Messungen der wichtigsten Attribute der Objekte, wie Fensterhöhe und -breite, Flächen, Stammdurchmesser, Baumhöhe und Kronenfläche, erzielt werden können. Der gesamte Ansatz ist geeignet für die Rekonstruktion von Gebäudefassaden und für die korrekte Extraktion von Bäumen sowie ihre Unterscheidung zu anderen urbanen Objekten wie zum Beispiel Straßenschilder oder Leitpfosten. Aus diesem Grund sind die beiden Methoden angemessen um Daten von heterogener Qualität zu verarbeiten. Des Weiteren bieten sie flexible Frameworks für das viele Erweiterungen vorstellbar sind.Up-to-date 3D urban models are becoming increasingly important in various urban application areas, such as urban planning, virtual tourism, and navigation systems. Many of these applications often demand the modelling of 3D buildings, enriched with façade information, and also single trees among other urban objects. Nowadays, Mobile Laser Scanning (MLS) technique is being progressively used to capture objects in urban settings, thus becoming a leading data source for the modelling of these two urban objects. The 3D point clouds of urban scenes consist of large amounts of data representing numerous objects with significant size variability, complex and incomplete structures, and holes (noise and data gaps) or variable point densities. For this reason, novel strategies on processing of mobile laser scanning point clouds, in terms of the extraction and modelling of salient façade structures and trees, are of vital importance. The present study proposes two new methods for the reconstruction of building façades and the extraction of trees from MLS point clouds. The first method aims at the reconstruction of building façades with explicit semantic information such as windows, doors and balconies. It runs automatically during all processing steps. For this purpose, several algorithms are introduced based on the general knowledge on the geometric shape and structural arrangement of façade features. The initial classification has been performed using a local height histogram analysis together with a planar growing method, which allows for classifying points as object and ground points. The point cloud that has been labelled as object points is segmented into planar surfaces that could be regarded as the main entity in the feature recognition process. Knowledge of the building structure is used to define rules and constraints, which provide essential guidance for recognizing façade features and reconstructing their geometric models. In order to recognise features on a wall such as windows and doors, a hole-based method is implemented. Some holes that resulted from occlusion could subsequently be eliminated by means of a new rule-based algorithm. Boundary segments of a feature are connected into a polygon representing the geometric model by introducing a primitive shape based method, in which topological relations are analysed taking into account the prior knowledge about the primitive shapes. Possible outlines are determined from the edge points detected from the angle-based method. The repetitive patterns and similarities are exploited to rectify geometrical and topological inaccuracies of the reconstructed models. Apart from developing the 3D façade model reconstruction scheme, the research focuses on individual tree segmentation and derivation of attributes of urban trees. The second method aims at extracting individual trees from the remaining point clouds. Knowledge about trees specially pertaining to urban areas is used in the process of tree extraction. An innovative shape based approach is developed to transfer this knowledge to machine language. The usage of principal direction for identifying stems is introduced, which consists of searching point segments representing a tree stem. The output of the algorithm is, segmented individual trees that can be used to derive accurate information about the size and locations of each individual tree. The reliability of the two methods is verified against three different data sets obtained from different laser scanner systems. The results of both methods are quantitatively evaluated using a set of measures pertaining to the quality of the façade reconstruction and tree extraction. The performance of the developed algorithms referring to the façade reconstruction, tree stem detection and the delineation of individual tree crowns as well as their limitations are discussed. The results show that MLS point clouds are suited to document urban objects rich in details. From the obtained results, accurate measurements of the most important attributes relevant to the both objects (building façades and trees), such as window height and width, area, stem diameter, tree height, and crown area are obtained acceptably. The entire approach is suitable for the reconstruction of building façades and for the extracting trees correctly from other various urban objects, especially pole-like objects. Therefore, both methods are feasible to cope with data of heterogeneous quality. In addition, they provide flexible frameworks, from which many extensions can be envisioned

Automatic Alignment of 3D Multi-Sensor Point Clouds

Automatic 3D point cloud alignment is a major research topic in photogrammetry, computer vision and computer graphics. In this research, two keypoint feature matching approaches have been developed and proposed for the automatic alignment of 3D point clouds, which have been acquired from different sensor platforms and are in different 3D conformal coordinate systems. The first proposed approach is based on 3D keypoint feature matching. First, surface curvature information is utilized for scale-invariant 3D keypoint extraction. Adaptive non-maxima suppression (ANMS) is then applied to retain the most distinct and well-distributed set of keypoints. Afterwards, every keypoint is characterized by a scale, rotation and translation invariant 3D surface descriptor, called the radial geodesic distance-slope histogram. Similar keypoints descriptors on the source and target datasets are then matched using bipartite graph matching, followed by a modified-RANSAC for outlier removal. The second proposed method is based on 2D keypoint matching performed on height map images of the 3D point clouds. Height map images are generated by projecting the 3D point clouds onto a planimetric plane. Afterwards, a multi-scale wavelet 2D keypoint detector with ANMS is proposed to extract keypoints on the height maps. Then, a scale, rotation and translation-invariant 2D descriptor referred to as the Gabor, Log-Polar-Rapid Transform descriptor is computed for all keypoints. Finally, source and target height map keypoint correspondences are determined using a bi-directional nearest neighbour matching, together with the modified-RANSAC for outlier removal. Each method is assessed on multi-sensor, urban and non-urban 3D point cloud datasets. Results show that unlike the 3D-based method, the height map-based approach is able to align source and target datasets with differences in point density, point distribution and missing point data. Findings also show that the 3D-based method obtained lower transformation errors and a greater number of correspondences when the source and target have similar point characteristics. The 3D-based approach attained absolute mean alignment differences in the range of 0.23m to 2.81m, whereas the height map approach had a range from 0.17m to 1.21m. These differences meet the proximity requirements of the data characteristics and the further application of fine co-registration approaches

Proceedings. 9th 3DGeoInfo Conference 2014, [11-13 November 2014, Dubai]

It is known that, scientific disciplines such as geology, geophysics, and reservoir exploration intrinsically use 3D geo-information in their models and simulations. However, 3D geo-information is also urgently needed in many traditional 2D planning areas such as civil engineering, city and infrastructure modeling, architecture, environmental planning etc. Altogether, 3DGeoInfo is an emerging technology that will greatly influence the market within the next few decades. The 9th International 3DGeoInfo Conference aims at bringing together international state-of-the-art researchers and practitioners facilitating the dialogue on emerging topics in the field of 3D geo-information. The conference in Dubai offers an interdisciplinary forum of sub- and above-surface 3D geo-information researchers and practitioners dealing with data acquisition, modeling, management, maintenance, visualization, and analysis of 3D geo-information

Rapid Inspection of Pavement Markings Using Mobile Laser Scanning Point Clouds

Intelligent Transportation System (ITS) is the combination of information technology, sensors and communications for more efficient, safer, more secure and more eco-friendly surface transport. One of the most viable forms of ITS is the driverless car, which exist mainly as prototypes. Serval automobile manufacturers (e.g. Ford, GM, BMW, Toyota, Tesla, Honda) and non-automobile companies (e.g. Apple, Google, Nokia, Baidu, Huawei) have invested in this field, and wider commercialization of the driverless car is estimated in 2025 to 2030. Currently, the key elements of the driverless car are the sensors and a prior 3D map. The sensors mounted on the vehicle are the “eyes” of the driverless car to capture the 3D data of its environment. Comparing its environment and a pre-prepared prior known 3D map, the driverless car can distinguish moving targets (e.g. vehicles, pedestrians) and permanent surface features (e.g. buildings, trees, roads, traffic signs) and take relevant actions. With a centimetre-accuracy prior map, the intractable perception problem is transformed into a solvable localization task. The most important technology for generating the prior map is Mobile Laser Scanning (MLS). MLS technology can safely and rapidly acquire highly dense and accurate georeferenced 3D point clouds with the measurement of surface reflectivity. Therefore, the 3D point clouds with intensity data not only offer the detailed 3D surface of the road but also contains pavement marking information that are embedded in the prior map for automatic navigation. Relevant researches have been focused on the pavement marking extraction from MLS data to collect, update and maintain the 3D prior maps. However, the accuracy and efficiency of automatic extraction of pavement markings can be further improved by intensity correction and window-based enhancement. Thus, this study aims at building a robust method for semi-automated information extraction of pavement markings detected from MLS point clouds. The proposed workflow consists of three components: preprocessing, extraction, and classification. In preprocessing, the 3D MLS point clouds are converted into the radiometrically corrected and enhanced 2D intensity imagery of the road surface. Then the pavement markings are automatically extracted with the intensity using a set of algorithms, including Otsu’s thresholding, neighbour-counting filtering, and region growing. Finally, the extracted pavement markings are classified with the geometric parameters using a manually defined decision tree. Case studies are conducted using the MLS datasets acquired in Kingston (Ontario, Canada) and Xiamen (Fujian, China), respectively, with significantly different road environments by two RIEGL VMX-450 systems. The results demonstrated that the proposed workflow and method can achieve 93% in completeness, 95% in correctness, and 94% in F-score respectively when using Xiamen dataset, while 84%, 93%, 89% respectively when using Kingston dataset

Optimising mobile laser scanning for underground mines

Despite several technological advancements, underground mines are still largely relied on visual inspections or discretely placed direct-contact measurement sensors for routine monitoring. Such approaches are manual and often yield inconclusive, unreliable and unscalable results besides exposing mine personnel to field hazards. Mobile laser scanning (MLS) promises an automated approach that can generate comprehensive information by accurately capturing large-scale 3D data. Currently, the application of MLS has relatively remained limited in mining due to challenges in the post-registration of scans and the unavailability of suitable processing algorithms to provide a fully automated mapping solution. Additionally, constraints such as the absence of a spatial positioning network and the deficiency of distinguishable features in underground mining spaces pose challenges in mobile mapping. This thesis aims to address these challenges in mine inspections by optimising different aspects of MLS: (1) collection of large-scale registered point cloud scans of underground environments, (2) geological mapping of structural discontinuities, and (3) inspection of structural support features. Firstly, a spatial positioning network was designed using novel three-dimensional unique identifiers (3DUID) tags and a 3D registration workflow (3DReG), to accurately obtain georeferenced and coregistered point cloud scans, enabling multi-temporal mapping. Secondly, two fully automated methods were developed for mapping structural discontinuities from point cloud scans – clustering on local point descriptors (CLPD) and amplitude and phase decomposition (APD). These methods were tested on both surface and underground rock mass for discontinuity characterisation and kinematic analysis of the failure types. The developed algorithms significantly outperformed existing approaches, including the conventional method of compass and tape measurements. Finally, different machine learning approaches were used to automate the recognition of structural support features, i.e. roof bolts from point clouds, in a computationally efficient manner. Roof bolts being mapped from a scanned point cloud provided an insight into their installation pattern, which underpinned the applicability of laser scanning to inspect roof supports rapidly. Overall, the outcomes of this study lead to reduced human involvement in field assessments of underground mines using MLS, demonstrating its potential for routine multi-temporal monitoring