FLAT2D: Fast localization from approximate transformation into 2D

Many autonomous vehicles require precise localization into a prior map in order to support planning and to leverage semantic information within those maps (e.g. that the right lane is a turn-only lane.) A popular approach in automotive systems is to use infrared intensity maps of the ground surface to localize, making them susceptible to failures when the surface is obscured by snow or when the road is repainted. An emerging alternative is to localize based on the 3D structure around the vehicle; these methods are robust to these types of changes, but the maps are costly both in terms of storage and the computational cost of matching. In this paper, we propose a fast method for localizing based on 3D structure around the vehicle using a 2D representation. This representation retains many of the advantages of "full" matching in 3D, but comes with dramatically lower space and computational requirements. We also introduce a variation of Graph-SLAM tailored to support localization, allowing us to make use of graph-based error-recovery techniques in our localization estimate. Finally, we present real-world localization results for both an indoor mobile robotic platform and an autonomous golf cart, demonstrating that autonomous vehicles do not need full 3D matching to accurately localize in the environment

Assessing temporal behavior in lidar point clouds of urban environments

Self-driving cars and robots that run autonomously over long periods of time need high-precision and up-to-date models of the changing environment. The main challenge for creating long term maps of dynamic environments is to identify changes and adapt the map continuously. Changes can occur abruptly, gradually, or even periodically. In this work, we investigate how dense mapping data of several epochs can be used to identify the temporal behavior of the environment. This approach anticipates possible future scenarios where a large fleet of vehicles is equipped with sensors which continuously capture the environment. This data is then being sent to a cloud based infrastructure, which aligns all datasets geometrically and subsequently runs scene analysis on it, among these being the analysis for temporal changes of the environment. Our experiments are based on a LiDAR mobile mapping dataset which consists of 150 scan strips (a total of about 1 billion points), which were obtained in multiple epochs. Parts of the scene are covered by up to 28 scan strips. The time difference between the first and last epoch is about one year. In order to process the data, the scan strips are aligned using an overall bundle adjustment, which estimates the surface (about one billion surface element unknowns) as well as 270,000 unknowns for the adjustment of the exterior orientation parameters. After this, the surface misalignment is usually below one centimeter. In the next step, we perform a segmentation of the point clouds using a region growing algorithm. The segmented objects and the aligned data are then used to compute an occupancy grid which is filled by tracing each individual LiDAR ray from the scan head to every point of a segment. As a result, we can assess the behavior of each segment in the scene and remove voxels from temporal objects from the global occupancy grid.DFG/GRK/215

Classification and Change Detection in Mobile Mapping LiDAR Point Clouds

Creating 3D models of the static environment is an important task for the advancement of driver assistance systems and autonomous driving. In this work, a static reference map is created from a Mobile Mapping “light detection and ranging” (LiDAR) dataset. The data was obtained in 14 measurement runs from March to October 2017 in Hannover and consists in total of about 15 billion points. The point cloud data are first segmented by region growing and then processed by a random forest classification, which divides the segments into the five static classes (“facade”, “pole”, “fence”, “traffic sign”, and “vegetation”) and three dynamic classes (“vehicle”, “bicycle”, “person”) with an overall accuracy of 94%. All static objects are entered into a voxel grid, to compare different measurement epochs directly. In the next step, the classified voxels are combined with the result of a visibility analysis. Therefore, we use a ray tracing algorithm to detect traversed voxels and differentiate between empty space and occlusion. Each voxel is classified as suitable for the static reference map or not by its object class and its occupation state during different epochs. Thereby, we avoid to eliminate static voxels which were occluded in some of the measurement runs (e.g. parts of a building occluded by a tree). However, segments that are only temporarily present and connected to static objects, such as scaffolds or awnings on buildings, are not included in the reference map. Overall, the combination of the classification with the subsequent entry of the classes into a voxel grid provides good and useful results that can be updated by including new measurement data

Contributions to Intelligent Scene Understanding of Unstructured Environments from 3D lidar sensors

Además, la viabilidad de este enfoque es evaluado mediante la implementación de cuatro tipos de clasificadores de aprendizaje supervisado encontrados en métodos de procesamiento de escenas: red neuronal, máquina de vectores de soporte, procesos gaussianos, y modelos de mezcla gaussiana. La segmentación de objetos es un paso más allá hacia el entendimiento de escena, donde conjuntos de puntos 3D correspondientes al suelo y otros objetos de la escena son aislados. La tesis propone nuevas contribuciones a la segmentación de nubes de puntos basados en mapas de vóxeles caracterizados geométricamente. En concreto, la metodología propuesta se compone de dos pasos: primero, una segmentación del suelo especialmente diseñado para entornos naturales; y segundo, el posterior aislamiento de objetos individuales. Además, el método de segmentación del suelo es integrado en una nueva técnica de mapa de navegabilidad basado en cuadrícula de ocupación el cuál puede ser apropiado para robots móviles en entornos naturales. El diseño y desarrollo de un nuevo y asequible sensor lidar 3D de alta resolución también se ha propuesto en la tesis. Los nuevos MBLs, tales como los desarrollados por Velodyne, están siendo cada vez más un tipo de sensor 3D asequible y popular que ofrece alto ratio de datos en un campo de visión vertical (FOV) limitado. El diseño propuesto consiste en una plataforma giratoria que mejora la resolución y el FOV vertical de un Velodyne VLP-16 de 16 haces. Además, los complejos patrones de escaneo producidos por configuraciones de MBL que rotan se analizan tanto en simulaciones de esfera hueca como en escáneres reales en entornos representativos. Fecha de Lectura de Tesis: 11 de julio 2018.Ingeniería de Sistemas y Automática Resumen tesis: Los sensores lidar 3D son una tecnología clave para navegación, localización, mapeo y entendimiento de escenas en vehículos no tripulados y robots móviles. Esta tecnología, que provee nubes de puntos densas, puede ser especialmente adecuada para nuevas aplicaciones en entornos naturales o desestructurados, tales como búsqueda y rescate, exploración planetaria, agricultura, o exploración fuera de carretera. Esto es un desafío como área de investigación que incluye disciplinas que van desde el diseño de sensor a la inteligencia artificial o el aprendizaje automático (machine learning). En este contexto, esta tesis propone contribuciones al entendimiento inteligente de escenas en entornos desestructurados basado en medidas 3D de distancia a nivel del suelo. En concreto, las contribuciones principales incluyen nuevas metodologías para la clasificación de características espaciales, segmentación de objetos, y evaluación de navegabilidad en entornos naturales y urbanos, y también el diseño y desarrollo de un nuevo lidar rotatorio multi-haz (MBL). La clasificación de características espaciales es muy relevante porque es extensamente requerida como un paso fundamental previo a los problemas de entendimiento de alto nivel de una escena. Las contribuciones de la tesis en este respecto tratan de mejorar la eficacia, tanto en carga computacional como en precisión, de clasificación de aprendizaje supervisado de características de forma espacial (forma tubular, plana o difusa) obtenida mediante el análisis de componentes principales (PCA). Esto se ha conseguido proponiendo un concepto eficiente de vecindario basado en vóxel en una contribución original que define los procedimientos de aprendizaje “offline” y clasificación “online” a la vez que cinco definiciones alternativas de vectores de características basados en PCA

Volume Component Analysis for Classification of LiDAR Data

One of the most difficult challenges of working with LiDAR data is the large amount of data points that are produced. Analysing these large data sets is an extremely time consuming process. For this reason, automatic perception of LiDAR scenes is a growing area of research. Currently, most LiDAR feature extraction relies on geometrical features specific to the point cloud of interest. These geometrical features are scene-specific, and often rely on the scale and orientation of the object for classification. This paper proposes a robust method for reduced dimensionality feature extraction of 3D objects using a volume component analysis (VCA) approach. This VCA approach is based on principal component analysis (PCA). PCA is a method of reduced feature extraction that computes a covariance matrix from the original input vector. The eigenvectors corresponding to the largest eigenvalues of the covariance matrix are used to describe an image. Block-based PCA is an adapted method for feature extraction in facial images because PCA, when performed in local areas of the image, can extract more significant features than can be extracted when the entire image is considered. The image space is split into several of these blocks, and PCA is computed individually for each block. This VCA proposes that a LiDAR point cloud can be represented as a series of voxels whose values correspond to the point density within that relative location. From this voxelized space, block-based PCA is used to analyze sections of the space where the sections, when combined, will represent features of the entire 3-D object. These features are then used as the input to a support vector machine which is trained to identify four classes of objects, vegetation, vehicles, buildings and barriers with an overall accuracy of 93.8%

VISUAL SEMANTIC SEGMENTATION AND ITS APPLICATIONS

This dissertation addresses the difficulties of semantic segmentation when dealing with an extensive collection of images and 3D point clouds. Due to the ubiquity of digital cameras that help capture the world around us, as well as the advanced scanning techniques that are able to record 3D replicas of real cities, the sheer amount of visual data available presents many opportunities for both academic research and industrial applications. But the mere quantity of data also poses a tremendous challenge. In particular, the problem of distilling useful information from such a large repository of visual data has attracted ongoing interests in the fields of computer vision and data mining. Structural Semantics are fundamental to understanding both natural and man-made objects. Buildings, for example, are like languages in that they are made up of repeated structures or patterns that can be captured in images. In order to find these recurring patterns in images, I present an unsupervised frequent visual pattern mining approach that goes beyond co-location to identify spatially coherent visual patterns, regardless of their shape, size, locations and orientation. First, my approach categorizes visual items from scale-invariant image primitives with similar appearance using a suite of polynomial-time algorithms that have been designed to identify consistent structural associations among visual items, representing frequent visual patterns. After detecting repetitive image patterns, I use unsupervised and automatic segmentation of the identified patterns to generate more semantically meaningful representations. The underlying assumption is that pixels capturing the same portion of image patterns are visually consistent, while pixels that come from different backdrops are usually inconsistent. I further extend this approach to perform automatic segmentation of foreground objects from an Internet photo collection of landmark locations. New scanning technologies have successfully advanced the digital acquisition of large-scale urban landscapes. In addressing semantic segmentation and reconstruction of this data using LiDAR point clouds and geo-registered images of large-scale residential areas, I develop a complete system that simultaneously uses classification and segmentation methods to first identify different object categories and then apply category-specific reconstruction techniques to create visually pleasing and complete scene models

AUTOMATED TREE-LEVEL FOREST QUANTIFICATION USING AIRBORNE LIDAR

Traditional forest management relies on a small field sample and interpretation of aerial photography that not only are costly to execute but also yield inaccurate estimates of the entire forest in question. Airborne light detection and ranging (LiDAR) is a remote sensing technology that records point clouds representing the 3D structure of a forest canopy and the terrain underneath. We present a method for segmenting individual trees from the LiDAR point clouds without making prior assumptions about tree crown shapes and sizes. We then present a method that vertically stratifies the point cloud to an overstory and multiple understory tree canopy layers. Using the stratification method, we modeled the occlusion of higher canopy layers with respect to point density. We also present a distributed computing approach that enables processing the massive data of an arbitrarily large forest. Lastly, we investigated using deep learning for coniferous/deciduous classification of point cloud segments representing individual tree crowns. We applied the developed methods to the University of Kentucky Robinson Forest, a natural, majorly deciduous, closed-canopy forest. 90% of overstory and 47% of understory trees were detected with false positive rates of 14% and 2% respectively. Vertical stratification improved the detection rate of understory trees to 67% at the cost of increasing their false positive rate to 12%. According to our occlusion model, a point density of about 170 pt/m² is needed to segment understory trees located in the third layer as accurately as overstory trees. Using our distributed processing method, we segmented about two million trees within a 7400-ha forest in 2.5 hours using 192 processing cores, showing a speedup of ~170. Our deep learning experiments showed high classification accuracies (~82% coniferous and ~90% deciduous) without the need to manually assemble the features. In conclusion, the methods developed are steps forward to remote, accurate quantification of large natural forests at the individual tree level