Using Assembled 2D LiDAR Data for Single Plant Detection

A 2D laser scanner was mounted on the front of the small 4-wheel autonomous robot with differential steering, in an angle of 30 degrees pointing downwards. The machine was able to drive between maize rows and collect timestamped data simultaneously. The position of the vehicle was tracked by a highly precise total station. The data of the total station and the laser scanner was fused to generate a 3D point cloud. This 3D representation was used to search for single plant positions, what could later be used for additional applications like single plant treatment and precision weeding. First all points belonging to the ground plane were removed. Afterwards outliers were filtered. For separating the resulting points, a k-d tree clustering was used. Of each single point cloud cluster the 3D centroid was evaluated and assumed as the resulting plant position. This was done on three different growth stages of the plants. Results showed good detection rates up to 70.7 % with a root mean square error of 3.6 cm, precise enough to allow single plant treatment

Using Lidar Intensity for Robot Navigation

We present Multi-Layer Intensity Map, a novel 3D object representation for robot perception and autonomous navigation. Intensity maps consist of multiple stacked layers of 2D grid maps each derived from reflected point cloud intensities corresponding to a certain height interval. The different layers of intensity maps can be used to simultaneously estimate obstacles' height, solidity/density, and opacity. We demonstrate that intensity maps' can help accurately differentiate obstacles that are safe to navigate through (e.g. beaded/string curtains, pliable tall grass), from ones that must be avoided (e.g. transparent surfaces such as glass walls, bushes, trees, etc.) in indoor and outdoor environments. Further, to handle narrow passages, and navigate through non-solid obstacles in dense environments, we propose an approach to adaptively inflate or enlarge the obstacles detected on intensity maps based on their solidity, and the robot's preferred velocity direction. We demonstrate these improved navigation capabilities in real-world narrow, dense environments using a real Turtlebot and Boston Dynamics Spot robots. We observe significant increases in success rates to more than 50%, up to a 9.5% decrease in normalized trajectory length, and up to a 22.6% increase in the F-score compared to current navigation methods using other sensor modalities.Comment: 9 pages, 7 figure

LiDAR-based Weather Detection: Automotive LiDAR Sensors in Adverse Weather Conditions

Technologische Verbesserungen erhöhen den Automatisierungsgrad von Fahrzeugen. Der natürliche Schritt ist dabei, den Fahrer dort zu unterstützen, wo er es am meisten wünscht: bei schlechtem Wetter. Das Wetter beeinflusst alle Sensoren, die zur Wahrnehmung der Umgebung verwendet werden, daher ist es entscheidend, diese Effekte zu berücksichtigen und abzuschwächen. Die vorliegende Dissertation konzentriert sich auf die gerade entstehende Technologie der automobilen Light Detection and Ranging (LiDAR)-Sensoren und trägt zur Entwicklung von autonomen Fahrzeugen bei, die in der Lage sind, unter verschiedenen Wetterbedingungen zu fahren. Die Grundlage ist der erste LiDAR-Punktwolken-Datensatz mit dem Schwerpunkt auf schlechte Wetterbedingungen, welcher punktweise annonatatierte Wetterinformationen enthält, während er unter kontrollierten Wetterbedingungen aufgezeichnet wurde. Dieser Datensatz wird durch eine neuartige Wetter-Augmentation erweitert, um realistische Wettereffekte erzeugen zu können. Ein neuartiger Ansatz zur Klassifizierung des Wetterzustands und der erste CNN-basierte Entrauschungsalgorithmus werden entwickelt. Das Ergebnis ist eine genaue Vorhersage des Wetterstatus und eine Verbesserung der Punktwolkenqualität. Kontrollierte Umgebungen unter verschiedenen Wetterbedingungen ermöglichen die Evaluierung der oben genannten Ansätze und liefern wertvolle Informationen für das automatisierte und autonome Fahren

Advances in Waveform and Photon Counting Lidar Processing for Forest Vegetation Applications

Full waveform (FW) and photon counting LiDAR (PCL) data have garnered greater attention due to increasing data availability, a wealth of information they contain and promising prospects for large scale vegetation mapping. However, many factors such as complex processing steps and scarce non-proprietary tools preclude extensive and practical uses of these data for vegetation characterization. Therefore, the overall goal of this study is to develop algorithms to process FW and PCL data and to explore their potential in real-world applications. Study I explored classical waveform decomposition methods such as the Gaussian decomposition, Richardson–Lucy (RL) deconvolution and a newly introduced optimized Gold deconvolution to process FW LiDAR data. Results demonstrated the advantages of the deconvolution and decomposition method, and the three approaches generated satisfactory results, while the best performances varied when different criteria were used. Built upon Study I, Study II applied the Bayesian non-linear modeling concepts for waveform decomposition and quantified the propagation of error and uncertainty along the processing steps. The performance evaluation and uncertainty analysis at the parameter, derived point cloud and surface model levels showed that the Bayesian decomposition could enhance the credibility of decomposition results in a probabilistic sense to capture the true error of estimates and trace the uncertainty propagation along the processing steps. In study III, we exploited FW LiDAR data to classify tree species through integrating machine learning methods (the Random forests (RF) and Conditional inference forests (CF)) and Bayesian inference method. Results of classification accuracy highlighted that the Bayesian method was a superior alternative to machine learning methods, and rendered users with more confidence for interpreting and applying classification results to real-world tasks such as forest inventory. Study IV focused on developing a framework to derive terrain elevation and vegetation canopy height from test-bed sensor data and to pre-validate the capacity of the upcoming Ice, Cloud and Land Elevation Satellite-2 (ICESat-2) mission. The methodology developed in this study illustrates plausible ways of processing the data that are structurally similar to expected ICESat-2 data and holds the potential to be a benchmark for further method adjustment once genuine ICESat-2 are available

Single-Sensor Solution to Tree Species Classification Using Multispectral Airborne Laser Scanning

This paper investigated the potential of multispectral airborne laser scanning (ALS) data for individual tree detection and tree species classification. The aim was to develop a single-sensor solution for forest mapping that is capable of providing species-specific information, required for forest management and planning purposes. Experiments were conducted using 1903 ground measured trees from 22 sample plots and multispectral ALS data, acquired with an Optech Titan scanner over a boreal forest, mainly consisting of Scots pine (Pinus Sylvestris), Norway spruce (Picea Abies), and birch (Betula sp.), in southern Finland. ALS-features used as predictors for tree species were extracted from segmented tree objects and used in random forest classification. Different combinations of features, including point cloud features, and intensity features of single and multiple channels, were tested. Among the field-measured trees, 61.3% were correctly detected. The best overall accuracy (OA) of tree species classification achieved for correctly-detected trees was 85.9% (Kappa = 0.75), using a point cloud and single-channel intensity features combination, which was not significantly different from the ones that were obtained either using all features (OA = 85.6%, Kappa = 0.75), or single-channel intensity features alone (OA = 85.4%, Kappa = 0.75). Point cloud features alone achieved the lowest accuracy, with an OA of 76.0%. Field-measured trees were also divided into four categories. An examination of the classification accuracy for four categories of trees showed that isolated and dominant trees can be detected with a detection rate of 91.9%, and classified with a high overall accuracy of 90.5%. The corresponding detection rate and accuracy were 81.5% and 89.8% for a group of trees, 26.4% and 79.1% for trees next to a larger tree, and 7.2% and 53.9% for trees situated under a larger tree, respectively. The results suggest that Channel 2 (1064 nm) contains more information for separating pine, spruce, and birch, followed by channel 1 (1550 nm) and channel 3 (532 nm) with an overall accuracy of 81.9%, 78.3%, and 69.1%, respectively. Our results indicate that the use of multispectral ALS data has great potential to lead to a single-sensor solution for forest mapping.Peer reviewe

Hierarchical higher order crf for the classification of airborne lidar point clouds in urban areas

We propose a novel hierarchical approach for the classification of airborne 3D lidar points. Spatial and semantic context is incorporated via a two-layer Conditional Random Field (CRF). The first layer operates on a point level and utilises higher order cliques. Segments are generated from the labelling obtained in this way. They are the entities of the second layer, which incorporates larger scale context. The classification result of the segments is introduced as an energy term for the next iteration of the point-based layer. This framework iterates and mutually propagates context to improve the classification results. Potentially wrong decisions can be revised at later stages. The output is a labelled point cloud as well as segments roughly corresponding to object instances. Moreover, we present two new contextual features for the segment classification: the distance and the orientation of a segment with respect to the closest road. It is shown that the classification benefits from these features. In our experiments the hierarchical framework improve the overall accuracies by 2.3% on a point-based level and by 3.0% on a segment-based level, respectively, compared to a purely point-based classification

Study of Seasonal change and Water Stress Condition in Plant Leaf Using Polarimetric Lidar Measurement

Study of vegetation is of great importance to the improvement of agriculture and forest management. Although there have been various attempts to characterize vegetation using remote sensing techniques, polarimetric lidar is a novel remote sensing tool that has shown potential in vegetation remote sensing. In this thesis, a near-infrared polarimetric lidar at 1064 nm was used to investigate the effects of seasonal change and water stress condition on plant leaves. Two variables, time and water content, affected the plant leaf laser depolarization ratio measurement. The first study focused on the maple tree in order to figure out how seasonal change affected the maple leaf depolarization. Seasonal trendline was obtained and revealed an overall downward trend over time. In the second study, the leaves from maple, lemon, and rubber trees were investigated to study the effect of water stress on the depolarization ratio. It was discovered that the leaf depolarization ratio increased for more water content and went down for less water content. In addition, leaf samples were collected in the morning, afternoon, and evening, respectively, to study the diurnal change. Statistical analysis suggested that depolarization ratio did not change significantly for the different times of a day. It was suggested that the seasonal change had a greater effect on depolarization than the diurnal change. This study demonstrates that the near-infrared polarimetric lidar system has an ability to remotely characterize the vegetation internal conditions that may not be visible to the human eyes. Furthermore, the lidar system has the potential to differentiate the various plant species based on the depolarization ratio. In conclusion, the polarimetric lidar system at 1064-nm is an effective and sensitive enough remote sensing tool which can be widely used in active remote sensing

Calibration of full-waveform airborne laser scanning data for 3D object segmentation

Phd ThesisAirborne Laser Scanning (ALS) is a fully commercial technology, which has seen rapid uptake from the photogrammetry and remote sensing community to classify surface features and enhance automatic object recognition and extraction processes. 3D object segmentation is considered as one of the major research topics in the field of laser scanning for feature recognition and object extraction applications. The demand for automatic segmentation has significantly increased with the emergence of full-waveform (FWF) ALS, which potentially offers an unlimited number of return echoes. FWF has shown potential to improve available segmentation and classification techniques through exploiting the additional physical observables which are provided alongside the standard geometric information. However, use of the FWF additional information is not recommended without prior radiometric calibration, taking into consideration all the parameters affecting the backscattered energy. The main focus of this research is to calibrate the additional information from FWF to develop the potential of point clouds for segmentation algorithms. Echo amplitude normalisation as a function of local incidence angle was identified as a particularly critical aspect, and a novel echo amplitude normalisation approach, termed the Robust Surface Normal (RSN) method, has been developed. Following the radar equation, a comprehensive radiometric calibration routine is introduced to account for all variables affecting the backscattered laser signal. Thereafter, a segmentation algorithm is developed, which utilises the raw 3D point clouds to estimate the normal for individual echoes based on the RSN method. The segmentation criterion is selected as the normal vector augmented by the calibrated backscatter signals. The developed segmentation routine aims to fully integrate FWF data to improve feature recognition and 3D object segmentation applications. The routine was tested over various feature types from two datasets with different properties to assess its potential. The results are compared to those delivered through utilizing only geometric information, without the additional FWF radiometric information, to assess performance over existing methods. The results approved the potential of the FWF additional observables to improve segmentation algorithms. The new approach was validated against manual segmentation results, revealing a successful automatic implementation and achieving an accuracy of 82%

Influence of complex environments on LiDAR-Based robot navigation

La navigation sécuritaire et efficace des robots mobiles repose grandement sur l’utilisation des capteurs embarqués. L’un des capteurs qui est de plus en plus utilisé pour cette tâche est le Light Detection And Ranging (LiDAR). Bien que les recherches récentes montrent une amélioration des performances de navigation basée sur les LiDARs, faire face à des environnements non structurés complexes ou des conditions météorologiques difficiles reste problématique. Dans ce mémoire, nous présentons une analyse de l’influence de telles conditions sur la navigation basée sur les LiDARs. Notre première contribution est d’évaluer comment les LiDARs sont affectés par les flocons de neige durant les tempêtes de neige. Pour ce faire, nous créons un nouvel ensemble de données en faisant l’acquisition de données durant six précipitations de neige. Une analyse statistique de ces ensembles de données, nous caractérisons la sensibilité de chaque capteur et montrons que les mesures de capteurs peuvent être modélisées de manière probabilistique. Nous montrons aussi que les précipitations de neige ont peu d’influence au-delà de 10 m. Notre seconde contribution est d’évaluer l’impact de structures tridimensionnelles complexes présentes en forêt sur les performances d’un algorithme de reconnaissance d’endroits. Nous avons acquis des données dans un environnement extérieur structuré et en forêt, ce qui permet d’évaluer l’influence de ces derniers sur les performances de reconnaissance d’endroits. Notre hypothèse est que, plus deux balayages laser sont proches l’un de l’autre, plus la croyance que ceux-ci proviennent du même endroit sera élevée, mais modulé par le niveau de complexité de l’environnement. Nos expériences confirment que la forêt, avec ses réseaux de branches compliqués et son feuillage, produit plus de données aberrantes et induit une chute plus rapide des performances de reconnaissance en fonction de la distance. Notre conclusion finale est que, les environnements complexes étudiés influencent négativement les performances de navigation basée sur les LiDARs, ce qui devrait être considéré pour développer des algorithmes de navigation robustes.To ensure safe and efficient navigation, mobile robots heavily rely on their ability to use on-board sensors. One such sensor, increasingly used for robot navigation, is the Light Detection And Ranging (LiDAR). Although recent research showed improvement in LiDAR-based navigation, dealing with complex unstructured environments or difficult weather conditions remains problematic. In this thesis, we present an analysis of the influence of such challenging conditions on LiDAR-based navigation. Our first contribution is to evaluate how LiDARs are affected by snowflakes during snowstorms. To this end, we create a novel dataset by acquiring data during six snowfalls using four sensors simultaneously. Based on statistical analysis of this dataset, we characterized the sensitivity of each device and showed that sensor measurements can be modelled in a probabilistic manner. We also showed that falling snow has little impact beyond a range of 10 m. Our second contribution is to evaluate the impact of complex of three-dimensional structures, present in forests, on the performance of a LiDAR-based place recognition algorithm. We acquired data in structured outdoor environment and in forest, which allowed evaluating the impact of the environment on the place recognition performance. Our hypothesis was that the closer two scans are acquired from each other, the higher the belief that the scans originate from the same place will be, but modulated by the level of complexity of the environments. Our experiments confirmed that forests, with their intricate network of branches and foliage, produce more outliers and induce recognition performance to decrease more quickly with distance when compared with structured outdoor environment. Our conclusion is that falling snow conditions and forest environments negatively impact LiDAR-based navigation performance, which should be considered to develop robust navigation algorithms

A Case Study of Microphysical Structures and Hydrometeor Phase in Convection Using Radar Doppler Spectra at Darwin, Australia

To understand the microphysical processes that impact diabatic heating and cloud lifetimes in convection, we need to characterize the spatial distribution of supercooled liquid water. To address this observational challenge, vertically pointing active sensors at the Darwin Atmospheric Radiation Measurement (ARM) site are used to classify cloud phase within a deep convective cloud in a shallow to deep convection transitional case. The cloud cannot be fully observed by a lidar due to signal attenuation. Thus we develop an objective method for identifying hydrometeor classes, including mixed-phase conditions, using k-means clustering on parameters that describe the shape of the Doppler spectra from vertically pointing Ka band cloud radar. This approach shows that multiple, overlapping mixed-phase layers exist within the cloud, rather than a single region of supercooled liquid, indicating complexity to how ice growth and diabatic heating occurs in the vertical structure of the cloud