Data-driven covariance estimation for the iterative closest point algorithm

Les nuages de points en trois dimensions sont un format de données très commun en robotique mobile. Ils sont souvent produits par des capteurs spécialisés de type lidar. Les nuages de points générés par ces capteurs sont utilisés dans des tâches impliquant de l’estimation d’état, telles que la cartographie ou la localisation. Les algorithmes de recalage de nuages de points, notamment l’algorithme ICP (Iterative Closest Point), nous permettent de prendre des mesures d’égo-motion nécessaires à ces tâches. La fusion des recalages dans des chaînes existantes d’estimation d’état dépend d’une évaluation précise de leur incertitude. Cependant, les méthodes existantes d’estimation de l’incertitude se prêtent mal aux données en trois dimensions. Ce mémoire vise à estimer l’incertitude de recalages 3D issus d’Iterative Closest Point (ICP). Premièrement, il pose des fondations théoriques desquelles nous pouvons articuler une estimation de la covariance. Notamment, il révise l’algorithme ICP, avec une attention spéciale sur les parties qui sont importantes pour l’estimation de la covariance. Ensuite, un article scientifique inséré présente CELLO-3D, notre algorithme d’estimation de la covariance d’ICP. L’article inséré contient une validation expérimentale complète du nouvel algorithme. Il montre que notre algorithme performe mieux que les méthodes existantes dans une grande variété d’environnements. Finalement, ce mémoire est conclu par des expérimentations supplémentaires, qui sont complémentaires à l’article.Three-dimensional point clouds are an ubiquitous data format in robotics. They are produced by specialized sensors such as lidars or depth cameras. The point clouds generated by those sensors are used for state estimation tasks like mapping and localization. Point cloud registration algorithms, such as Iterative Closest Point (ICP), allow us to make ego-motion measurements necessary to those tasks. The fusion of ICP registrations in existing state estimation frameworks relies on an accurate estimation of their uncertainty. Unfortunately, existing covariance estimation methods often scale poorly to the 3D case. This thesis aims to estimate the uncertainty of ICP registrations for 3D point clouds. First, it poses theoretical foundations from which we can articulate a covariance estimation method. It reviews the ICP algorithm, with a special focus on the parts of it that are pertinent to covariance estimation. Then, an inserted article introduces CELLO-3D, our data-driven covariance estimation method for ICP. The article contains a thorough experimental validation of the new algorithm. The latter is shown to perform better than existing covariance estimation techniques in a wide variety of environments. Finally, this thesis comprises supplementary experiments, which complement the article

Quality of sawmilling output predictions according to the size of the lot - The size matters!

Lors de l'évaluation de modèles d'apprentissage automatique supervisé, on considère généralement le rendement de prédiction moyen obtenu sur les tests individuels comme mesure de choix. Toutefois, lorsque le modèle est destiné à prédire quels produits du bois seront obtenus lors du sciage de certains billots, c'est généralement la performance pour un lot complet qui importe. Dans cet article, nous montrons l'impact de cette nuance en termes d'évaluation du modèle. En fait, la qualité d'une prédiction (globale) s'améliore considérablement lorsque l'on augmente la taille des lots, ce qui offre un solide soutien à l'utilisation de ces modèles en pratique.When comparing supervised learning models, one generally considers the average prediction performance obtained over individual test samples. However, when using machine learning to predict which lumber products will be obtained when sawing logs, it is usually the performance over the entire lot that matters. In this paper, we show the impact of this by evaluating a model performance for various batch sizes. The quality of a (global) prediction improves tremendously when batch size increases, which offers a strong support for the use of such models in practice

On the use of rapid-scan, low point density terrestrial laser scanning (TLS) for structural assessment of complex forest environments

Forests fulfill an important role in natural ecosystems, e.g., they provide food, fiber, habitat, and biodiversity, all of which contribute to stable ecosystems. Assessing and modeling the structure and characteristics in forests can lead to a better understanding and management of these resources. Traditional methods for collecting forest traits, known as “forest inventory”, is achieved using rough proxies, such as stem diameter, tree height, and foliar coverage; such parameters are limited in their ability to capture fine-scale structural variation in forest environments. It is in this context that terrestrial laser scanning (TLS) has come to the fore as a tool for addressing the limitations of traditional forest structure evaluation methods. However, there is a need for improving TLS data processing methods. In this work, we developed algorithms to assess the structure of complex forest environments – defined by their stem density, intricate root and stem structures, uneven-aged nature, and variable understory - using data collected by a low-cost, portable TLS system, the Compact Biomass Lidar (CBL). The objectives of this work are listed as follow: 1. Assess the utility of terrestrial lidar scanning (TLS) to accurately map elevation changes (sediment accretion rates) in mangrove forest; 2. Evaluate forest structural attributes, e.g., stems and roots, in complex forest environments toward biophysical characterization of such forests; and 3. Assess canopy-level structural traits (leaf area index; leaf area density) in complex forest environments to estimate biomass in rapidly changing environments. The low-cost system used in this research provides lower-resolution data, in terms of scan angular resolution and resulting point density, when compared to higher-cost commercial systems. As a result, the algorithms developed for evaluating the data collected by such systems should be robust to issues caused by low-resolution 3D point cloud data. The data used in various parts of this work were collected from three mangrove forests on the western Pacific island of Pohnpei in the Federated States of Micronesia, as well as tropical forests in Hawai’i, USA. Mangrove forests underscore the economy of this region, where more than half of the annual household income is derived from these forests. However, these mangrove forests are endangered by sea level rise, which necessitates an evaluation of the resilience of mangrove forests to climate change in order to better protect and manage these ecosystems. This includes the preservation of positive sediment accretion rates, and stimulating the process of root growth, sedimentation, and peat development, all of which are influenced by the forest floor elevation, relative to sea level. Currently, accretion rates are measured using surface elevation tables (SETs), which are posts permanently placed in mangrove sediments. The forest floor is measured annually with respect to the height of the SETs to evaluate changes in elevation (Cahoon et al. 2002). In this work, we evaluated the ability of the CBL system for measuring such elevation changes, to address objective #1. Digital Elevation Models (DEMs) were produced for plots, based on the point cloud resulted from co-registering eight scans, spaced 45 degree, per plot. DEMs are refined and produced using Cloth Simulation Filtering (CSF) and kriging interpolation. CSF was used because it minimizes the user input parameters, and kriging was chosen for this study due its consideration of the overall spatial arrangement of the points using semivariogram analysis, which results in a more robust model. The average consistency of the TLS-derived elevation change was 72%, with and RMSE value of 1.36 mm. However, what truly makes the TLS method more tenable, is the lower standard error (SE) values when compared to manual methods (10-70x lower). In order to achieve our second objective, we assessed structural characteristics of the above-mentioned mangrove forest and also for tropical forests in Hawaii, collected with the same CBL scanner. The same eight scans per plot (20 plots) were co-registered using pairwise registration and the Iterative Closest Point (ICP). We then removed the higher canopy using a normal change rate assessment algorithm. We used a combination of geometric classification techniques, based on the angular orientation of the planes fitted to points (facets), and machine learning 3D segmentation algorithms to detect tree stems and above-ground roots. Mangrove forests are complex forest environments, containing above-ground root mass, which can create confusion for both ground detection and structural assessment algorithms. As a result, we needed to train a supporting classifier on the roots to detect which root lidar returns were classified as stems. The accuracy and precision values for this classifier were assessed via manual investigation of the classification results in all 20 plots. The accuracy and precision for stem classification were found to be 82% and 77%, respectively. The same values for root detection were 76% and 68%, respectively. We simulated the stems using alpha shapes in order to assess their volume in the final step. The consistency of the volume evaluation was found to be 85%. This was obtained by comparing the mean stem volume (m3/ha) from field data and the TLS data in each plot. The reported accuracy is the average value for all 20 plots. Additionally, we compared the diameter-at-breast-height (DBH), recorded in the field, with the TLS-derived DBH to obtain a direct measure of the precision of our stem models. DBH evaluation resulted in an accuracy of 74% and RMSE equaled 7.52 cm. This approach can be used for automatic stem detection and structural assessment in a complex forest environment, and could contribute to biomass assessment in these rapidly changing environments. These stem and root structural assessment efforts were complemented by efforts to estimate canopy-level structural attributes of the tropical Hawai’i forest environment; we specifically estimated the leaf area index (LAI), by implementing a density-based approach. 242 scans were collected using the portable low-cost TLS (CBL), in a Hawaii Volcano National Park (HAVO) flux tower site. LAI was measured for all the plots in the site, using an AccuPAR LP-80 Instrument. The first step in this work involved detection of the higher canopy, using normal change rate assessment. After segmenting the higher canopy from the lidar point clouds, we needed to measure Leaf Area Density (LAD), using a voxel-based approach. We divided the canopy point cloud into five layers in the Z direction, after which each of these five layers were divided into voxels in the X direction. The sizes of these voxels were constrained based on interquartile analysis and the number of points in each voxel. We hypothesized that the power returned to the lidar system from woody materials, like branches, exceeds that from leaves, due to the liquid water absorption of the leaves and higher reflectivity for woody material at the 905 nm lidar wavelength. We evaluated leafy and woody materials using images from projected point clouds and determined the density of these regions to support our hypothesis. The density of points in a 3D grid size of 0.1 m, which was determined by investigating the size of the branches in the lower portion of the higher canopy, was calculated in each of the voxels. Note that “density” in this work is defined as the total number of points per grid cell, divided by the volume of that cell. Subsequently, we fitted a kernel density estimator to these values. The threshold was set based on half of the area under the curve in each of the distributions. The grid cells with a density below the threshold were labeled as leaves, while those cells with a density above the threshold were set as non-leaves. We then modeled the LAI using the point densities derived from TLS point clouds, achieving a R2 value of 0.88. We also estimated the LAI directly from lidar data by using the point densities and calculating leaf area density (LAD), which is defined as the total one-sided leaf area per unit volume. LAI can be obtained as the sum of the LAD values in all the voxels. The accuracy of LAI estimation was found to be 90%. Since the LAI values cannot be considered spatially independent throughout all the plots in this site, we performed a semivariogram analysis on the field-measured LAI data. This analysis showed that the LAI values can be assumed to be independent in plots that are at least 30 m apart. As a result, we divided the data into six subsets, where each of the plots were 30 meter spaced for each subset. LAI model R2 values for these subsets ranged between 0.84 - 0.96. The results bode well for using this method for automatic estimation of LAI values in complex forest environments, using a low-cost, low point density, rapid-scan TLS

Reconstructing plant architecture from 3D laser scanner data

En infographie, les modèles virtuels de plantes sont de plus en plus réalistes visuellement. Cependant, dans le contexte de la biologie et l'agronomie, l'acquisition de modèles précis de plantes réelles reste un problème majeur pour la construction de modèles quantitatifs du développement des plantes. Récemment, des scanners laser 3D permettent d'acquérir des images 3D avec pour chaque pixel une profondeur correspondant à la distance entre le scanner et la surface de l'objet visé. Cependant, une plante est généralement un ensemble important de petites surfaces sur lesquelles les méthodes classiques de reconstruction échouent. Dans cette thèse, nous présentons une méthode pour reconstruire des modèles virtuels de plantes à partir de scans laser. Mesurer des plantes avec un scanner laser produit des données avec différents niveaux de précision. Les scans sont généralement denses sur la surface des branches principales mais recouvrent avec peu de points les branches fines. Le cur de notre méthode est de créer itérativement un squelette de la structure de la plante en fonction de la densité locale de points. Pour cela, une méthode localement adaptative a été développée qui combine une phase de contraction et un algorithme de suivi de points. Nous présentons également une procédure d'évaluation quantitative pour comparer nos reconstructions avec des structures reconstruites par des experts de plantes réelles. Pour cela, nous explorons d'abord l'utilisation d'une distance d'édition entre arborescence. Finalement, nous formalisons la comparaison sous forme d'un problème d'assignation pour trouver le meilleur appariement entre deux structures et quantifier leurs différences. (Résumé d'auteur

Tree Structure Retrieval for Apple Trees from 3D Pointcloud

3D reconstruction is a challenging problem and has been an important research topic in the areas of remote sensing and computer vision for many years. Existing 3D reconstruction approaches are not suitable for orchard applications due to complicated tree structures. Current tree reconstruction has included models specific to trees of a certain density, but the impact of varying Leaf Area Index(LAI) on model performance has not been studied. To better manage an apple orchard, this thesis proposes methods for evaluating an apple canopy density mapping system as an input for a variable-rate sprayer for both trellis-structured (2D) and standalone (3D) apple orchards using a 2D LiDAR (Light Detection and Ranging). The canopy density mapping system has been validated for robustness and repeatability with multiple scans. The consistency of the whole row during multiple passes has a correlation R^2 = 0.97. The proposed system will help the decision-making in a variable-rate sprayer. To further study the individual tree structure, this thesis proposes a novel and fast approach to reconstruct and analyse 3D trees over a range of Leaf Area Index (LAI) values from LiDAR for morphology analysis for height, branch length and angles of real and simulated apple trees. After using Principal Component Analysis (PCA) to extract the trunk points, an improved Mean Shift algorithm is introduced as Adapted Mean Shift (AMS) to classify different branch clusters and extract the branch nodes. A full evaluation workflow of tree parameters including trunk and branches is introduced for morphology analysis to investigate the accuracy of the approach over different LAI values. Tree height, branch length, and branch angles were analysed and compared to the ground truth for trees with a range of LAI values. When the LAI is smaller than 0.1, the accuracy for height and length is greater than 90\% and the accuracy for the angles is around 80\%. When the LAI is greater than 0.1, the branch accuracy reduces to 40\%. This analysis of tree reconstruction performance concerning LAI values, as well as the combination of efficient and accurate structure reconstruction, opens the possibility of improving orchard management and botanical studies on a large scale. To improve the accuracy of traditional tree structure analysis, a deep learning approach is introduced to pre-process and classify unbalanced, in-homogeneous, and noisy point cloud data. TreeNet is inspired by 3D U-Net, adding classes and median filters to segment trunk, branch, and leave parts. TreeNet outperformed 3D U-Net and SVM in the case of Kappa, Matthews Correlation Coefficient(MCC), and F1-score value in segmentation. The TreeNet-AMS combined method also showed improvement in tree structure analysis than the traditional AMS method mentioned above. Following on from this research, efficient tree structure analysis on tree height, trunk length, branch position, and branch length could be conducted. Knowing the tree morphology is proved to be closely relevant to thinning, spraying and yield, the proposed work will then largely benefit the relevant studies in agriculture and forestry

Online Semantic Labeling of Deformable Tissues for Medical Applications

University of Minnesota Ph.D. dissertation. May 2017. Major: Mechanical Engineering. Advisor: Timothy Kowalewski. 1 computer file (PDF); ix, 133 pages.Surgery remains dangerous, and accurate knowledge of what is presented to the surgeon can be of great importance. One technique to automate this problem is non-rigid tracking of time-of-flight camera scans. This requires accurate sensors and prior information as well as an accurate non-rigid tracking algorithm. This thesis presents an evaluation of four algorithms for tracking and semantic labeling of deformable tissues for medical applications, as well as additional studies on a stretchable flexible smart skin and dynamic 3D bioprinting. The algorithms were developed and tested for this study, and were evaluated in terms of speed and accuracy. The algorithms tested were affine iterative closest point, nested iterative closest point, affine fast point feature histograms, and nested fast point feature histograms. The algorithms were tested against simulated data as well as direct scans. The nested iterative closest point algorithm provided the best balance of speed and accuracy while providing semantic labeling in both simulation as well as using directly scanned data. This shows that fast point feature histograms are not suitable for nonrigid tracking of geometric feature poor human tissues. Secondary experiments were also performed to show that the graphics processing unit provides enough speed to perform iterative closest point algorithms in real-time and that time of flight depth sensing works through an endoscope. Additional research was conducted on related topics, leading to the development of a novel stretchable flexible smart skin sensor and an active 3D bioprinting system for moving human anatomy