RANSAC for Robotic Applications: A Survey

Random Sample Consensus, most commonly abbreviated as RANSAC, is a robust estimation method for the parameters of a model contaminated by a sizable percentage of outliers. In its simplest form, the process starts with a sampling of the minimum data needed to perform an estimation, followed by an evaluation of its adequacy, and further repetitions of this process until some stopping criterion is met. Multiple variants have been proposed in which this workflow is modified, typically tweaking one or several of these steps for improvements in computing time or the quality of the estimation of the parameters. RANSAC is widely applied in the field of robotics, for example, for finding geometric shapes (planes, cylinders, spheres, etc.) in cloud points or for estimating the best transformation between different camera views. In this paper, we present a review of the current state of the art of RANSAC family methods with a special interest in applications in robotics.This work has been partially funded by the Basque Government, Spain, under Research Teams Grant number IT1427-22 and under ELKARTEK LANVERSO Grant number KK-2022/00065; the Spanish Ministry of Science (MCIU), the State Research Agency (AEI), the European Regional Development Fund (FEDER), under Grant number PID2021-122402OB-C21 (MCIU/AEI/FEDER, UE); and the Spanish Ministry of Science, Innovation and Universities, under Grant FPU18/04737

Point Cloud-Based Target-Oriented 3D Path Planning for UAVs

This paper explores 3D path planning for unmanned aerial vehicles (UAVs) in 3D point cloud environments. Derivative maps such as dense point clouds, mesh maps, octomaps, etc. are frequently used for path planning purposes. A target-oriented 3D path planning algorithm, directly using point clouds to compute optimized trajectories for an UAV, is presented in this article. This approach searches for obstacle-free, low computational cost, smooth, and dynamically feasible paths by analyzing a point cloud of the target environment, using a modified connect RRT-based path planning algorithm, with a k-d tree based obstacle avoidance strategy and three-step optimization. This presented approach bypasses the common 3D map discretization, directly leveraging point cloud data. Following trajectory generation, the algorithm creates way-point based, closed loop quadrotor controls for pitch, roll, and yaw attitude angle as well as dynamics commands for the UAV. Simulations of UAV 3D path planning based on different target points in the point cloud map are presented, showing the effectiveness and feasibility of this approach

Generation of Horizontally Curved Driving Lines for Autonomous Vehicles Using Mobile Laser Scanning Data

The development of autonomous vehicle desiderates tremendous advances in three-dimensional (3D) high-definition roadmaps. These roadmaps are capable of providing 3D positioning information with 10-to-20 cm accuracy. With the assistance of 3D high-definition roadmaps, the intractable autonomous driving problem is transformed into a solvable localization issue. The Mobile Laser Scanning (MLS) systems can collect accurate, high-density 3D point clouds in road environments for generating 3D high-definition roadmaps. However, few studies have been concentrated on the driving line generation from 3D MLS point clouds for highly autonomous driving, particularly for accident-prone horizontal curves with the problems of ambiguous traffic situations and unclear visual clues. This thesis attempts to develop an effective method for semi-automated generation of horizontally curved driving lines using MLS data. The framework of research methodology proposed in this thesis consists of three steps, including road surface extraction, road marking extraction, and driving line generation. Firstly, the points covering road surface are extracted using curb-based road surface extraction algorithms depending on both the elevation and slope differences. Then, road markings are identified and extracted according to a sequence of algorithms consisting of geo-referenced intensity image generation, multi-threshold road marking extraction, and statistical outlier removal. Finally, the conditional Euclidean clustering algorithm is employed followed by the nonlinear least-squares curve-fitting algorithm for generating horizontally curved driving lines. A total of six test datasets obtained in Xiamen, China by a RIEGL VMX-450 system were used to evaluate the performance and efficiency of the proposed methodology. The experimental results demonstrate that the proposed road marking extraction algorithms can achieve 90.89% in recall, 93.04% in precision and 91.95% in F1-score, respectively. Moreover, the unmanned aerial vehicle (UAV) imagery with 4 cm was used for validation of the proposed driving line generation algorithms. The validation results demonstrate that the horizontally curved driving lines can be effectively generated within 15 cm-level localization accuracy using MLS point clouds. Finally, a comparative study was conducted both visually and quantitatively to indicate the accuracy and reliability of the generated driving lines

Path Planning Framework for Unmanned Ground Vehicles on Uneven Terrain

In this thesis, I address the problem of long-range path planning on uneven terrain for non-holonomic wheeled mobile robots (WMR). Uneven terrain path planning is essential for search-and-rescue, surveillance, military, humanitarian, agricultural, constructing missions, etc. These missions necessitate the generation of a feasible sequence of waypoints, or reference states, to navigate a WMR from the initial location to the final target location through the uneven terrain. The feasibility of navigating through a given path over uneven terrain can be undermined by various terrain features. Examples of such features are loose soil, vegetation, boulders, steeply sloped terrain, or a combination of all of these elements. I propose a three-stage framework to solve the problem of rapid long-range path planning. In the first stage, RRT-Connect provides a rapid discovery of the feasible solution. Afterward, Informed RRT* improves the feasible solution. Finally, Shortcut heuristics improves the solution locally. To improve the computational speed of path planning algorithms, we developed an accelerated version of the traversability estimation on point clouds based on Principal Component Analysis. The benchmarks demonstrate the efficacy of the path planning approach

Development of a novel data acquisition and processing methodology applied to the boresight alignment of marine mobile LiDAR systems

Le système LiDAR mobile (SLM) est une technologie d'acquisition de données de pointe qui permet de cartographier les scènes du monde réel en nuages de points 3D. Les applications du SLM sont très vastes, de la foresterie à la modélisation 3D des villes, en passant par l'évaluation de l'inventaire routier et la cartographie des infrastructures portuaires. Le SLM peut également être monté sur diverses plateformes, telles que des plateformes aériennes, terrestres, marines, etc. Indépendamment de l'application et de la plateforme, pour s'assurer que le SLM atteigne sa performance optimale et sa meilleure précision, il est essentiel de traiter correctement les erreurs systématiques du système, spécialement l'erreur des angles de visée à laquelle on s'intéresse particulièrement dans cette thèse. L'erreur des angles de visée est définie comme le désalignement rotationnel des deux parties principales du SLM, le système de positionnement et d'orientation et le scanneur LiDAR, introduit par trois angles de visée. En fait, de petites variations angulaires dans ces paramètres peuvent causer des problèmes importants d'incertitude géométrique dans le nuage de points final et il est vital d'employer une méthode d'alignement pour faire face à la problématique de l'erreur des angles de visée de ces systèmes. La plupart des méthodes existantes d'alignement des angles de visée qui ont été principalement développées pour les SLM aériens et terrestres, tirent profit d'éléments in-situ spécifiques et présents sur les sites de levés et adéquats pour ces méthodes. Par exemple, les éléments linéaires et planaires extraits des toits et des façades des maisons. Cependant, dans les environnements sans présence de ces éléments saillants comme la forêt, les zones rurales, les ports, où l'accès aux éléments appropriées pour l'alignement des angles de visée est presque impossible, les méthodes existantes fonctionnent mal, voire même pas du tout. Par conséquent, cette recherche porte sur l'alignement des angles de visée d'un SLM dans un environnement complexe. Nous souhaitons donc introduire une procédure d'acquisition et traitement pour une préparation adéquate des données, qui servira à la méthode d'alignement des angles de visée du SLM. Tout d'abord, nous explorons les différentes possibilités des éléments utilisés dans les méthodes existantes qui peuvent aider à l'identification de l'élément offrant le meilleur potentiel pour l'estimation des angles de visée d'un SLM. Ensuite, nous analysons, parmi un grand nombre de possibles configurations d'éléments (cibles) et patrons de lignes de balayage, celle qui nous apparaît la meilleure. Cette analyse est réalisée dans un environnement de simulation dans le but de générer différentes configurations de cibles et de lignes de balayage pour l'estimation des erreurs des angles de visée afin d'isoler la meilleure configuration possible. Enfin, nous validons la configuration proposée dans un scénario réel, soit l'étude de cas du port de Montréal. Le résultat de la validation révèle que la configuration proposée pour l'acquisition et le traitement des données mène à une méthode rigoureuse d'alignement des angles de visée qui est en même temps précise, robuste et répétable. Pour évaluer les résultats obtenus, nous avons également mis en œuvre une méthode d'évaluation de la précision relative, qui démontre l'amélioration de la précision du nuage de points après l'application de la procédure d'alignement des angles de visée.A Mobile LiDAR system (MLS) is a state-of-the-art data acquisition technology that maps real-world scenes in the form of 3D point clouds. The MLS's list of applications is vast, from forestry to 3D city modeling and road inventory assessment to port infrastructure mapping. The MLS can also be mounted on various platforms, such as aerial, terrestrial, marine, and so on. Regardless of the application and the platform, to ensure that the MLS achieves its optimal performance and best accuracy, it is essential to adequately address the systematic errors of the system, especially the boresight error. The boresight error is the rotational misalignment offset of the two main parts of the MLS, the positioning and orientation system (POS) and the LiDAR scanner. Minor angular parameter variations can cause important geometric accuracy issues in the final point cloud. Therefore, it is vital to employ an alignment method to cope with the boresight error problem of such systems. Most of the existing boresight alignment methods, which have been mainly developed for aerial and terrestrial MLS, take advantage of the in-situ tie-features in the environment that are adequate for these methods. For example, tie-line and tie-plane are extracted from building roofs and facades. However, in low-feature environments like forests, rural areas, ports, and harbors, where access to suitable tie-features for boresight alignment is nearly impossible, the existing methods malfunction or do not function. Therefore, this research addresses the boresight alignment of a marine MLS in a low-feature maritime environment. Thus, we aim to introduce an acquisition procedure for suitable data preparation, which will serve as input for the boresight alignment method of a marine MLS. First, we explore various tie-features introduced in the existing ways that eventually assist in the identification of the suitable tie-feature for the boresight alignment of a marine MLS. Second, we study the best configuration for the data acquisition procedure, i.e., tie-feature(s) characteristics and the necessary scanning line pattern. This study is done in a simulation environment to achieve the best visibility of the boresight errors on the selected suitable tie-feature. Finally, we validate the proposed configuration in a real-world scenario, which is the port of Montreal case study. The validation result reveals that the proposed data acquisition and processing configuration results in an accurate, robust, and repeatable rigorous boresight alignment method. We have also implemented a relative accuracy assessment to evaluate the obtained results, demonstrating an accuracy improvement of the point cloud after the boresight alignment procedure

Using AI and Robotics for EV battery cable detection.: Development and implementation of end-to-end model-free 3D instance segmentation for industrial purposes

Master's thesis in Information- and communication technology (IKT590)This thesis describes a novel method for capturing point clouds and segmenting instances of cabling found on electric vehicle battery packs. The use of cutting-edge perception algorithm architectures, such as graph-based and voxel-based convolution, in industrial autonomous lithium-ion battery pack disassembly is being investigated. The thesis focuses on the challenge of getting a desirable representation of any battery pack using an ABB robot in conjunction with a high-end structured light camera, with "end-to-end" and "model-free" as design constraints. The thesis employs self-captured datasets comprised of several battery packs that have been captured and labeled. Following that, the datasets are used to create a perception system. This thesis recommends using HDR functionality in an industrial application to capture the full dynamic range of the battery packs. To adequately depict 3D features, a three-point-of-view capture sequence is deemed necessary. A general capture process for an entire battery pack is also presented, but a next-best-scan algorithm is likely required to ensure a "close to complete" representation. Graph-based deep-learning algorithms have been shown to be capable of being scaled up to50,000inputs while still exhibiting strong performance in terms of accuracy and processing time. The results show that an instance segmenting system can be implemented in less than two seconds. Using off-the-shelf hardware, demonstrate that a 3D perception system is industrially viable and competitive with a 2D perception system

Perception of Unstructured Environments for Autonomous Off-Road Vehicles

Autonome Fahrzeuge benötigen die Fähigkeit zur Perzeption als eine notwendige Voraussetzung für eine kontrollierbare und sichere Interaktion, um ihre Umgebung wahrzunehmen und zu verstehen. Perzeption für strukturierte Innen- und Außenumgebungen deckt wirtschaftlich lukrative Bereiche, wie den autonomen Personentransport oder die Industrierobotik ab, während die Perzeption unstrukturierter Umgebungen im Forschungsfeld der Umgebungswahrnehmung stark unterrepräsentiert ist. Die analysierten unstrukturierten Umgebungen stellen eine besondere Herausforderung dar, da die vorhandenen, natürlichen und gewachsenen Geometrien meist keine homogene Struktur aufweisen und ähnliche Texturen sowie schwer zu trennende Objekte dominieren. Dies erschwert die Erfassung dieser Umgebungen und deren Interpretation, sodass Perzeptionsmethoden speziell für diesen Anwendungsbereich konzipiert und optimiert werden müssen. In dieser Dissertation werden neuartige und optimierte Perzeptionsmethoden für unstrukturierte Umgebungen vorgeschlagen und in einer ganzheitlichen, dreistufigen Pipeline für autonome Geländefahrzeuge kombiniert: Low-Level-, Mid-Level- und High-Level-Perzeption. Die vorgeschlagenen klassischen Methoden und maschinellen Lernmethoden (ML) zur Perzeption bzw.~Wahrnehmung ergänzen sich gegenseitig. Darüber hinaus ermöglicht die Kombination von Perzeptions- und Validierungsmethoden für jede Ebene eine zuverlässige Wahrnehmung der möglicherweise unbekannten Umgebung, wobei lose und eng gekoppelte Validierungsmethoden kombiniert werden, um eine ausreichende, aber flexible Bewertung der vorgeschlagenen Perzeptionsmethoden zu gewährleisten. Alle Methoden wurden als einzelne Module innerhalb der in dieser Arbeit vorgeschlagenen Perzeptions- und Validierungspipeline entwickelt, und ihre flexible Kombination ermöglicht verschiedene Pipelinedesigns für eine Vielzahl von Geländefahrzeugen und Anwendungsfällen je nach Bedarf. Low-Level-Perzeption gewährleistet eine eng gekoppelte Konfidenzbewertung für rohe 2D- und 3D-Sensordaten, um Sensorausfälle zu erkennen und eine ausreichende Genauigkeit der Sensordaten zu gewährleisten. Darüber hinaus werden neuartige Kalibrierungs- und Registrierungsansätze für Multisensorsysteme in der Perzeption vorgestellt, welche lediglich die Struktur der Umgebung nutzen, um die erfassten Sensordaten zu registrieren: ein halbautomatischer Registrierungsansatz zur Registrierung mehrerer 3D~Light Detection and Ranging (LiDAR) Sensoren und ein vertrauensbasiertes Framework, welches verschiedene Registrierungsmethoden kombiniert und die Registrierung verschiedener Sensoren mit unterschiedlichen Messprinzipien ermöglicht. Dabei validiert die Kombination mehrerer Registrierungsmethoden die Registrierungsergebnisse in einer eng gekoppelten Weise. Mid-Level-Perzeption ermöglicht die 3D-Rekonstruktion unstrukturierter Umgebungen mit zwei Verfahren zur Schätzung der Disparität von Stereobildern: ein klassisches, korrelationsbasiertes Verfahren für Hyperspektralbilder, welches eine begrenzte Menge an Test- und Validierungsdaten erfordert, und ein zweites Verfahren, welches die Disparität aus Graustufenbildern mit neuronalen Faltungsnetzen (CNNs) schätzt. Neuartige Disparitätsfehlermetriken und eine Evaluierungs-Toolbox für die 3D-Rekonstruktion von Stereobildern ergänzen die vorgeschlagenen Methoden zur Disparitätsschätzung aus Stereobildern und ermöglichen deren lose gekoppelte Validierung. High-Level-Perzeption konzentriert sich auf die Interpretation von einzelnen 3D-Punktwolken zur Befahrbarkeitsanalyse, Objekterkennung und Hindernisvermeidung. Eine Domänentransferanalyse für State-of-the-art-Methoden zur semantischen 3D-Segmentierung liefert Empfehlungen für eine möglichst exakte Segmentierung in neuen Zieldomänen ohne eine Generierung neuer Trainingsdaten. Der vorgestellte Trainingsansatz für 3D-Segmentierungsverfahren mit CNNs kann die benötigte Menge an Trainingsdaten weiter reduzieren. Methoden zur Erklärbarkeit künstlicher Intelligenz vor und nach der Modellierung ermöglichen eine lose gekoppelte Validierung der vorgeschlagenen High-Level-Methoden mit Datensatzbewertung und modellunabhängigen Erklärungen für CNN-Vorhersagen. Altlastensanierung und Militärlogistik sind die beiden Hauptanwendungsfälle in unstrukturierten Umgebungen, welche in dieser Arbeit behandelt werden. Diese Anwendungsszenarien zeigen auch, wie die Lücke zwischen der Entwicklung einzelner Methoden und ihrer Integration in die Verarbeitungskette für autonome Geländefahrzeuge mit Lokalisierung, Kartierung, Planung und Steuerung geschlossen werden kann. Zusammenfassend lässt sich sagen, dass die vorgeschlagene Pipeline flexible Perzeptionslösungen für autonome Geländefahrzeuge bietet und die begleitende Validierung eine exakte und vertrauenswürdige Perzeption unstrukturierter Umgebungen gewährleistet

UAV-Enabled Surface and Subsurface Characterization for Post-Earthquake Geotechnical Reconnaissance

Major earthquakes continue to cause significant damage to infrastructure systems and the loss of life (e.g. 2016 Kaikoura, New Zealand; 2016 Muisne, Ecuador; 2015 Gorkha, Nepal). Following an earthquake, costly human-led reconnaissance studies are conducted to document structural or geotechnical damage and to collect perishable field data. Such efforts are faced with many daunting challenges including safety, resource limitations, and inaccessibility of sites. Unmanned Aerial Vehicles (UAV) represent a transformative tool for mitigating the effects of these challenges and generating spatially distributed and overall higher quality data compared to current manual approaches. UAVs enable multi-sensor data collection and offer a computational decision-making platform that could significantly influence post-earthquake reconnaissance approaches. As demonstrated in this research, UAVs can be used to document earthquake-affected geosystems by creating 3D geometric models of target sites, generate 2D and 3D imagery outputs to perform geomechanical assessments of exposed rock masses, and characterize subsurface field conditions using techniques such as in situ seismic surface wave testing. UAV-camera systems were used to collect images of geotechnical sites to model their 3D geometry using Structure-from-Motion (SfM). Key examples of lessons learned from applying UAV-based SfM to reconnaissance of earthquake-affected sites are presented. The results of 3D modeling and the input imagery were used to assess the mechanical properties of landslides and rock masses. An automatic and semi-automatic 2D fracture detection method was developed and integrated with a 3D, SfM, imaging framework. A UAV was then integrated with seismic surface wave testing to estimate the shear wave velocity of the subsurface materials, which is a critical input parameter in seismic response of geosystems. The UAV was outfitted with a payload release system to autonomously deliver an impulsive seismic source to the ground surface for multichannel analysis of surface waves (MASW) tests. The UAV was found to offer a mobile but higher-energy source than conventional seismic surface wave techniques and is the foundational component for developing the framework for fully-autonomous in situ shear wave velocity profiling.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145793/1/wwgreen_1.pd