Living in a Material World: Learning Material Properties from Full-Waveform Flash Lidar Data for Semantic Segmentation

Advances in lidar technology have made the collection of 3D point clouds fast and easy. While most lidar sensors return per-point intensity (or reflectance) values along with range measurements, flash lidar sensors are able to provide information about the shape of the return pulse. The shape of the return waveform is affected by many factors, including the distance that the light pulse travels and the angle of incidence with a surface. Importantly, the shape of the return waveform also depends on the material properties of the reflecting surface. In this paper, we investigate whether the material type or class can be determined from the full-waveform response. First, as a proof of concept, we demonstrate that the extra information about material class, if known accurately, can improve performance on scene understanding tasks such as semantic segmentation. Next, we learn two different full-waveform material classifiers: a random forest classifier and a temporal convolutional neural network (TCN) classifier. We find that, in some cases, material types can be distinguished, and that the TCN generally performs better across a wider range of materials. However, factors such as angle of incidence, material colour, and material similarity may hinder overall performance.Comment: In Proceedings of the Conference on Robots and Vision (CRV'23), Montreal, Canada, Jun. 6-8, 202

Synchronisation et calibrage entre un Lidar 3D et une centrale inertielle pour la localisation précise d'un véhicule autonome

International audienceLaser remote sensing (Lidar) is a technology increasingly used especially in the perception layers of autonomous vehicles. As the vehicle moves during measurement, Lidar data must be referenced in a fixed frame which is usually done thanks to an inertial measurement unit (IMU). However, these sensors are not designed to work together natively thus it is necessary to synchronize and calibrate them carefully. This article presents a method for characterizing timing offsets between a 3D Lidar and an inertial measurement unit. It also explains how to implement the usual methods for pose estimation between an IMU and a Lidar when using such sensors in real conditions.La télédétection par laser (Lidar) est une technologie de plus en plus utilisée en particulier dans les fonctions de perception et localisation nécessaires à la conduite autonome. L'acquisition des données Lidar doit être couplée à la mesure du mouvement du véhicule par une centrale inertielle. Ces capteurs n'étant pas conçus pour fonctionner ensemble nativement, il est nécessaire de maitriser leur synchronisation et leur calibrage géométrique. Cet article présente une méthode pour caractériser les décalages temporels entre un Lidar 3D et une centrale inertielle. Il explique aussi comment mettre en œuvre les méthodes de la littérature pour le calcul de la pose entre centrale inertielle et Lidar sur un véhicule utilisé en conditions réelles

Contribution à la localisation robuste embarquée pour la navigation autonome

There has been a great and quick development in autonomous mobile robotics over the past few years. This growth mainly concerns autonomous vehicles and service robotics. Whatever the field of application, the localization task plays a key role in the intelligence of the mobile robots. As a result, this thesis is focused on exploring new challenges in embedding this aspect of artificial intelligence over two topics.First, we tackled the ARGOS Challenge. We have proposed a 6 Degrees of Freedom localization method based on multi-layer LiDAR data. Results show that the localization is accurate enough to perform autonomous control of the robot. Targeted applications include Oil and Gas platform patrolling. Such environments are more challenging than regular environments found in the literature. The proposed method takes into account concerns regarding CPU and memory consumption as well as embeddability. The likelihood field concept was extended to 3D. The method was benchmarked in a lab, an industrial site with single-layer and multi-layer LiDARs and 3 different robots. Robots are being located in their environment with an average error of 2.5cm using 16% of a CPU core.Secondly, we focused on topological localization. In fact, the approach used in the ARGOS Challenge requires a small area as initial step. Our second approach enables to find the location of a road vehicle over several square kilometers. Our method is based on sensors widely available on any modern cars (ABS and ESP). The map used is a topological representation of OpenStreet map data. Based on this information, we achieve an averageerror of 4m. This figure is close to GPS-based accuracy.Finally, our works are based on a specific experimental approach. In fact, developing new localization algorithms requires fine tunings. Moreover, tests, reliability and certifications of autonomous systems are still challenging for either the scientific community or the industry actors. The proposed methodology tends to propose a solution by mixing simulation, small-scale testing and full-scale testing.Nous constatons ces dernières années un essor important de la robotique mobile autonome avec deux grands domaines d’application : la robotique des services et le véhicule autonome. Quel que soit le domaine visé, la fonction de localisation est déterminante pour l’autonomie de ces futurs mobiles. A ce titre, nous avons consacré ces travaux de thèse à l’approfondissement des problématiques de localisation à travers deux sujets proposant des méthodes embarquées.Dans un premier temps, avec le challenge international de robotique Argos : nous nous sommes attelés à une localisation 6 degrés de liberté basée LiDAR multi-nappes, avec une précision suffisante pour assurer le contrôle du robot. Les applications envisagées étant la surveillance des installations pétrochimiques, l’environnement rencontré est plus complexe que ceux couramment décrits dans la littérature. Nous avons mis particulièrement l’accent sur le côté embarquable, soit la consommation de ressources aussi bien en termes de mémoire que de CPU de l’algorithme. Pour cela, nous avons étendule concept de likelihood field en 3D. L’algorithme a été évalué en laboratoire puis sur un site industriel, avec deux types de LiDAR et sur 3 robots. Nous atteignons une précision d’environ 2.5cm en utilisant 16% d’un core d’un processeur actuel.Cette méthode de localisation n’autorise pas une initialisation sur une large zone. Dans un second temps, nous avons donc proposé une localisation topologique de véhicules routiers permettant une convergence sur plusieurs kilomètres carrés. Nous n’utilisons que des capteurs présents dans tous les véhicules (ABS et ESP) et la carte est représentée par un graphe contenant des données Open Street Map. A partir de ces informations, nous obtenons une précision inférieure à 4 m, c’est-à-dire semblable à celle d’un GPS standard.Nos travaux portent également sur une méthodologie d’expérimentation. En effet, la mise au point d’algorithmes de localisation nécessite de nombreux ajustements. De plus, le test, la fiabilité puis la qualification de ces systèmes autonomes demeurent de véritables enjeux aussi bien pour la communauté scientifique qu’industrielle. Cette méthodologie contribue à répondre à ces challenges en fusionnant le développement des tests réalisés en simulation, à échelle réduite et pleine échelle

Millimeter Wave FMCW RADARs for Perception, Recognition and Localization in Automotive Applications: A Survey

MmWave (millimeter wave) Frequency Modulated Continuous Waves (FMCW) RADARs are sensors based on frequency-modulated electromagnetic which see their environment in 3D at a long-range. The recent introduction of millimeter-wave RADARs with frequencies from 60 GHz to 300 GHz has broadened their potential applications thanks to their improved accuracy in angle, range, and velocity. MmWave FMCW RADARs have better resolution and accuracy than narrowband and ultra-wideband (UWB) RADARs. In comparison with cameras and LiDARs, they possess several strong advantages such as long-range perception, robustness to lightning, and weather conditions while being cheaper. However, their noisy and lower-density outputs even compared to other technologies of RADARs, and their ability to measure the targets' velocities require specific algorithms tailored for them. Working principles of mmWave FMCW RADARs are presented as well as the separate ways to represent data and their applications. This paper describes algorithms and applications adapted or developed for these sensors in automotive applications. Finally, current challenges and directions for future works are presented

Vehicle Positioning in Road Networks without GPS

International audienc

Particle filter meets hybrid octrees: an octree-based ground vehicle localization approach without learning

International audienc

Demonstration of the Salvo Enhanced No Escape Zone Concept using ground mobile robots

International audienc

IMU/LIDAR based positioning of a gangway for maintenance operations on wind farms

International audienc