Position estimation using a stereo camera as part of the perception system in a Formula Student car

This thesis presents a part of the implementation of the perception system in an autonomous Formula Student vehicle. More precisely, it develops two different pipelines to process the data from the two main sensors of the vehicle: a LiDAR and a stereo camera. The first, a stereo camera system which is based on two monocular cameras, provides traffic cone position estimations based on the detections made by a convolutional neural network. These positions are obtained by using a self-designed stereo processing algorithm, based on 2D-3D position estimates and keypoint extraction and matching. The second is a sensor fusion system that first registers both sensors based on an extrinsic calibration system that has been implemented. Then, it exploits the neural network detection from the stereo system to project the LiDAR point cloud onto the image, obtaining a balance between accurate detection and position estimation. These two systems are evaluated, compared and integrated into "Xaloc". The Formula Student vehicle developed by the Driverless UPC team.Esta tesis presenta una parte de la implementación del sistema de percepción en un vehículo autónomo de Formula Student. Concretamente, se desarrollan dos sistemas diferentes para el procesado de datos de los dos sensores principales del vehículo: un LiDAR y una cámara estéreo. El sistema de cámara estéreo se basa en dos cámaras monoculares y proporciona estimaciones de la posición de los conos de tráfico que delimitan la pista en base a las detecciones realizadas por una red neuronal convolucional. Estas posiciones se obtienen mediante el uso de un algoritmo de procesamiento estéreo de diseño propio, basado en estimaciones de posición 2D-3D y en extracción y correspondencia de "keypoints". El segundo es un sistema de fusión de sensores que primero registra ambos sensores basándose en un sistema de calibración extrínseco que se ha implementado. Luego, usa la detección hecha con la red neuronal del sistema estéreo para proyectar la nube de puntos LiDAR en la imagen, obteniendo un lo mejor de cada sensor: una detección robusta y una estimación de posición muy precisa. Estos dos sistemas se evalúan, comparan e integran en "Xaloc" el vehículo sin conductor del equipo de Formula Student Driverless UPC.Aquesta tesi presenta una part de la implementació del sistema de percepció en un vehicle autònom de Formula Student. En concret, es desenvolupen dos sistemes diferents per processar les dades dels dos principals sensors del vehicle: un LiDAR i una càmera estèreo. El sistema de càmera estèreo es basa en dues càmeres monoculars, i proporciona estimacions de les posicions dels cons de trànsit que delimiten la pista basades en les deteccions fetes amb una xarxa neuronal convolucional. Aquestes posicions s'obtenen mitjançant un algoritme de processament d'estèreo propi, basat en estimacions de posició 2D-3D i en extracció i correspondència de keypoints. El segon és un sistema de fusió de sensors que registra els dos sensors en base a un sistema de calibratge extrínsec que s'ha implementat. A continuació, fa servir les deteccions de la xarxa neuronal del sistema estèreo per projectar el núvol de punts LiDAR a la imatge, obtenint un equilibri entre una bona detecció en imatge i la precisió del núvol de punts LiDAR. Aquests dos sistemes són avaluats, comparats i integrats al "Xaloc" el vehicle sense conductor de l'equip de Formula Student Driverless UPC

Design of a perception system for the Formula Student Driverless competition: from vehicle sensorization to SLAM

openFormula Student Driverless is an international racing competition held among universities, where the vehicles must complete a set of trials without any human intervention. Together with RaceUP, the Formula Student team of the University of Padova, this thesis represents the beginning of the project to build an autonomous prototype to compete in the Driverless Cup in the 2024 season. Three important aspects of an autonomous system design will be tackled: vehicle sensorization, perception, and simultaneous localization and mapping (SLAM), with the main focus on the development of the last one. The proposed approach for the back-end is based on the optimization of a factor graph, holding information about car poses and landmarks positions, by exploiting spatial and kinematic constraints between its vertices. The full back-end pipeline has been tested thoroughly, step by step, allowing to obtain satisfactory results on the different virtual tracks used for testing. Using both modern and classical techniques, we can process information produced by the stereo camera and the LIDAR, to be able to localize the colored cones delimiting the track. The estimation of cones positions serves then as input for other important modules of the car, such as the control part and the SLAM pipeline. Finally, a complete dataset has been acquired by properly sensorizing RaceUP's last year's car: having real data represents a helpful resource to make experiments and validate the system, even without the availability of the actual vehicle prototype.Formula Student Driverless is an international racing competition held among universities, where the vehicles must complete a set of trials without any human intervention. Together with RaceUP, the Formula Student team of the University of Padova, this thesis represents the beginning of the project to build an autonomous prototype to compete in the Driverless Cup in the 2024 season. Three important aspects of an autonomous system design will be tackled: vehicle sensorization, perception, and simultaneous localization and mapping (SLAM), with the main focus on the development of the last one. The proposed approach for the back-end is based on the optimization of a factor graph, holding information about car poses and landmarks positions, by exploiting spatial and kinematic constraints between its vertices. The full back-end pipeline has been tested thoroughly, step by step, allowing to obtain satisfactory results on the different virtual tracks used for testing. Using both modern and classical techniques, we can process information produced by the stereo camera and the LIDAR, to be able to localize the colored cones delimiting the track. The estimation of cones positions serves then as input for other important modules of the car, such as the control part and the SLAM pipeline. Finally, a complete dataset has been acquired by properly sensorizing RaceUP's last year's car: having real data represents a helpful resource to make experiments and validate the system, even without the availability of the actual vehicle prototype

Enhancement of a Formula Student car perception system using a global 3D map

La implementació d'un precís sistema de localització i mapeig al nostre cotxe de Formula Student sense conductor ha revolucionat el sistema de percepció del cotxe d'aquesta temporada. Ara, cal un nou mètode que aprofiti aquest mapa 3D millorat. En aquest mapa s'obtenen les posicions dels cons, que es classifiquen amb la informació extreta de les imatges de la càmera per calcular els límits de la pista. Aquesta tesi proposa un nou sistema per classificar i fer un seguiment dels cons (CCAT) i un altre sistema (Urimits) per estendre els límits de pista parcials dependents del color fent servir cons no classificats. Tots dos sistemes han aconseguit una millora respecte als de la temporada passada pel que fa a abast i precisió. Ara és possible detectar els límits de la pista tot tancant la volta abans que el cotxe físicament completi la tornada.The implementation of an accurate localization and mapping system in our driverless Formula Student car has revolutionized this season's car perception pipeline. Now, a new system that takes advantage of this improved 3D map is needed. Cone positions are obtained in this map, and these are classified with the information extracted from camera images in order to compute the track limits. This thesis proposes a new system to classify and keep track of cones (CCAT) and another system (Urimits) to extend partial color dependant track limits using unclassified cones. Both systems have achieved an enhancement over last season's in range and accuracy. Now the possibility of detecting the whole track limits before the car completes the lap is possible

Imitation Accelerated Q-learning on a Simulated Formula Student Driverless Racecar

Master's thesis in Information- and communication technology (IKT590)In the international Formula Student competition, only a handful compete in the driverless category. Most of them using expensive hardware such as LIDAR’s. By leveraging reinforcement learning, a cheaper camera based system can be created .In order to train this system a simulator based on a fork of Microsoft’s AirSim by Formula Technion was used. A virtual replica of a Formula Student car designed for 2020 by Align Racing UiA, functioned as the test vehicle. In order to decrease the required training time, a pre-trained imitation learning network was used. This was implemented into a Deep Q-Learning network in four different methods. The most successful method was able to accelerate the learning process by 36%

Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car

With the rising popularity of autonomous navigation research, Formula Student (FS) events are introducing a Driverless Vehicle (DV) category to their event list. This paper presents the initial investigation into utilising Deep Reinforcement Learning (RL) for end-to-end control of an autonomous FS race car for these competitions. We train two state-of-the-art RL algorithms in simulation on tracks analogous to the full-scale design on a Turtlebot2 platform. The results demonstrate that our approach can successfully learn to race in simulation and then transfer to a real-world racetrack on the physical platform. Finally, we provide insights into the limitations of the presented approach and guidance into the future directions for applying RL toward full-scale autonomous FS racing.Comment: Accepted at the Australasian Conference on Robotics and Automation (ACRA 2022