Visual computing techniques for automated LIDAR annotation with application to intelligent transport systems

106 p.The concept of Intelligent Transport Systems (ITS) refers to the application of communication and information technologies to transport with the aim of making it more efficient, sustainable, and safer. Computer vision is increasingly being used for ITS applications, such as infrastructure management or advanced driver-assistance systems. The latest progress in computer vision, thanks to the Deep Learning techniques, and the race for autonomous vehicle, have created a growing requirement for annotated data in the automotive industry. The data to be annotated is composed by images captured by the cameras of the vehicles and LIDAR data in the form of point clouds. LIDAR sensors are used for tasks such as object detection and localization. The capacity of LIDAR sensors to identify objects at long distances and to provide estimations of their distance make them very appealing sensors for autonomous driving.This thesis presents a method to automate the annotation of lane markings with LIDAR data. The state of the art of lane markings detection based on LIDAR data is reviewed and a novel method is presented. The precision of the method is evaluated against manually annotated data. Its usefulness is also evaluated, measuring the reduction of the required time to annotate new data thanks to the automatically generated pre-annotations. Finally, the conclusions of this thesis and possible future research lines are presented

Visual computing techniques for automated LIDAR annotation with application to intelligent transport systems

106 p.The concept of Intelligent Transport Systems (ITS) refers to the application of communication and information technologies to transport with the aim of making it more efficient, sustainable, and safer. Computer vision is increasingly being used for ITS applications, such as infrastructure management or advanced driver-assistance systems. The latest progress in computer vision, thanks to the Deep Learning techniques, and the race for autonomous vehicle, have created a growing requirement for annotated data in the automotive industry. The data to be annotated is composed by images captured by the cameras of the vehicles and LIDAR data in the form of point clouds. LIDAR sensors are used for tasks such as object detection and localization. The capacity of LIDAR sensors to identify objects at long distances and to provide estimations of their distance make them very appealing sensors for autonomous driving.This thesis presents a method to automate the annotation of lane markings with LIDAR data. The state of the art of lane markings detection based on LIDAR data is reviewed and a novel method is presented. The precision of the method is evaluated against manually annotated data. Its usefulness is also evaluated, measuring the reduction of the required time to annotate new data thanks to the automatically generated pre-annotations. Finally, the conclusions of this thesis and possible future research lines are presented

Vision-based Detection, Tracking and Classification of Vehicles using Stable Features with Automatic Camera Calibration

A method is presented for segmenting and tracking vehicles on highways using a camera that is relatively low to the ground. At such low angles, 3D perspective effects cause significant appearance changes over time, as well as severe occlusions by vehicles in neighboring lanes. Traditional approaches to occlusion reasoning assume that the vehicles initially appear well-separated in the image, but in our sequences it is not uncommon for vehicles to enter the scene partially occluded and remain so throughout. By utilizing a 3D perspective mapping from the scene to the image, along with a plumb line projection, a subset of features is identified whose 3D coordinates can be accurately estimated. These features are then grouped to yield the number and locations of the vehicles, and standard feature tracking is used to maintain the locations of the vehicles over time. Additional features are then assigned to these groups and used to classify vehicles as cars or trucks. The technique uses a single grayscale camera beside the road, processes image frames incrementally, works in real time, and produces vehicle counts with over 90% accuracy on challenging sequences. Adverse weather conditions are handled by augmenting feature tracking with a boosted cascade vehicle detector (BCVD). To overcome the need of manual camera calibration, an algorithm is presented which uses BCVD to calibrate the camera automatically without relying on any scene-specific image features such as road lane markings

Chartopolis - A Self Driving Car Test Bed

abstract: This thesis presents an autonomous vehicle test bed which can be used to conduct studies on the interaction between human-driven vehicles and autonomous vehicles on the road. The test bed will make use of a fleet of robots which is a microcosm of an autonomous vehicle performing all the vital tasks like lane following, traffic signal obeying and collision avoidance with other vehicles on the road. The robots use real-time image processing and closed-loop control techniques to achieve automation. The testbed also features a manual control mode where a user can choose to control the car with a joystick by viewing a video relayed to the control station. Stochastic rogue vehicle processes will be introduced into the system which will emulate random behaviors in an autonomous vehicle. The test bed was experimented to perform a comparative study of driving capabilities of the miniature self-driving car and a human driver.Dissertation/ThesisMasters Thesis Electrical Engineering 201

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas

A perspective on emerging automotive safety applications, derived from lessons learned through participation in the DARPA Grand Challenges

This paper reports on various aspects of the Intelligent Vehicle Systems (IVS) team's involvement in the recent 2007 DARPA Urban Challenge, wherein our platform, the autonomous “XAV-250,'' competed as one of the 11 finalists qualifying for the event. We provide a candid discussion of the hardware and software design process that led to our team's entry, along with lessons learned at this event and derived from participation in the two previous Grand Challenges. In addition, we give an overview of our vision-, radar-, and LIDAR-based perceptual sensing suite, its fusion with a military-grade inertial navigation package, and the map-based control and planning architectures used leading up to and during the event. The underlying theme of this article is to elucidate how the development of future automotive safety systems can potentially be accelerated by tackling the technological challenges of autonomous ground vehicle robotics. Of interest, we will discuss how a production manufacturing mindset imposes a unique set of constraints upon approaching the problem and how this worked for and against us, given the very compressed timeline of the contests. © 2008 Wiley Periodicals, Inc.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/61244/1/20264_ftp.pd

Reliable localization methods for intelligent vehicles based on environment perception

Mención Internacional en el título de doctorIn the near past, we would see autonomous vehicles and Intelligent Transport Systems (ITS) as a potential future of transportation. Today, thanks to all the technological advances in recent years, the feasibility of such systems is no longer a question. Some of these autonomous driving technologies are already sharing our roads, and even commercial vehicles are including more Advanced Driver-Assistance Systems (ADAS) over the years. As a result, transportation is becoming more efficient and the roads are considerably safer. One of the fundamental pillars of an autonomous system is self-localization. An accurate and reliable estimation of the vehicle’s pose in the world is essential to navigation. Within the context of outdoor vehicles, the Global Navigation Satellite System (GNSS) is the predominant localization system. However, these systems are far from perfect, and their performance is degraded in environments with limited satellite visibility. Additionally, their dependence on the environment can make them unreliable if it were to change. Accordingly, the goal of this thesis is to exploit the perception of the environment to enhance localization systems in intelligent vehicles, with special attention to their reliability. To this end, this thesis presents several contributions: First, a study on exploiting 3D semantic information in LiDAR odometry is presented, providing interesting insights regarding the contribution to the odometry output of each type of element in the scene. The experimental results have been obtained using a public dataset and validated on a real-world platform. Second, a method to estimate the localization error using landmark detections is proposed, which is later on exploited by a landmark placement optimization algorithm. This method, which has been validated in a simulation environment, is able to determine a set of landmarks so the localization error never exceeds a predefined limit. Finally, a cooperative localization algorithm based on a Genetic Particle Filter is proposed to utilize vehicle detections in order to enhance the estimation provided by GNSS systems. Multiple experiments are carried out in different simulation environments to validate the proposed method.En un pasado no muy lejano, los vehículos autónomos y los Sistemas Inteligentes del Transporte (ITS) se veían como un futuro para el transporte con gran potencial. Hoy, gracias a todos los avances tecnológicos de los últimos años, la viabilidad de estos sistemas ha dejado de ser una incógnita. Algunas de estas tecnologías de conducción autónoma ya están compartiendo nuestras carreteras, e incluso los vehículos comerciales cada vez incluyen más Sistemas Avanzados de Asistencia a la Conducción (ADAS) con el paso de los años. Como resultado, el transporte es cada vez más eficiente y las carreteras son considerablemente más seguras. Uno de los pilares fundamentales de un sistema autónomo es la autolocalización. Una estimación precisa y fiable de la posición del vehículo en el mundo es esencial para la navegación. En el contexto de los vehículos circulando en exteriores, el Sistema Global de Navegación por Satélite (GNSS) es el sistema de localización predominante. Sin embargo, estos sistemas están lejos de ser perfectos, y su rendimiento se degrada en entornos donde la visibilidad de los satélites es limitada. Además, los cambios en el entorno pueden provocar cambios en la estimación, lo que los hace poco fiables en ciertas situaciones. Por ello, el objetivo de esta tesis es utilizar la percepción del entorno para mejorar los sistemas de localización en vehículos inteligentes, con una especial atención a la fiabilidad de estos sistemas. Para ello, esta tesis presenta varias aportaciones: En primer lugar, se presenta un estudio sobre cómo aprovechar la información semántica 3D en la odometría LiDAR, generando una base de conocimiento sobre la contribución de cada tipo de elemento del entorno a la salida de la odometría. Los resultados experimentales se han obtenido utilizando una base de datos pública y se han validado en una plataforma de conducción del mundo real. En segundo lugar, se propone un método para estimar el error de localización utilizando detecciones de puntos de referencia, que posteriormente es explotado por un algoritmo de optimización de posicionamiento de puntos de referencia. Este método, que ha sido validado en un entorno de simulación, es capaz de determinar un conjunto de puntos de referencia para el cual el error de localización nunca supere un límite previamente fijado. Por último, se propone un algoritmo de localización cooperativa basado en un Filtro Genético de Partículas para utilizar las detecciones de vehículos con el fin de mejorar la estimación proporcionada por los sistemas GNSS. El método propuesto ha sido validado mediante múltiples experimentos en diferentes entornos de simulación.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridSecretario: Joshué Manuel Pérez Rastelli.- Secretario: Jorge Villagrá Serrano.- Vocal: Enrique David Martí Muño

Laserkeilausaineiston ja katunäkymäkuvien hyödyntäminen tieympäristön seurannassa

Utilization of laser scanning has increased during the past few years in many fields of applications, for example, in road environment monitoring. Mild winters, increasing rainfalls and frost are deteriorating the surface and structure of the road causing road damages. The road environment and its condition can be examined for example with laser scanning and street view images. Utilization of laser scanning data and street view images in road environment monitoring was studied in this thesis. The main focus was on the road damages and drainage. Also individual trees were detected nearby road scenes. TerraModeler and TerraScan software were used for investigations. Five different lidar datasets were used to detect road damages and drainage. Both mobile and helicopter-based lidar data were available from Jakomäki area. In Rauma case, there were two datasets collected from the helicopter but the point densities were different. In addition, to helicopter-based lidar data, there were also street view images available from BlomSTREET service in Hyvinkää case. The results between the datasets were compared. Aim was to investigate if same damages can be found from the several datasets that have different point densities. Lidar data for individual tree detection was collected by helicopter from Korppoo area. Tree locations were also measured with a tachymeter to get reference data for automatic detection. Heights of the trees were manually determined from the point cloud. Manually measured heights and locations were compared with automatically detected ones. Detection of rut depths, slopes and drainage is possible from the high point density datasets. From lower point density datasets it is not possible to detect for example rut depths. Point cloud is possible to color by slopes, which may give some information about rut locations even from lower point density datasets. Obtaining slopes and drainage accurately is also possible from lower point density data. With TerraModeler water gathering points can be obtained. Panorama pictures from BlomSTREET can be utilized for ensuring if there is a rainwater outlet or if water will gather as a puddle. Tree locations were detected in a meter accuracy with automatic method. Successful detection of tree heights and locations is dependent on many things. Successful classification of the data and creation of tree models are the most important parameters.Laserkeilaus on yleistynyt ja sitä hyödynnetään useissa eri sovelluksissa kuten esimerkiksi tiesovelluksissa. Leudot ja sateiset talvet sekä routa kuluttavat tien pintaa ja rakennetta aiheuttaen tievaurioita, jotka voivat olla vaaraksi liikenteelle. Tienkuntoa ja sen ympäristöä voidaan tarkastella esimerkiksi laserkeilausaineistojen sekä katunäkymäkuvien avulla. Työssä tutkittiin kuinka laserkeilausaineistoa ja katunäkymäkuvia voidaan hyödyntää tieympäristön seurannassa. Tutkimuksessa keskityttiin tarkastelemaan tievaurioita ja kuivatusta sekä tiealueiden läheisyydessä sijaitsevien puiden tunnistusta. Tutkimuksessa käytettiin TerraModeler ja TerraScan ohjelmistoja. Tievaurioita ja kuivatusta tutkittiin viidestä eri aineistosta kolmelta eri alueelta. Jakomäen alueelta tien ominaisuuksia tutkittiin sekä mobiili- että helikopterilaserkeilausaineistosta ja Rauman alueelta vaurioita kartoitettiin kahdesta eri helikopterilla kerätystä pistetiheyden aineistosta. Hyvinkäältä helikopterilla kerätyn laserkeilausaineiston lisäksi oli saatavilla katunäkymäkuvia BlomSTREET palvelusta. Aineistoista saatuja tuloksia vertailtiin keskenään ja tutkittiin, onko niistä mahdollista havaita samankaltaisia tuloksia. Yksittäisen puun tunnistukseen käytettiin helikopterilla kerättyä laserkeilausaineistoa Korppoon alueelta ja referenssinä aineistolle toimi maastossa mitatut puiden sijainnit. Automaattisesti määritettyjen puiden sijaintia verrattiin maastossa mitattuihin sijainteihin. Myös puiden korkeus määritettiin pistepilvestä manuaalisesti ja tätä verrattiin automaattiseen korkeuden määritykseen. Korkean pistetiheyden laserkeilausaineistoilla on mahdollista tutkia tien urautumista, tien kaltevuuksia ja kuivatusta. Matalamman pistetiheyden aineistoista ei pystytä määrittämään esimerkiksi urasyvyyksiä. Pistepilvi on mahdollista värjätä kaltevuuksien mukaan, minkä avulla urautumista voidaan havaita jossain määrin myös matalampien pistetiheyksien aineistoista. Tien kaltevuuksia ja kuivatusta pystytään havaitsemaan tarkasti jopa alhaisista pistetiheyden aineistoista. TerraModelerin avulla voidaan määrittää alueet, johon sadevesi kasautuu. BlomSTREET 360 panoraamakuvien avulla pystytään tarkastamaan onko kohdassa sadevesikaivo vai kerääntyykö vesi lammikoiksi. Yksittäisten puiden sijainnin määrittäminen onnistui noin metrin tarkkuudella, mutta sijainnin ja korkeuden määrittämisen onnistuminen on riippuvainen monesta tekijästä. Pistepilven luokittelun onnistumisen lisäksi yksi tärkeä tekijä on puiden muodoista tehdyt mallit, joiden avulla TerraScan ohjelmisto etsii yksittäisiä puita