Reliable localization methods for intelligent vehicles based on environment perception

Mención Internacional en el título de doctorIn the near past, we would see autonomous vehicles and Intelligent Transport Systems (ITS) as a potential future of transportation. Today, thanks to all the technological advances in recent years, the feasibility of such systems is no longer a question. Some of these autonomous driving technologies are already sharing our roads, and even commercial vehicles are including more Advanced Driver-Assistance Systems (ADAS) over the years. As a result, transportation is becoming more efficient and the roads are considerably safer. One of the fundamental pillars of an autonomous system is self-localization. An accurate and reliable estimation of the vehicle’s pose in the world is essential to navigation. Within the context of outdoor vehicles, the Global Navigation Satellite System (GNSS) is the predominant localization system. However, these systems are far from perfect, and their performance is degraded in environments with limited satellite visibility. Additionally, their dependence on the environment can make them unreliable if it were to change. Accordingly, the goal of this thesis is to exploit the perception of the environment to enhance localization systems in intelligent vehicles, with special attention to their reliability. To this end, this thesis presents several contributions: First, a study on exploiting 3D semantic information in LiDAR odometry is presented, providing interesting insights regarding the contribution to the odometry output of each type of element in the scene. The experimental results have been obtained using a public dataset and validated on a real-world platform. Second, a method to estimate the localization error using landmark detections is proposed, which is later on exploited by a landmark placement optimization algorithm. This method, which has been validated in a simulation environment, is able to determine a set of landmarks so the localization error never exceeds a predefined limit. Finally, a cooperative localization algorithm based on a Genetic Particle Filter is proposed to utilize vehicle detections in order to enhance the estimation provided by GNSS systems. Multiple experiments are carried out in different simulation environments to validate the proposed method.En un pasado no muy lejano, los vehículos autónomos y los Sistemas Inteligentes del Transporte (ITS) se veían como un futuro para el transporte con gran potencial. Hoy, gracias a todos los avances tecnológicos de los últimos años, la viabilidad de estos sistemas ha dejado de ser una incógnita. Algunas de estas tecnologías de conducción autónoma ya están compartiendo nuestras carreteras, e incluso los vehículos comerciales cada vez incluyen más Sistemas Avanzados de Asistencia a la Conducción (ADAS) con el paso de los años. Como resultado, el transporte es cada vez más eficiente y las carreteras son considerablemente más seguras. Uno de los pilares fundamentales de un sistema autónomo es la autolocalización. Una estimación precisa y fiable de la posición del vehículo en el mundo es esencial para la navegación. En el contexto de los vehículos circulando en exteriores, el Sistema Global de Navegación por Satélite (GNSS) es el sistema de localización predominante. Sin embargo, estos sistemas están lejos de ser perfectos, y su rendimiento se degrada en entornos donde la visibilidad de los satélites es limitada. Además, los cambios en el entorno pueden provocar cambios en la estimación, lo que los hace poco fiables en ciertas situaciones. Por ello, el objetivo de esta tesis es utilizar la percepción del entorno para mejorar los sistemas de localización en vehículos inteligentes, con una especial atención a la fiabilidad de estos sistemas. Para ello, esta tesis presenta varias aportaciones: En primer lugar, se presenta un estudio sobre cómo aprovechar la información semántica 3D en la odometría LiDAR, generando una base de conocimiento sobre la contribución de cada tipo de elemento del entorno a la salida de la odometría. Los resultados experimentales se han obtenido utilizando una base de datos pública y se han validado en una plataforma de conducción del mundo real. En segundo lugar, se propone un método para estimar el error de localización utilizando detecciones de puntos de referencia, que posteriormente es explotado por un algoritmo de optimización de posicionamiento de puntos de referencia. Este método, que ha sido validado en un entorno de simulación, es capaz de determinar un conjunto de puntos de referencia para el cual el error de localización nunca supere un límite previamente fijado. Por último, se propone un algoritmo de localización cooperativa basado en un Filtro Genético de Partículas para utilizar las detecciones de vehículos con el fin de mejorar la estimación proporcionada por los sistemas GNSS. El método propuesto ha sido validado mediante múltiples experimentos en diferentes entornos de simulación.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridSecretario: Joshué Manuel Pérez Rastelli.- Secretario: Jorge Villagrá Serrano.- Vocal: Enrique David Martí Muño

A Systematic Survey of Control Techniques and Applications: From Autonomous Vehicles to Connected and Automated Vehicles

Vehicle control is one of the most critical challenges in autonomous vehicles (AVs) and connected and automated vehicles (CAVs), and it is paramount in vehicle safety, passenger comfort, transportation efficiency, and energy saving. This survey attempts to provide a comprehensive and thorough overview of the current state of vehicle control technology, focusing on the evolution from vehicle state estimation and trajectory tracking control in AVs at the microscopic level to collaborative control in CAVs at the macroscopic level. First, this review starts with vehicle key state estimation, specifically vehicle sideslip angle, which is the most pivotal state for vehicle trajectory control, to discuss representative approaches. Then, we present symbolic vehicle trajectory tracking control approaches for AVs. On top of that, we further review the collaborative control frameworks for CAVs and corresponding applications. Finally, this survey concludes with a discussion of future research directions and the challenges. This survey aims to provide a contextualized and in-depth look at state of the art in vehicle control for AVs and CAVs, identifying critical areas of focus and pointing out the potential areas for further exploration

Distributed Fault Detection in Formation of Multi-Agent Systems with Attack Impact Analysis

Autonomous Underwater Vehicles (AUVs) are capable of performing a variety of deepwater marine applications as in multiple mobile robots and cooperative robot reconnaissance. Due to the environment that AUVs operate in, fault detection and isolation as well as the formation control of AUVs are more challenging than other Multi-Agent Systems (MASs). In this thesis, two main challenges are tackled. We first investigate the formation control and fault accommodation algorithms for AUVs in presence of abnormal events such as faults and communication attacks in any of the team members. These undesirable events can prevent the entire team to achieve a safe, reliable, and efficient performance while executing underwater mission tasks. For instance, AUVs may face unexpected actuator/sensor faults and the communication between AUVs can be compromised, and consequently make the entire multi-agent system vulnerable to cyber-attacks. Moreover, a possible deception attack on network system may have a negative impact on the environment and more importantly the national security. Furthermore, there are certain requirements for speed, position or depth of the AUV team. For this reason, we propose a distributed fault detection scheme that is able to detect and isolate faults in AUVs while maintaining their formation under security constraints. The effects of faults and communication attacks with a control theoretical perspective will be studied. Another contribution of this thesis is to study a state estimation problem for a linear dynamical system in presence of a Bias Injection Attack (BIA). For this purpose, a Kalman Filter (KF) is used, where we show that the impact of an attack can be analyzed as the solution of a quadratically constrained problem for which the exact solution can be found efficiently. We also introduce a lower bound for the attack impact in terms of the number of compromised actuators and a combination of sensors and actuators. The theoretical findings are accompanied by simulation results and numerical can study examples

Control and communication systems for automated vehicles cooperation and coordination

Mención Internacional en el título de doctorThe technological advances in the Intelligent Transportation Systems (ITS) are exponentially improving over the last century. The objective is to provide intelligent and innovative services for the different modes of transportation, towards a better, safer, coordinated and smarter transport networks. The Intelligent Transportation Systems (ITS) focus is divided into two main categories; the first is to improve existing components of the transport networks, while the second is to develop intelligent vehicles which facilitate the transportation process. Different research efforts have been exerted to tackle various aspects in the fields of the automated vehicles. Accordingly, this thesis is addressing the problem of multiple automated vehicles cooperation and coordination. At first, 3DCoAutoSim driving simulator was developed in Unity game engine and connected to Robot Operating System (ROS) framework and Simulation of Urban Mobility (SUMO). 3DCoAutoSim is an abbreviation for "3D Simulator for Cooperative Advanced Driver Assistance Systems (ADAS) and Automated Vehicles Simulator". 3DCoAutoSim was tested under different circumstances and conditions, afterward, it was validated through carrying-out several controlled experiments and compare the results against their counter reality experiments. The obtained results showed the efficiency of the simulator to handle different situations, emulating real world vehicles. Next is the development of the iCab platforms, which is an abbreviation for "Intelligent Campus Automobile". The platforms are two electric golf-carts that were modified mechanically, electronically and electrically towards the goal of automated driving. Each iCab was equipped with several on-board embedded computers, perception sensors and auxiliary devices, in order to execute the necessary actions for self-driving. Moreover, the platforms are capable of several Vehicle-to-Everything (V2X) communication schemes, applying three layers of control, utilizing cooperation architecture for platooning, executing localization systems, mapping systems, perception systems, and finally several planning systems. Hundreds of experiments were carried-out for the validation of each system in the iCab platform. Results proved the functionality of the platform to self-drive from one point to another with minimal human intervention.Los avances tecnológicos en Sistemas Inteligentes de Transporte (ITS) han crecido de forma exponencial durante el último siglo. El objetivo de estos avances es el de proveer de sistemas innovadores e inteligentes para ser aplicados a los diferentes medios de transporte, con el fin de conseguir un transporte mas eficiente, seguro, coordinado e inteligente. El foco de los ITS se divide principalmente en dos categorías; la primera es la mejora de los componentes ya existentes en las redes de transporte, mientras que la segunda es la de desarrollar vehículos inteligentes que hagan más fácil y eficiente el transporte. Diferentes esfuerzos de investigación se han llevado a cabo con el fin de solucionar los numerosos aspectos asociados con la conducción autónoma. Esta tesis propone una solución para la cooperación y coordinación de múltiples vehículos. Para ello, en primer lugar se desarrolló un simulador (3DCoAutoSim) de conducción basado en el motor de juegos Unity, conectado al framework Robot Operating System (ROS) y al simulador Simulation of Urban Mobility (SUMO). 3DCoAutoSim ha sido probado en diferentes condiciones y circunstancias, para posteriormente validarlo con resultados a través de varios experimentos reales controlados. Los resultados obtenidos mostraron la eficiencia del simulador para manejar diferentes situaciones, emulando los vehículos en el mundo real. En segundo lugar, se desarrolló la plataforma de investigación Intelligent Campus Automobile (iCab), que consiste en dos carritos eléctricos de golf, que fueron modificados eléctrica, mecánica y electrónicamente para darle capacidades autónomas. Cada iCab se equipó con diferentes computadoras embebidas, sensores de percepción y unidades auxiliares, con la finalidad de transformarlos en vehículos autónomos. Además, se les han dado capacidad de comunicación multimodal (V2X), se les han aplicado tres capas de control, incorporando una arquitectura de cooperación para operación en modo tren, diferentes esquemas de localización, mapeado, percepción y planificación de rutas. Innumerables experimentos han sido realizados para validar cada uno de los diferentes sistemas incorporados. Los resultados prueban la funcionalidad de esta plataforma para realizar conducción autónoma y cooperativa con mínima intervención humana.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Francisco Javier Otamendi Fernández de la Puebla.- Secretario: Hanno Hildmann.- Vocal: Pietro Cerr

A systematic review of perception system and simulators for autonomous vehicles research

This paper presents a systematic review of the perception systems and simulators for autonomous vehicles (AV). This work has been divided into three parts. In the first part, perception systems are categorized as environment perception systems and positioning estimation systems. The paper presents the physical fundamentals, principle functioning, and electromagnetic spectrum used to operate the most common sensors used in perception systems (ultrasonic, RADAR, LiDAR, cameras, IMU, GNSS, RTK, etc.). Furthermore, their strengths and weaknesses are shown, and the quantification of their features using spider charts will allow proper selection of different sensors depending on 11 features. In the second part, the main elements to be taken into account in the simulation of a perception system of an AV are presented. For this purpose, the paper describes simulators for model-based development, the main game engines that can be used for simulation, simulators from the robotics field, and lastly simulators used specifically for AV. Finally, the current state of regulations that are being applied in different countries around the world on issues concerning the implementation of autonomous vehicles is presented.This work was partially supported by DGT (ref. SPIP2017-02286) and GenoVision (ref. BFU2017-88300-C2-2-R) Spanish Government projects, and the “Research Programme for Groups of Scientific Excellence in the Region of Murcia" of the Seneca Foundation (Agency for Science and Technology in the Region of Murcia – 19895/GERM/15)

Machine Learning for Multi-Robot Semantic Simultaneous Localization and Mapping

RÉSUMÉ L’automatisation et la robotique prennent une place de plus en plus importante dans notre vie quotidienne, avec de nombreuses utilisations possibles. Les robots pourraient nous épargner des tâches dangereuses et pénibles, ou rendre des choses impossibles jusqu’à maintenant possibles. Pour que les robots s’intègrent en toute sécurité dans notre monde et dans de nouveaux environnements inconnus, il est clef qu’ils soient équipés d’une capacité de per-ception, et en particulier qu’ils puissent se localiser par rapport à leur entourage. Afin d’être réellement indépendants, les robots doivent pouvoir le faire en se basant uniquement sur leurs propres capteurs, les plus couramment utilisés étant les caméras. Une solution pour obtenir de telles estimations est d’utiliser un algorithme de cartographie et localisa-tion simultanée (SLAM), dans lequel le robot va simultanément construire une carte de son environnement et estimer son propre état. Le SLAM avec un seul robot a fait l’objet de nombreux travaux scientifiques, et est désormais considéré comme un domaine de recherche mature. Cependant, l’utilisation d’une équipe de robots peut o˙rir plusieurs avantages en termes de robustesse, d’eÿcacité et de performances pour de nombreuses tâches. Dans ce cas, des algorithmes de SLAM multi-robots sont nécessaires pour permettre à chaque robot de bénéficier de l’expérience de toute l’équipe. Le SLAM multi-robot peut s’appuyer sur des solutions SLAM classiques, mais nécessite des adaptations et fait face à des contraintes de calculs et de communications supplémentaires. Un défi particulier dans le SLAM multi-robots est la nécessité pour les robots de trouver des fermetures de boucles inter-robots: des liens entre les trajectoires de di˙érents robots qui peuvent être trouvés lorsqu’ils visitent le même endroit. Deux catégories d’approches sont possibles pour détecter les fermetures de boucles inter-robots. Dans les méthodes indirectes, les robots communiquent pour vérifier s’ils ont cartographié un espace commun, puis tentent de trouver des fermetures de boucles à partir des données recueillies par chacun des robots dans cet espace. Dans les méthodes directes, les robots s’appuient directement sur les données de leurs capteurs pour estimer les fermetures de boucles. Chaque approche a des avantages et des inconvénients, mais les méthodes indi-rectes ont été plus étudiées récemment. Ce mémoire s’appuie sur les avancées récentes de la vision par ordinateur pour présenter des contributions à chaque catégorie d’approches pour la détection de fermetures de boucles inter-robots. Une première contribution est présentée pour la détection de fermetures de boucles indirecte dans une équipe de robots entièrement en communication. Elle utilise des constellations, une représentation sémantique compacte de l’environnement basée sur les objets qui le compose.----------ABSTRACT Automation and robotics are becoming more and more common in our daily lives, with many possible applications. Deploying robots in the world can extend what humans are capable of doing, and can save us from dangerous and strenuous tasks. For robots to be safely sent out in our real world, and in new unknown environments, one key capability they need is to perceive their environment, and particularly to localize themselves with respect to their surroundings. To truly be able to be deployed anywhere, robots should be able to do so relying only on their sensors, the most commonly used being cameras. One way to generate such an estimate is by using a simultaneous localization and mapping (SLAM) algorithm, in which the robot will concurrently build a map of its environment and estimate its state within it. Single-robot SLAM has been extensively researched and is now considered a mature field. However, using a team of robots can provide several benefits in terms of robustness, eÿciency, and performance for many tasks. In this case, multi-robot SLAM algorithms are required to allow each robot to benefit from the whole team’s experience. Multi-robot SLAM can build on top of single-robot SLAM solutions, but requires adaptations and faces computation and communication constraints. One particular challenge that arises in multi-robot SLAM is the need for robots to find inter-robot loop closures: relationships between trajectories of di˙erent robots that can be found when they visit the same place. Two categories of approaches are possible to detect inter-robot loop closures. In indirect methods, robots communicate to find if they have mapped the same area, and then attempt to find loop closures using data gathered by each robot in the place that was jointly visited. In direct methods, robots directly rely on data they gather from their sensors to estimate the loop closures. Each approach has its own benefits and challenges, with indirect methods being more popular in recent works. This thesis builds on recent computer vision advancements to present contributions to each category of approaches for inter-robot loop closure detection. A first approach is presented for indirect loop closure detection in a team of fully connected robots. It relies on constellations, a compact semantic representation of the environment based on objects that are in it. Descriptors and comparison methods for constellations are designed to robustly recognize places based on their constellation with minimal data exchange. These are used in a decentralized place recognition mechanism that is scalable as the size of the team increases. The proposed method performs comparably to state-of-the-art solutions in terms of performance and data exchanges require, while being more meaningful and interpretable

Mobile Robots Navigation

Mobile robots navigation includes different interrelated activities: (i) perception, as obtaining and interpreting sensory information; (ii) exploration, as the strategy that guides the robot to select the next direction to go; (iii) mapping, involving the construction of a spatial representation by using the sensory information perceived; (iv) localization, as the strategy to estimate the robot position within the spatial map; (v) path planning, as the strategy to find a path towards a goal location being optimal or not; and (vi) path execution, where motor actions are determined and adapted to environmental changes. The book addresses those activities by integrating results from the research work of several authors all over the world. Research cases are documented in 32 chapters organized within 7 categories next described

Probabilistic Framework for Sensor Management

A probabilistic sensor management framework is introduced, which maximizes the utility of sensor systems with many different sensing modalities by dynamically configuring the sensor system in the most beneficial way. For this purpose, techniques from stochastic control and Bayesian estimation are combined such that long-term effects of possible sensor configurations and stochastic uncertainties resulting from noisy measurements can be incorporated into the sensor management decisions