Exploitation des données cartographiques pour la perception de véhicules intelligents

This thesis is situated in the domains of robotics and data fusion, and concerns geographic information systems. We study the utility of adding digital maps, which model the urban environment in which the vehicle evolves, as a virtual sensor improving the perception results. Indeed, the maps contain a phenomenal quantity of information about the environment : its geometry, topology and additional contextual information. In this work, we extract road surface geometry and building models in order to deduce the context and the characteristics of each detected object. Our method is based on an extension of occupancy grids : the evidential perception grids. It permits to model explicitly the uncertainty related to the map and sensor data. By this means, the approach presents also the advantage of representing homogeneously the data originating from various sources : lidar, camera or maps. The maps are handled on equal terms with the physical sensors. This approach allows us to add geographic information without imputing unduly importance to it, which is essential in presence of errors. In our approach, the information fusion result, stored in a perception grid, is used to predict the stateof environment on the next instant. The fact of estimating the characteristics of dynamic elements does not satisfy the hypothesis of static world. Therefore, it is necessary to adjust the level of certainty attributed to these pieces of information. We do so by applying the temporal discounting. Due to the fact that existing methods are not well suited for this application, we propose a family of discoun toperators that take into account the type of handled information. The studied algorithms have been validated through tests on real data. We have thus developed the prototypes in Matlab and the C++ software based on Pacpus framework. Thanks to them, we present the results of experiments performed in real conditions.La plupart des logiciels contrôlant les véhicules intelligents traite de la compréhension de la scène. De nombreuses méthodes existent actuellement pour percevoir les obstacles de façon automatique. La majorité d’entre elles emploie ainsi les capteurs extéroceptifs comme des caméras ou des lidars. Cette thèse porte sur les domaines de la robotique et de la fusion d’information et s’intéresse aux systèmes d’information géographique. Nous étudions ainsi l’utilité d’ajouter des cartes numériques, qui cartographient le milieu urbain dans lequel évolue le véhicule, en tant que capteur virtuel améliorant les résultats de perception. Les cartes contiennent en effet une quantité phénoménale d’information sur l’environnement : sa géométrie, sa topologie ainsi que d’autres informations contextuelles. Dans nos travaux, nous avons extrait la géométrie des routes et des modèles de bâtiments afin de déduire le contexte et les caractéristiques de chaque objet détecté. Notre méthode se base sur une extension de grilles d’occupations : les grilles de perception crédibilistes. Elle permet de modéliser explicitement les incertitudes liées aux données de cartes et de capteurs. Elle présente également l’avantage de représenter de façon uniforme les données provenant de différentes sources : lidar, caméra ou cartes. Les cartes sont traitées de la même façon que les capteurs physiques. Cette démarche permet d’ajouter les informations géographiques sans pour autant leur donner trop d’importance, ce qui est essentiel en présence d’erreurs. Dans notre approche, le résultat de la fusion d’information contenu dans une grille de perception est utilisé pour prédire l’état de l’environnement à l’instant suivant. Le fait d’estimer les caractéristiques des éléments dynamiques ne satisfait donc plus l’hypothèse du monde statique. Par conséquent, il est nécessaire d’ajuster le niveau de certitude attribué à ces informations. Nous y parvenons en appliquant l’affaiblissement temporel. Étant donné que les méthodes existantes n’étaient pas adaptées à cette application, nous proposons une famille d’opérateurs d’affaiblissement prenant en compte le type d’information traitée. Les algorithmes étudiés ont été validés par des tests sur des données réelles. Nous avons donc développé des prototypes en Matlab et des logiciels en C++ basés sur la plate-forme Pacpus. Grâce à eux nous présentons les résultats des expériences effectués en conditions réelles

Fusion de données multi capteurs pour la détection et le suivi d'objets mobiles à partir d'un véhicule autonome

La perception est un point clé pour le fonctionnement d'un véhicule autonome ou même pour un véhicule fournissant des fonctions d'assistance. Un véhicule observe le monde externe à l'aide de capteurs et construit un modèle interne de l'environnement extérieur. Il met à jour en continu ce modèle de l'environnement en utilisant les dernières données des capteurs. Dans ce cadre, la perception peut être divisée en deux étapes : la première partie, appelée SLAM (Simultaneous Localization And Mapping) s'intéresse à la construction d'une carte de l'environnement extérieur et à la localisation du véhicule hôte dans cette carte, et deuxième partie traite de la détection et du suivi des objets mobiles dans l'environnement (DATMO pour Detection And Tracking of Moving Objects). En utilisant des capteurs laser de grande précision, des résultats importants ont été obtenus par les chercheurs. Cependant, avec des capteurs laser de faible résolution et des données bruitées, le problème est toujours ouvert, en particulier le problème du DATMO. Dans cette thèse nous proposons d'utiliser la vision (mono ou stéréo) couplée à un capteur laser pour résoudre ce problème. La première contribution de cette thèse porte sur l'identification et le développement de trois niveaux de fusion. En fonction du niveau de traitement de l'information capteur avant le processus de fusion, nous les appelons "fusion bas niveau", "fusion au niveau de la détection" et "fusion au niveau du suivi". Pour la fusion bas niveau, nous avons utilisé les grilles d'occupations. Pour la fusion au niveau de la détection, les objets détectés par chaque capteur sont fusionnés pour avoir une liste d'objets fusionnés. La fusion au niveau du suivi requiert le suivi des objets pour chaque capteur et ensuite on réalise la fusion entre les listes d'objets suivis. La deuxième contribution de cette thèse est le développement d'une technique rapide pour trouver les bords de route à partir des données du laser et en utilisant cette information nous supprimons de nombreuses fausses alarmes. Nous avons en effet observé que beaucoup de fausses alarmes apparaissent sur le bord de la route. La troisième contribution de cette thèse est le développement d'une solution complète pour la perception avec un capteur laser et des caméras stéréo-vision et son intégration sur un démonstrateur du projet européen Intersafe-2. Ce projet s'intéresse à la sécurité aux intersections et vise à y réduire les blessures et les accidents mortels. Dans ce projet, nous avons travaillé en collaboration avec Volkswagen, l'Université Technique de Cluj-Napoca, en Roumanie et l'INRIA Paris pour fournir une solution complète de perception et d'évaluation des risques pour le démonstrateur de Volkswagen.Perception is one of important steps for the functioning of an autonomous vehicle or even for a vehicle providing only driver assistance functions. Vehicle observes the external world using its sensors and builds an internal model of the outer environment configuration. It keeps on updating this internal model using latest sensor data. In this setting perception can be divided into two sub parts: first part, called SLAM(Simultaneous Localization And Mapping), is concerned with building an online map of the external environment and localizing the host vehicle in this map, and second part deals with finding moving objects in the environment and tracking them over time and is called DATMO(Detection And Tracking of Moving Objects). Using high resolution and accurate laser scanners successful efforts have been made by many researchers to solve these problems. However, with low resolution or noisy laser scanners solving these problems, especially DATMO, is still a challenge and there are either many false alarms, miss detections or both. In this thesis we propose that by using vision sensor (mono or stereo) along with laser sensor and by developing an effective fusion scheme on an appropriate level, these problems can be greatly reduced. The main contribution of this research is concerned with the identification of three fusion levels and development of fusion techniques for each level for SLAM and DATMO based perception architecture of autonomous vehicles. Depending on the amount of preprocessing required before fusion for each level, we call them low level, object detection level and track level fusion. For low level we propose to use grid based fusion technique and by giving appropriate weights (depending on the sensor properties) to each grid for each sensor a fused grid can be obtained giving better view of the external environment in some sense. For object detection level fusion, lists of objects detected for each sensor are fused to get a list of fused objects where fused objects have more information then their previous versions. We use a Bayesian fusion technique for this level. Track level fusion requires to track moving objects for each sensor separately and then do a fusion between tracks to get fused tracks. Fusion at this level helps remove false tracks. Second contribution of this research is the development of a fast technique of finding road borders from noisy laser data and then using these border information to remove false moving objects. Usually we have observed that many false moving objects appear near the road borders due to sensor noise. If they are not filtered out then they result into many false tracks close to vehicle making vehicle to apply breaks or to issue warning messages to the driver falsely. Third contribution is the development of a complete perception solution for lidar and stereo vision sensors and its intigration on a real vehicle demonstrator used for a European Union project (INTERSAFE-21). This project is concerned with the safety at intersections and aims at the reduction of injury and fatal accidents there. In this project we worked in collaboration with Volkswagen, Technical university of Cluj-Napoca Romania and INRIA Paris to provide a complete perception and risk assessment solution for this project.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Cooperative Situation Awareness in Transportation

Intelligent Transportation Systems (ITS) became a fast moving eld of research in the last decades, in particular in the context of continuously growing mobility and a high employment of resources starting from energy and material consumption to travel time and nally the human life. As it has already been experienced in other application areas, the introduction of communications technology is able to bring a revolutionary change in structures and behaviors long-believed to be carved in stone. The main idea behind this thesis is the usage of information not as a mere placeholder, e.g. a eld in a static message, but actively utilizing its content and dependencies. This requires an estimation of the actual worth of a single piece of information for the entity itself and the entities which are in communication range. This worth has to be the essential driver for the cooperative situation estimation. The active utilization of information and its cooperative dissemination provides the entities the opportunity to become situation aware and detect hazardous or inefficient situations early in advance. Since information always has a degree of uncertainty which is inherent to information in the real-world problem domain, as we are confronted with in ITS, probabilistic methods will be applied to model situation-relevant information. Conditional probability distributions in state transition models make for the evolvement of the situational information with the progress of time and handle causal dependencies between situational information. Together with a utility-based decision-making process dynamic probabilistic causal decision networks provide the functionality to select optimal actions given sequences of inaccurate and incomplete evidences. This thesis provides concepts and strategies that push forward the exploitation of information in a cooperative way within a probabilistic framework that allows to make various kinds of decisions with maximum utility. For the evaluation of the proposed concepts, the exemplary application Cooperative Adaptive Cruise Control (CACC) has been implemented on the basis of a particle lter which is used for the situation estimation. Initial simulations provided promising results and hence constitute a solid basis for future work in the eld of Cooperative Situation Awareness in Transportation

Fusion of Data from Heterogeneous Sensors with Distributed Fields of View and Situation Evaluation for Advanced Driver Assistance Systems

In order to develop a driver assistance system for pedestrian protection, pedestrians in the environment of a truck are detected by radars and a camera and are tracked across distributed fields of view using a Joint Integrated Probabilistic Data Association filter. A robust approach for prediction of the system vehicles trajectory is presented. It serves the computation of a probabilistic collision risk based on reachable sets where different sources of uncertainty are taken into account

Laser-Based Detection and Tracking of Moving Obstacles to Improve Perception of Unmanned Ground Vehicles

El objetivo de esta tesis es desarrollar un sistema que mejore la etapa de percepción de vehículos terrestres no tripulados (UGVs) heterogéneos, consiguiendo con ello una navegación robusta en términos de seguridad y ahorro energético en diferentes entornos reales, tanto interiores como exteriores. La percepción debe tratar con obstáculos estáticos y dinámicos empleando sensores heterogéneos, tales como, odometría, sensor de distancia láser (LIDAR), unidad de medida inercial (IMU) y sistema de posicionamiento global (GPS), para obtener la información del entorno con la precisión más alta, permitiendo mejorar las etapas de planificación y evitación de obstáculos. Para conseguir este objetivo, se propone una etapa de mapeado de obstáculos dinámicos (DOMap) que contiene la información de los obstáculos estáticos y dinámicos. La propuesta se basa en una extensión del filtro de ocupación bayesiana (BOF) incluyendo velocidades no discretizadas. La detección de velocidades se obtiene con Flujo Óptico sobre una rejilla de medidas LIDAR discretizadas. Además, se gestionan las oclusiones entre obstáculos y se añade una etapa de seguimiento multi-hipótesis, mejorando la robustez de la propuesta (iDOMap). La propuesta ha sido probada en entornos simulados y reales con diferentes plataformas robóticas, incluyendo plataformas comerciales y la plataforma (PROPINA) desarrollada en esta tesis para mejorar la colaboración entre equipos de humanos y robots dentro del proyecto ABSYNTHE. Finalmente, se han propuesto métodos para calibrar la posición del LIDAR y mejorar la odometría con una IMU

Information Fusion Methodology for Enhancing Situation Awareness in Connected Cars Environment

This dissertation introduces novel approaches to develop a comprehensive model to address situation awareness in the Internet of Cars, called Attention Assist Framework (AAF). The proposed framework utilizes both Low-Level Data Fusion (LLDF), and High-Level Information Fusion (HLIF) to implement traffic entity, situation, and impact assessment, as well as decision making. The Internet of Cars is the convergence of the Internet of Things and Vehicular Ad-hoc Networks (VANETs). In fact, VANETs are the communication platforms that make possible the implementation of the Internet of Cars, and has become an integral part of this research field due to its major role to improve vehicle and road safety, traffic efficiency, and convenience as well as comfort to both drivers and passengers. Significant amount of VANETs research work has been focused on specific areas such as safety, routing, broadcasting, Quality of Service (QoS), and security. Among them, road safety issues are deemed one of the most challenging problems of VANETs. Specifically, lack of proper situational awareness of drivers has been shown to be the main cause of road accidents which makes it a major factor in road safety. The traffic entity assessment relies on a LLDF framework that is able to incorporate various multi-sensor data fusion approaches with means of communication links in VANETs. This is used to implement a cooperative localization approach through fusing common data fusion methods, such as Extended Kalman Filter (EKF) and Unscented Transform (UT), and vehicle-to-vehicle communication in VANETs. Furthermore, traffic situation assessment is based on a fuzzy extension to the Multi-Entity Bayesian Networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Fuzzy Bayesian Networks in handling uncertainty, while tackling the inherent vagueness in the soft data created by human entities. Finally, the impact assessment and decision making is realized through incorporating notions of game theory into Fuzzy-MEBNs, and introducing Active Fuzzy-MEBN (ATFY-MEBN), which is capable in hypothesizing future situations by assessing the impact of the current situation upon taking the actions indicated by an optimal strategy. In fact, such strategies are achieved through solving the games that are generated through a novel situation-specific normal form games generation algorithm that aims to create games based on the given context. In general, ATFY-MEBN presents the concepts of players and actions, and includes new game components, along with a 2-tier architecture, to efficiently model impact assessment and decision making. To demonstrate the capabilities of the proposed framework, a collision warning system simulator is developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of both local and global information sources available in the VANETs environment, and suggests an optimal action by assessing the impact of the current situation through generating and solving situation-specific games. Accordingly, first, the entities that highly influence the safety aspect, as well as both their casual and semantic relationships are identified. Next, an ATFY-MEBN-based model is presented, which allows for modeling these entities along with their relationships in specific contexts, assessing the current states of the situations of interest, predicting their future states, and finally suggesting optimal decision. Therefore, if the likelihood of being in a near-collision situation is determined to be high, and if the relevant situation-specific game is generated, then the impact of deciding on different combinations of actions that the game players take are calculated through a pre-fixed payoff function. Finally, the completed game is solved by finding its dominant strategy, that subsequently, results in proposing the optimal action to the driver. Our experimental results are divided into three main sections, through which we evaluate the capabilities of the traffic entity, situation, and impact assessment methods. Accordingly, the performance of the proposed cooperative localization approach is assessed by comparing its results with the ground truth solution and that of the other localization methods in various driving test cases. Moreover, two distinct single-vehicle and multi-vehicles categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capabilities of the proposed traffic assessment model to efficiently achieve situation and threat assessment on the road. Finally, the impact assessment and decision making models are evaluated through two different scenarios of driving in highway and intersection that are formed with various number of player vehicles, and their actions

Kollektive Perzeption in fahrzeugbasierten Ad-hoc Netzwerken

In combination with the current developments in the area of automatically driving vehicles, the introduction of inter-vehicle communication plays a crucial role for realising the long-term objective of what is known as cooperative driving. A cornerstone for the expansion of automated vehicles is their thorough understanding of the current driving environment. For this purpose, each vehicle generates an environment model containing information about other perceived traffic participants and objects. Local perception sensors are important data providers for this model, as they contribute implicit knowledge about the environment. In combination with a direct communication link between traffic participants, explicit knowledge can be added to the environment model as well. The key concept developed within this thesis is called Collective Perception: it focuses on sharing data gathered by local perception sensors of one vehicle with other traffic participants by means of inter-vehicle communication. As a result of this concept, future applications relying on a comprehensive understanding of the current driving environment are made feasible. The analyses presented in this thesis employ a vehicular ad-hoc network (VANET) based on the standardised framework of the European IEEE 802.11p-based ITS G5 protocol stack for inter-vehicle communication. The effectiveness of the technology relies on an existing communication link between a sufficient number of communication partners - the critical mass. The expansion of inter-vehicle communication, however, can be supported by capacitating indirect effects. Collective Perception is one representative of these effects, as the information density within the network between the vehicles is increased, even at low market penetration rates. At the core of Collective Perception stands the introduction of a message format which serves as a vehicle for the exchange of sensor data within a VANET. The development of the message is influenced by two perspectives: First, the vehicle perspective affects the relevant contents of the message required by data-fusion processes and application algorithms. Second, from the network perspective, constraints resulting from the network stack and effects caused by congestion control mechanisms have to be considered. This thesis addresses both perspectives to develop a holistic concept for exchanging sensor data within a VANET.Im Zusammenhang mit den aktuellen Entwicklungen im Themenbereich automatisch fahrender Fahrzeuge spielt die Einführung der Fahrzeug-zu-Fahrzeug-Kommunikation eine zunehmend wichtige Rolle, um langfristig kooperatives Fahren zu realisieren. Eine Voraussetzung für dessen Umsetzung ist dabei die umfassende Wahrnehmung der aktuellen Fahrumgebung. Jedes Fahrzeug erstellt dafür ein sogenanntes Umfeldmodell, welches Informationen über andere Verkehrsteilnehmer und Objekte beinhaltet. Eine wichtige Datenquelle für dieses Modell sind zum einen lokale Umfeldsensoren, welche implizites Wissen über die aktuelle Fahrumgebung beisteuern. Zum anderen kann dem Umfeldmodell bei einer direkten Kommunikationsverbindung mit anderen Verkehrsteilnehmern auch explizites Wissen hinzugefügt werden. Im Rahmen dieser Arbeit wird ein Konzept zur Realisierung der sogenannten kollektiven Wahrnehmung entwickelt: Hierbei wird Fahrzeugen der Austausch lokaler Sensordaten mit anderen Verkehrsteilnehmern unter Verwendung der Fahrzeug-zu-Fahrzeug-Kommunikation ermöglicht. Somit können zukünftige Fahrerassistenzfunktionen auf ein umfassenderes Umfeldmodell zugreifen. Den im Rahmen der Arbeit durchgeführten Analysen liegt ein fahrzeugbasiertes Ad-hoc Netzwerk zugrunde, welches auf dem europäischen IEEE 802.11p basierten ITS G5 Protokollstapel beruht. Die Effektivität der Technologie fußt hierbei auf der Existenz der sogenannten kritischen Masse: Eine ausreichende Anzahl an Kommunikationspartnern muss zugegen sein, damit der Technologie ein Nutzen zugemessen werden kann. Die Verbreitung der Technologie kann jedoch durch indirekte Effekte unterstützt werden. Die kollektive Wahrnehmung ist ein Repräsentant dieser indirekten Effekte, da die Informationsdichte in dem zwischen den Fahrzeugen bestehenden Netzwerk selbst bei niedrigen Marktausstattungsraten erhöht wird. Im Rahmen der Arbeit wird daher ein neues Nachrichtenformat entwickelt, welches von zwei Perspektiven beeinflusst: Die Sicht der fahrzeugseitigen Assistenzsysteme und deren Datenfusionsalgorithmen beeinflusst die notwendigen Inhalte der Nachricht. Weiterhin werden aus der Netzwerksicht durch Mechanismen wie denen der Lastkontrolle und den bestehenden Nachrichtengrößenbeschränkungen spezifische Anforderungen gestellt. Beide Untersuchungen werden dabei in der Arbeit zur Erstellung eines ganzheitlichen Konzeptes für die kollektive Wahrnehmung verbunden

Multi-Sensor Data Fusion for Travel Time Estimation

The importance of travel time estimation has increased due to the central role it plays in a number of emerging intelligent transport systems and services including Advanced Traveller Information Systems (ATIS), Urban Traffic Control (UTC), Dynamic Route Guidance (DRG), Active Traffic Management (ATM), and network performance monitoring. Along with the emerging of new sensor technologies, the much greater volumes of near real time data provided by these new sensor systems create opportunities for significant improvement in travel time estimation. Data fusion as a recent technique leads to a promising solution to this problem. This thesis presents the development and testing of new methods of multi-sensor data fusion for the accurate, reliable and robust estimation of travel time. This thesis reviews the state-of-art data fusion approaches and its application in transport domain, and discusses both of opportunities and challenging of applying data fusion into travel time estimation in a heterogeneous real time data environment. For a particular England highway scenario where ILDs and ANPR data are largely available, a simple but practical fusion method is proposed to estimate the travel time based on a novel relationship between space-mean-speed and time-mean-speed. In developing a general fusion framework which is able to fuse ILDs, GPS and ANPR data, the Kalman filter is identified as the most appropriate fundamental fusion technique upon which to construct the required framework. This is based both on the ability of the Kalman filter to flexibly accommodate well-established traffic flow models which describe the internal physical relation between the observed variables and objective estimates and on its ability to integrate and propagate in a consistent fashion the uncertainty associated with different data sources. Although the standard linear Kalman filter has been used for multi-sensor travel time estimation in the previous research, the novelty of this research is to develop a nonlinear Kalman filter (EKF and UKF) fusion framework which improves the estimation performance over those methods based on the linear Kalman filter. This proposed framework is validated by both of simulation and real-world scenarios, and is demonstrated the effectiveness of estimating travel time by fusing multi-sensor sources

Fusion-layer-based machine vision for intelligent transportation systems/

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 307-317).Environment understanding technology is very vital for intelligent vehicles that are expected to automatically respond to fast changing environment and dangerous situations. To obtain perception abilities, we should automatically detect static and dynamic obstacles, and obtain their related information, such as, locations, speed, collision/occlusion possibility, and other dynamic current/historic information. Conventional methods independently detect individual information, which is normally noisy and not very reliable. Instead we propose fusion-based and layered-based information-retrieval methodology to systematically detect obstacles and obtain their location/timing information for visible and infrared sequences. The proposed obstacle detection methodologies take advantage of connection between different information and increase the computational accuracy of obstacle information estimation, thus improving environment understanding abilities, and driving safety.by Yajun Fang.Ph.D

Assessing the sustainability performance of inter-urban intelligent transport

The implementation of ITS to increase the efficiency of saturated highways has become increasingly prevalent. It is a high level objective for many international governments and operators that highways should be managed in a way that is both sustainable i.e. environmental, social and economically sound and supportive of a Low-Carbon-Energy Future. Some clarity is therefore needed to understand how Intelligent Transport Systems perform within the constraints of that objective. This thesis describes the development of performance criteria that reflect the contributions of Information Communication Technology (ICT) emissions, vehicle emissions and the embedded carbon within the physical transport infrastructure that typically comprises three types of Intelligent Transport System. Active Traffic Management, Intelligent Speed Adaptation and the Automated Highway System are a collection of systems designed to transform the road network into a highly efficient and congestion free transport solution and all possess varying levels of uncertainty in terms of sustainability performance. The performance criteria form part of a new framework methodology ‘EnvFUSION’ (Environmental Fusion for ITS) outlined here. An attributional LCA and c-LCA (consequential lifecycle assessment) are both undertaken which forms part of a data fusion process using data from various sources. The models forecast improvements for the three ITS technologies in-line with social acceptability, economic profitability and major carbon reduction scenarios up to 2050 on one of the UK's most congested highways. Analytical Hierarchy Process and Dempster-Shafer theory are used to weight criteria which form part of an Intelligent Transport Sustainability Index. Overall performance is then synthesized. Results indicate that there will be a substantial increase in socio-economic and emissions benefits, provided that the policies are in place and targets are reached which would otherwise delay their realisation. To conclude, an integrated strategic performance management framework is proposed which performs socio-technical comparisons of four key performance areas between ITS schemes in order to identify energy and emission hotspots