Localization and analysis of critical areas in urban scenarios

This paper presents an application of a pedestrian detection system aimed at localizing potentially dangerous situations in specific urban scenarios. The approach used in this work differs from the one implemented in traditional pedestrian detection systems, which are designed to localize all pedestrians appearing in the area in front of the vehicle. This application first locates critical areas in the urban environment, and then it searches for pedestrians in these areas only. The environment is reconstructed with a standard laser scanner system, while the following check for the presence of pedestrians is performed thanks to the fusion with a vision system. The great advantages of such an approach are that pedestrian recognition is performed on a very limited image area -therefore boosting its timing performance- and no assessment on the danger level is finally required before providing the result to either the driver or an on-board computer for automatic manoeuvres

Environment-Detection-and-Mapping Algorithm for Autonomous Driving in Rural or Off-Road Environment

Abstract—This paper presents an environment-detection-and-mapping algorithm for autonomous driving that is provided in real time and for both rural and off-road environments. Environment-detection-and-mapping algorithms have been de-signed to consist of two parts: 1) lane, pedestrian-crossing, and speed-bump detection algorithms using cameras and 2) obstacle detection algorithm using LIDARs. The lane detection algorithm returns lane positions using one camera and the vision module “VisLab Embedded Lane Detector (VELD), ” and the pedestrian-crossing and speed-bump detection algorithms return the position of pedestrian crossings and speed bumps. The obstacle detection algorithm organizes data from LIDARs and generates a local obstacle position map. The designed algorithms have been im-plemented on a passenger car using six LIDARs, three cameras, and real-time devices, including personal computers (PCs). Vehicle tests have been conducted, and test results have shown that the vehicle can reach the desired goal with the proposed algorithm. Index Terms—Autonomous driving, lane detection, obstacle de-tection, pedestrian-crossing detection, speed-bump detection. I

Environment perception based on LIDAR sensors for real road applications

The recent developments in applications that have been designed to increase road safety require reliable and trustworthy sensors. Keeping this in mind, the most up-to-date research in the field of automotive technologies has shown that LIDARs are a very reliable sensor family. In this paper, a new approach to road obstacle classification is proposed and tested. Two different LIDAR sensors are compared by focusing on their main characteristics with respect to road applications. The viability of these sensors in real applications has been tested, where the results of this analysis are presented.The recent developments in applications that have been designed to increase road safety require reliable and trustworthy sensors. Keeping this in mind, the most up-to-date research in the field of automotive technologies has shown that LIDARs are a very reliable sensor family. In this paper, a new approach to road obstacle classification is proposed and tested. Two different LIDAR sensors are compared by focusing on their main characteristics with respect to road applications. The viability of these sensors in real applications has been tested, where the results of this analysis are presented.The work reported in this paper has been partly funded by the Spanish Ministry of Science and Innovation (TRA2007- 67786-C02-01, TRA2007-67786-C02-02, and TRA2009- 07505) and the CAM project SEGVAUTO-II.Publicad

Identifying and Tracking Pedestrians Based on Sensor Fusion and Motion Stability Predictions

The lack of trustworthy sensors makes development of Advanced Driver Assistance System (ADAS) applications a tough task. It is necessary to develop intelligent systems by combining reliable sensors and real-time algorithms to send the proper, accurate messages to the drivers. In this article, an application to detect and predict the movement of pedestrians in order to prevent an imminent collision has been developed and tested under real conditions. The proposed application, first, accurately measures the position of obstacles using a two-sensor hybrid fusion approach: a stereo camera vision system and a laser scanner. Second, it correctly identifies pedestrians using intelligent algorithms based on polylines and pattern recognition related to leg positions (laser subsystem) and dense disparity maps and u-v disparity (vision subsystem). Third, it uses statistical validation gates and confidence regions to track the pedestrian within the detection zones of the sensors and predict their position in the upcoming frames. The intelligent sensor application has been experimentally tested with success while tracking pedestrians that cross and move in zigzag fashion in front of a vehicle

Perception and intelligent localization for autonomous driving

Mestrado em Engenharia de Computadores e TelemáticaVisão por computador e fusão sensorial são temas relativamente recentes, no entanto largamente adoptados no desenvolvimento de robôs autónomos que exigem adaptabilidade ao seu ambiente envolvente. Esta dissertação foca-se numa abordagem a estes dois temas para alcançar percepção no contexto de condução autónoma. O uso de câmaras para atingir este fim é um processo bastante complexo. Ao contrário dos meios sensoriais clássicos que fornecem sempre o mesmo tipo de informação precisa e atingida de forma determinística, as sucessivas imagens adquiridas por uma câmara estão repletas da mais variada informação e toda esta ambígua e extremamente difícil de extrair. A utilização de câmaras como meio sensorial em robótica é o mais próximo que chegamos na semelhança com aquele que é o de maior importância no processo de percepção humana, o sistema de visão. Visão por computador é uma disciplina científica que engloba àreas como: processamento de sinal, inteligência artificial, matemática, teoria de controlo, neurobiologia e física. A plataforma de suporte ao estudo desenvolvido no âmbito desta dissertação é o ROTA (RObô Triciclo Autónomo) e todos os elementos que consistem o seu ambiente. No contexto deste, são descritas abordagens que foram introduzidas com fim de desenvolver soluções para todos os desafios que o robô enfrenta no seu ambiente: detecção de linhas de estrada e consequente percepção desta, detecção de obstáculos, semáforos, zona da passadeira e zona de obras. É também descrito um sistema de calibração e aplicação da remoção da perspectiva da imagem, desenvolvido de modo a mapear os elementos percepcionados em distâncias reais. Em consequência do sistema de percepção, é ainda abordado o desenvolvimento de auto-localização integrado numa arquitectura distribuída incluindo navegação com planeamento inteligente. Todo o trabalho desenvolvido no decurso da dissertação é essencialmente centrado no desenvolvimento de percepção robótica no contexto de condução autónoma.Computer vision and sensor fusion are subjects that are quite recent, however widely adopted in the development of autonomous robots that require adaptability to their surrounding environment. This thesis gives an approach on both in order to achieve perception in the scope of autonomous driving. The use of cameras to achieve this goal is a rather complex subject. Unlike the classic sensorial devices that provide the same type of information with precision and achieve this in a deterministic way, the successive images acquired by a camera are replete with the most varied information, that this ambiguous and extremely dificult to extract. The use of cameras for robotic sensing is the closest we got within the similarities with what is of most importance in the process of human perception, the vision system. Computer vision is a scientific discipline that encompasses areas such as signal processing, artificial intelligence, mathematics, control theory, neurobiology and physics. The support platform in which the study within this thesis was developed, includes ROTA (RObô Triciclo Autónomo) and all elements comprising its environment. In its context, are described approaches that introduced in the platform in order to develop solutions for all the challenges facing the robot in its environment: detection of lane markings and its consequent perception, obstacle detection, trafic lights, crosswalk and road maintenance area. It is also described a calibration system and implementation for the removal of the image perspective, developed in order to map the elements perceived in actual real world distances. As a result of the perception system development, it is also addressed self-localization integrated in a distributed architecture that allows navigation with long term planning. All the work developed in the course of this work is essentially focused on robotic perception in the context of autonomous driving

Advances in vision-based lane detection: algorithms, integration, assessment, and perspectives on ACP-based parallel vision

Lane detection is a fundamental aspect of most current advanced driver assistance systems (ADASs). A large number of existing results focus on the study of vision-based lane detection methods due to the extensive knowledge background and the low-cost of camera devices. In this paper, previous vision-based lane detection studies are reviewed in terms of three aspects, which are lane detection algorithms, integration, and evaluation methods. Next, considering the inevitable limitations that exist in the camera-based lane detection system, the system integration methodologies for constructing more robust detection systems are reviewed and analyzed. The integration methods are further divided into three levels, namely, algorithm, system, and sensor. Algorithm level combines different lane detection algorithms while system level integrates other object detection systems to comprehensively detect lane positions. Sensor level uses multi-modal sensors to build a robust lane recognition system. In view of the complexity of evaluating the detection system, and the lack of common evaluation procedure and uniform metrics in past studies, the existing evaluation methods and metrics are analyzed and classified to propose a better evaluation of the lane detection system. Next, a comparison of representative studies is performed. Finally, a discussion on the limitations of current lane detection systems and the future developing trends toward an Artificial Society, Computational experiment-based parallel lane detection framework is proposed

Real-Time GPS-Alternative Navigation Using Commodity Hardware

Modern navigation systems can use the Global Positioning System (GPS) to accurately determine position with precision in some cases bordering on millimeters. Unfortunately, GPS technology is susceptible to jamming, interception, and unavailability indoors or underground. There are several navigation techniques that can be used to navigate during times of GPS unavailability, but there are very few that result in GPS-level precision. One method of achieving high precision navigation without GPS is to fuse data obtained from multiple sensors. This thesis explores the fusion of imaging and inertial sensors and implements them in a real-time system that mimics human navigation. In addition, programmable graphics processing unit technology is leveraged to perform stream-based image processing using a computer\u27s video card. The resulting system can perform complex mathematical computations in a fraction of the time those same operations would take on a CPU-based platform. The resulting system is an adaptable, portable, inexpensive and self-contained software and hardware platform, which paves the way for advances in autonomous navigation, mobile cartography, and artificial intelligence

Data fusion architecture for intelligent vehicles

Traffic accidents are an important socio-economic problem. Every year, the cost in human lives and the economic consequences are inestimable. During the latest years, efforts to reduce or mitigate this problem have lead to a reduction in casualties. But, the death toll in road accidents is still a problem, which means that there is still much work to be done. Recent advances in information technology have lead to more complex applications, which have the ability to help or even substitute the driver in case of hazardous situations, allowing more secure and efficient driving. But these complex systems require more trustable and accurate sensing technology that allows detecting and identifying the surrounding environment as well as identifying the different objects and users. However, the sensing technology available nowadays is insufficient itself, and thus combining the different available technologies is mandatory in order to fulfill the exigent requirements of safety road applications. In this way, the limitations of every system are overcome. More dependable and reliable information can be thus obtained. These kinds of applications are called Data Fusion (DF) applications. The present document tries to provide a solution for the Data Fusion problem in the Intelligent Transport System (ITS) field by providing a set of techniques and algorithms that allow the combination of information from different sensors. By combining these sensors the basic performances of the classical approaches in ITS can be enhanced, satisfying the demands of safety applications. The works presented are related with two researching fields. Intelligent Transport System is the researching field where this thesis was established. ITS tries to use the recent advances in Information Technology to increase the security and efficiency of the transport systems. Data Fusion techniques, on the other hand, try to give solution to the process related with the combination of information from different sources, enhancing the basic capacities of the systems and adding trustability to the inferences. This work attempts to use the Data Fusion algorithms and techniques to provide solution to classic ITS applications. The sensors used in the present application include a laser scanner and computer vision. First is a well known sensor, widely used, and during more recent years have started to be applied in different ITS applications, showing advanced performance mainly related to its trustability. Second is a recent sensor in automotive applications widely used in all recent ITS advances in the last decade. Thanks to computer vision road security applications (e.g. traffic sign detection, driver monitoring, lane detection, pedestrian detection, etc.) advancements are becoming possible. The present thesis tries to solve the environment reconstruction problem, identifying users of the roads (i.e. pedestrians and vehicles) by the use of Data Fusion techniques. The solution delivers a complete level based solution to the Data Fusion problem. It provides different tools for detecting as well as estimates the degree of danger that involve any detection. Presented algorithms represents a step forward in the ITS world, providing novel Data Fusion based algorithms that allow the detection and estimation of movement of pedestrians and vehicles in a robust and trustable way. To perform such a demanding task other information sources were needed: GPS, inertial systems and context information. Finally, it is important to remark that in the frame of the present thesis, the lack of detection and identification techniques based in radar laser resulted in the need to research and provide more innovative approaches, based in the use of laser scanner, able to detect and identify the different actors involved in the road environment. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Los accidentes de tráfico son un grave problema social y económico, cada año el coste tanto en vidas humanas como económico es incontable, por lo que cualquier acción que conlleve la reducción o eliminación de esta lacra es importante. Durante los últimos años se han hecho avances para mitigar el número de accidentes y reducir sus consecuencias. Estos esfuerzos han dado sus frutos, reduciendo el número de accidentes y sus víctimas. Sin embargo el número de heridos y muertos en accidentes de este tipo es aún muy alto, por lo que no hay que rebajar los esfuerzos encaminados a hacer desaparecer tan importante problema. Los recientes avances en tecnologías de la información han permitido la creación de sistemas de ayuda a la conducción cada vez más complejos, capaces de ayudar e incluso sustituir al conductor, permitiendo una conducción más segura y eficiente. Pero estos complejos sistemas requieren de los sensores más fiables, capaces de permitir reconstruir el entorno, identificar los distintos objetos que se encuentran en él e identificar los potenciales peligros. Los sensores disponibles en la actualidad han demostrado ser insuficientes para tan ardua tarea, debido a los enormes requerimientos que conlleva una aplicación de seguridad en carretera. Por lo tanto, combinar los diferentes sensores disponibles se antoja necesario para llegar a los niveles de eficiencia y confianza que requieren este tipo de aplicaciones. De esta forma, las limitaciones de cada sensor pueden ser superadas, gracias al uso combinado de los diferentes sensores, cada uno de ellos proporcionando información que complementa la obtenida por otros sistemas. Este tipo de aplicaciones se denomina aplicaciones de Fusión Sensorial. El presente trabajo busca aportar soluciones en el entorno de los vehículos inteligentes, mediante técnicas de fusión sensorial, a clásicos problemas relacionados con la seguridad vial. Se buscará combinar diferentes sensores y otras fuentes de información, para obtener un sistema fiable, capaz de satisfacer las exigentes demandas de este tipo de aplicaciones. Los estudios realizados y algoritmos propuestos están enmarcados en dos campos de investigación bien conocidos y populares. Los Sistemas Inteligentes de Transporte (ITS- por sus siglas en ingles- Intelligent Transportation Systems), marco en el que se centra la presente tesis, que engloba las diferentes tecnologías que durante los últimos años han permitido dotar a los sistemas de transporte de mejoras que aumentan la seguridad y eficiencia de los sistemas de transporte tradicionales, gracias a las novedades en el campo de las tecnologías de la información. Por otro lado las técnicas de Fusión Sensorial (DF -por sus siglas en ingles- Data Fusión) engloban las diferentes técnicas y procesos necesarios para combinar diferentes fuentes de información, permitiendo mejorar las prestaciones y dando fiabilidad a los sistemas finales. La presente tesis buscará el empleo de las técnicas de Fusión Sensorial para dar solución a problemas relacionados con Sistemas Inteligentes de Transporte. Los sensores escogidos para esta aplicación son un escáner láser y visión por computador. El primero es un sensor ampliamente conocido, que durante los últimos años ha comenzado a emplearse en el mundo de los ITS con unos excelentes resultados. El segundo de este conjunto de sensores es uno de los sistemas más empleados durante los últimos años, para dotar de cada vez más complejos y versátiles aplicaciones en el mundo de los ITS. Gracias a la visión por computador, aplicaciones tan necesarias para la seguridad como detección de señales de tráfico, líneas de la carreta, peatones, etcétera, que hace unos años parecía ciencia ficción, están cada vez más cerca. La aplicación que se presenta pretende dar solución al problema de reconstrucción de entornos viales, identificando a los principales usuarios de la carretera (vehículos y peatones) mediante técnicas de Fusión Sensorial. La solución implementada busca dar una completa solución a todos los niveles del proceso de fusión sensorial, proveyendo de las diferentes herramientas, no solo para detectar los otros usuarios, sino para dar una estimación del peligro que cada una de estas detecciones implica. Para lograr este propósito, además de los sensores ya comentados han sido necesarias otras fuentes de información, como sensores GPS, inerciales e información contextual. Los algoritmos presentados pretenden ser un importante paso adelante en el mundo de los Sistemas Inteligentes de Transporte, proporcionando novedosos algoritmos basados en tecnologías de Fusión Sensorial que permitirán detectar y estimar el movimiento de los peatones y vehículos de forma fiable y robusta. Finalmente hay que remarcar que en el marco de la presente tesis, la falta de sistemas de detección e identificación de obstáculos basados en radar láser provocó la necesidad de implementar novedosos algoritmos que detectasen e identificasen, en la medida de lo posible y pese a las limitaciones de la tecnología, los diferentes obstáculos que se pueden encontrar en la carretera basándose en este sensor

Real-time simulator of collaborative and autonomous vehicles

Durant ces dernières décennies, l’apparition des systèmes d’aide à la conduite a essentiellement été favorisée par le développement des différentes technologies ainsi que par celui des outils mathématiques associés. Cela a profondément affecté les systèmes de transport et a donné naissance au domaine des systèmes de transport intelligents (STI). Nous assistons de nos jours au développement du marché des véhicules intelligents dotés de systèmes d’aide à la conduite et de moyens de communication inter-véhiculaire. Les véhicules et les infrastructures intelligents changeront le mode de conduite sur les routes. Ils pourront résoudre une grande partie des problèmes engendrés par le trafic routier comme les accidents, les embouteillages, la pollution, etc. Cependant, le bon fonctionnement et la fiabilité des nouvelles générations des systèmes de transport nécessitent une parfaite maitrise des différents processus de leur conception, en particulier en ce qui concerne les systèmes embarqués. Il est clair que l’identification et la correction des défauts des systèmes embarqués sont deux tâches primordiales à la fois pour la sauvegarde de la vie humaine, à la fois pour la préservation de l’intégrité des véhicules et des infrastructures urbaines. Pour ce faire, la simulation numérique en temps réel est la démarche la plus adéquate pour tester et valider les systèmes de conduite et les véhicules intelligents. Elle présente de nombreux avantages qui la rendent incontournable pour la conception des systèmes embarqués. Par conséquent, dans ce projet, nous présentons une nouvelle plateforme de simulation temps-réel des véhicules intelligents et autonomes en conduite collaborative. Le projet se base sur deux principaux composants. Le premier étant les produits d’OPAL-RT Technologies notamment le logiciel RT-LAB « en : Real Time LABoratory », l’application Orchestra et les machines de simulation dédiées à la simulation en temps réel et aux calculs parallèles, le second composant est Pro-SiVIC pour la simulation de la dynamique des véhicules, du comportement des capteurs embarqués et de l’infrastructure. Cette nouvelle plateforme (Pro-SiVIC/RT-LAB) permettra notamment de tester les systèmes embarqués (capteurs, actionneurs, algorithmes), ainsi que les moyens de communication inter-véhiculaire. Elle permettra aussi d’identifier et de corriger les problèmes et les erreurs logicielles, et enfin de valider les systèmes embarqués avant même le prototypage

Stereo vision for obstacle detection in autonomous vehicle navigation

Master'sMASTER OF ENGINEERIN