Young drivers’ pedestrian anti-collision braking operation data modelling for ADAS development

Smart cities and smart mobility come from intelligent systems designed by humans. Artificial Intelligence (AI) is contributing significantly to the development of these systems, and the automotive industry is the most prominent example of "smart" technology entering the market: there are Advanced Driver Assistance System (ADAS), Radar/LIDAR detection units and camera-based Computer Vision systems that can assess driving conditions. Actually, these technologies have become consumer goods and services in mass-produced vehicles to provide human drivers with tools for a more comfortable and safer driving. Nevertheless, they need to be further improved for progress in the transition to fully automated driving or simply to increase vehicle automation levels. To this end, it becomes imperative to accurately predict driver’s decisions, model human driving behaviors, and introduce more accurate risk assessment metrics. This paper presents a system that can learn to predict the future braking behavior of a driver in a typically urban vehicle-pedestrian conflict, i.e., when a pedestrian enters a zebra crossing from the curb and a vehicle is approaching. The algorithm proposes a sequential prediction of relevant operational indicators that continuously describe the encounter process. A car driving simulator was used to collect reliable data on braking behaviours of a cohort of 68 licensed university students, who faced the same urban scenario. The vehicle speed, steering wheel angle, and pedal activity were recorded as the participants approached the crosswalk, along with the azimuth angle of the pedestrian and the relative longitudinal distance between the vehicle and the pedestrian: the proposed system employs the vehicle information as human driving decisions and the pedestrian information as explanatory variables of the environmental state. In fact, the pedestrian’s polar coordinates are usually calculated by an on-board millimeter-wave radar which is typically used to perceive the environment around a vehicle. All mentioned information is represented in the form of time series data and is used to train a recurrent neural network in a supervised machine learning process. The main purpose of this research is to define a system of behavioral profiles in non-collision conditions that could be used for enhancing the existing intelligent driving systems, e.g., to reduce the number of warnings when the driver is not on a collision course with a pedestrian. Preliminary experiments reveal the feasibility of the proposed system

Modeling and Prediction of Driving Behaviors Using a Nonparametric Bayesian Method with AR Models

To develop a new generation advanced driver assistance system that avoids a dangerous condition in advance, we need to predict driving behaviors. Since a nonparametric Bayesian method with a two-level structure successfully predicted the symbolized behaviors only, we applied a nonparametric Bayesian method with linear dynamical systems to predicting the driving behavior. The method called the beta process autoregressive hidden Markov model (BP-AR-HMM) segments driving behaviors into states each of which corresponds to an AR model and it predicts future behaviors using the estimated future state sequence and the dynamical systems therein. Here, the segmentation as well as the parameters of the dynamical systems are determined using given training data in an unsupervised way. We carried out experiments with real driving data and found that the BP-AR-HMM predicted driving behaviors better than other methods

Naturalistic Driver Intention and Path Prediction using Machine Learning

Autonomous vehicles are still yet to be available to the public. This is because there are a number of challenges that have not been overcome to ensure that autonomous vehicles can safely and efficiently drive on public roads. Accurate prediction of other vehicles is vital for safe driving, as interacting with other vehicles is unavoidable on public streets. This thesis explores reasons why this problem of scene understanding is still unsolved, and presents methods for driver intention and path prediction. The thesis focuses on intersections, as this is a very complex scenario in which to predict the actions of human drivers. There is very limited data available for intersection studies from the perspective of an autonomous vehicle. This thesis presents a very large dataset of over 23,000 vehicle trajectories, used to validate the algorithms presented in this thesis. This dataset was collected using a lidar based vehicle detection and tracking system onboard a vehicle. Analytics of this data is presented. To determine the intent of vehicle at an intersection, a method for manoeuvre classification through the use of recurrent neural networks is presented. This allows accurate predictions of which destination a vehicle will take at an unsignalised intersection, based on that vehicle's approach. The final contribution of this thesis presents a method for driver path prediction, based on recurrent neural networks. It produces a multi-modal prediction for the vehicle’s path with uncertainty assigned to each mode. The output modes are not hand labelled, but instead learned from the data. This results in there not being a fixed number of output modes. Whilst the application of this method is vehicle prediction, this method shows significant promise to be used in other areas of robotics

Behavior Prediction at Multiple Time-Scales in Inner-City Scenarios

Garcia Ortiz M, Fritsch J, Kummert F, Gepperth A. Behavior Prediction at Multiple Time-Scales in Inner-City Scenarios. Presented at the Intelligent Vehicles Symposium

Learning Behavior Models for Interpreting and Predicting Traffic Situations

In this thesis, we present Bayesian state estimation and machine learning methods for predicting traffic situations. The cognitive ability to assess situations and behaviors of traffic participants, and to anticipate possible developments is an essential requirement for several applications in the traffic domain, especially for self-driving cars. We present a method for learning behavior models from unlabeled traffic observations and develop improved learning methods for decision trees

Probabilistic, Variable and Interaction-aware Situation Recognition

Future advanced driver assistance systems (ADAS) as well as autonomous driving functions will extend their applicability to more complex highway scenarios and inner-city traffic. For these systems it is a prerequisite to know how an encountered traffic scene is most likely going to evolve. Situation recognition aims to predict the high level behavior patterns traffic participants pursue. Thus, it provides valuable information that helps to predict the next few seconds of a traffic scene. The extension of ADAS and autonomous driving functions to more complex scenarios poses a problem to state-of-the-art situation recognition systems due to the variability of the encountered scene layouts, the presence of multiple interacting traffic participants and the concomitant large number of possible situation classes. This thesis proposes and discusses approaches that tackle these challenges. A novel discriminative maneuver estimation framework provides the possibility to assess traffic scenes with varying layout. It is based on reusable, partial classifiers that are combined online using a technique called pairwise probability coupling. The real-world evaluations indicate that the assembled probabilistic maneuver estimation is able to provide superior classification results. A novel interaction-aware situation recognition framework constructs a probabilistic situation assessment over multiple traffic participants without relying on independence assumptions. It allows to assess each traffic participant individually by using maneuver estimation systems that determine complete conditional distributions. A real-world evaluation outlines its applicability and shows its benefits. The challenges associated with the increasing number of possible situation classes are addressed in two ways. Both frameworks allow to reuse classifiers in different contexts. This reduces the number of models required to cope with a large variety of traffic scenes. Moreover, a situation hypotheses selection scheme provides an efficient way for reducing the number of situation hypotheses. This lowers the computational demands and eases the load on subsequent systems

Fahrerintentionserkennung zur lichtbasierten Kommunikation mit Fußgängern

Im heutigen Straßenverkehr ist neben zahlreichen formellen Regeln stets eine informelle Kommunikation zwischen Verkehrsteilnehmern zu beobachten. Besonders Fußgänger sind auf die Interaktion mit anderen Verkehrsteilnehmern angewiesen und suchen beim Überqueren der Straße Blickkontakt zu Autofahrern. Mit der zunehmenden Automation von Fahrzeugen und dem Einführen automatisierter Systeme wird diese Kommunikation zukünftig entfallen. Um auch automatisierten Fahrsystemen die Möglichkeit zu geben, mit Fußgängern zu kommunizieren, werden unterschiedliche Konzepte zur Fahrzeug-Fußgänger-Kommunikation evaluiert. Die im Rahmen dieser Arbeit durchgeführte Voruntersuchung mit 35 Teilnehmern und Onlinestudie mit 709 Teilnehmern zeigen, dass Zeichen für eine Fahrzeug-Fußgänger-Kommunikation häufig nicht intuitiv sind und sogar bekannte Symbole nur selten richtig gedeutet werden. In der Voruntersuchung ist lediglich ein Symbol intuitiv verständlich. Dieses Zeichen empfiehlt den Fußgängern anzuhalten und die Straße nicht zu überqueren. In der hier durchgeführten Onlinestudie werden zwei von neun untersuchten Symbolen intuitiv erkannt. Diese visualisieren die Nachricht „Vorfahrt gewähren“, während Zeichen für die Darstellung eines automatisierten Fahrmodus und zur Erkennung eines Fußgängers nur mit vorgegebenen Antworten signifikant verständlich sind. Die Onlinestudie zeigt weiter, dass Farben keine signifikante Unterstützung für die Verständlichkeit von Symbolen darstellen. Die einzige Ausnahme stellt dabei die bereits erlernte Farbe Grün für die Nachricht „Vorfahrt gewähren“ dar. Dies verdeutlicht, dass Zeichen zur Fahrzeug-Fußgänger-Kommunikation nicht intuitiv interpretiert werden können und deren Bedeutungen erst gelernt werden müssen. Eine Möglichkeit, Zeichen für eine Fahrzeug-Fußgänger-Kommunikation zu erlernen, ist die Symbole bereits im unassistierten Fahrbetrieb anzuzeigen, damit Fußgänger die bestehende informelle Kommunikation mit der neuartigen Fahrzeugkommunikation verbinden können. Hierfür ist vor allem das Erkennen der Fahrerintention an Fußgängerüberwegen notwendig, weshalb ein dreistufiger Algorithmus entwickelt wird. Dieser Algorithmus besteht aus einem rekurrenten neuronalen Netzwerk zur Prädiktion von fünf Signalen, einem Random Forest zur Interpretation dieser und einer Plausibilisierung bzw. Entscheidung, ob das Fahrzeug im Prädiktionshorizont von 2 s anhalten wird. Für ein durchschnittliches Fahrverhalten können so Richtig-positiv Raten von 94,0 % und Falsch-positiv Raten von 2,8 % erreicht werden. Mit einer zusätzlichen Personalisierung auf fahrer- bzw. fahrzeugspezifische Merkmale ist eine Prädiktion mit einer Richtig-positiv Rate von 95,6 % und Falsch-positiv Rate von 1,9 % möglich. Das Anpassen des Algorithmus erfolgt dabei mittels Transfer Learning. Durch Kombination der angepassten Fahrerintentionserkennung und der entwickelten Fahrzeug-Fußgänger-Kommunikation können bereits heutige Fahrzeuge automatisiert mit Fußgängern kommunizieren. Dadurch können diese Symbole für zukünftige Fahrsysteme erlernt und somit die Akzeptanz der automatisierten Fahrzeuge gesteigert werden. Darüber hinaus wird die Kommunikation der Fahrer mit anderen Verkehrsteilnehmern unterstützt

ANALISI E MODELLAZIONE DELLE INTERAZIONI VEICOLO-PEDONE PER LO SVILUPPO DI SISTEMI ATTIVI DI ASSISTENZA ALLA GUIDA E DI PROTEZIONE DEI PEDONI

La sicurezza e la mobilità dei pedoni sono requisiti basilari che dovrebbero caratterizzare ogni sistema di trasporto urbano. Tuttavia, le morti degli utenti della strada più vulnerabili costituiscono ancora oggi una componente significativa di tutte le vittime della strada nel Mondo. Nonostante gli innumerevoli sforzi compiuti per l’innovazione tecnologica dei veicoli e il riesame degli spazi urbani, le statistiche sull’incidentalità dimostrano la necessità e l’importanza di sviluppare sempre più affidabili sistemi di protezione in grado di diminuire gli impatti sociali ed economici del sistema di trasporto. Sebbene sul mercato di massa siano stati immessi molti sistemi di frenata automatica di emergenza (o AEB, dall’inglese Automatic Emergency Braking), una misura di sicurezza chiave nei veicoli moderni in grado di evitare o mitigare gli effetti di una collisione, diversi ricercatori hanno individuato una nuova strategia per lo sviluppo efficiente di questi sistemi: migliorare la sicurezza dei pedoni nel traffico urbano richiede sistemi “intelligenti” in grado, non solo di comprendere lo stato attuale dell’interazione veicolo-pedone, ma di anticipare proattivamente il futuro modello di rischio dell’evento. In altre parole, prevedere in anticipo le decisioni degli utenti nella scena di traffico, interpretare i comportamenti dei conducenti e definire accurate metriche di valutazione del rischio sono gli obbiettivi da perseguire per raggiungere nuovi traguardi nell’ambito della mobilità sostenibile. Questo elaborato discute la natura globale del problema della sicurezza dei pedoni e i diversi approcci che sono stati sviluppati dai gruppi di ricerca nel Mondo per affrontarlo. Inoltre, la tesi presenta nel dettaglio lo studio, l’implementazione e l’analisi di un innovativo modello di valutazione del rischio, recentemente oggetto di pubblicazione su rivista internazionale, per l’efficientamento dei sistemi di assistenza alla guida esistenti. Il modello proposto, basato su moderne tecniche di Machine Learning e processi di analisi in linea con la letteratura scientifica più recente, è in grado di predire, fino a tre secondi nel futuro, il livello di rischio atteso negli incontri tra veicolo e pedone sulle strisce pedonali in funzione della rappresentazione attuale della scena di traffico tratta da radar e telecamere esterne al veicolo. Infatti, l’algoritmo prototipato fornisce una previsione sequenziale, su più orizzonti temporali, di indicatori di sicurezza operativi che descrivono in continuo il processo di incontro e permettono di annotare le interazioni conflittuali gravi. L’applicazione è stata ottimizzata attraverso dati di mobilità, acquisiti con un simulatore di guida avanzato ad elevato grado di realismo, su un campione di giovani conducenti. Questi ultimi hanno affrontato diversi conflitti veicolo-pedone su un percorso urbano virtuale pianificato. La conoscenza acquisita dal modello in questo contesto potrà essere sfruttata per facilitare l’adattamento online del sistema a nuove situazioni operative e alle diverse caratteristiche comportamentali degli utenti