Predicting pedestrian crossing intentions using contextual information

El entorno urbano es uno de los escenarios m as complejos para un veh culo aut onomo, ya que lo comparte con otros tipos de usuarios conocidos como usuarios vulnerables de la carretera, con los peatones como mayor representante. Estos usuarios se caracterizan por su gran dinamicidad. A pesar del gran n umero de interacciones entre veh culos y peatones, la seguridad de estos ultimos no ha aumentado al mismo ritmo que la de los ocupantes de los veh culos. Por esta raz on, es necesario abordar este problema. Una posible estrategia estar a basada en conseguir que los veh culos anticipen el comportamiento de los peatones para minimizar situaciones de riesgo, especialmente presentes en el momento de cruce. El objetivo de esta tesis doctoral es alcanzar dicha anticipaci on mediante el desarrollo de t ecnicas de predicci on de la acci on de cruce de peatones basadas en aprendizaje profundo. Previo al dise~no e implementaci on de los sistemas de predicci on, se ha desarrollado un sistema de clasi caci on con el objetivo de discernir a los peatones involucrados en la escena vial. El sistema, basado en redes neuronales convolucionales, ha sido entrenado y validado con un conjunto de datos personalizado. Dicho conjunto se ha construido a partir de varios conjuntos existentes y aumentado mediante la inclusi on de im agenes obtenidas de internet. Este paso previo a la anticipaci on permitir a reducir el procesamiento innecesario dentro del sistema de percepci on del veh culo. Tras este paso, se han desarrollado dos sistemas como propuesta para abordar el problema de predicci on. El primer sistema, basado en redes convolucionales y recurrentes, obtiene una predicci on a corto plazo de la acci on de cruce realizada un segundo en el futuro. La informaci on de entrada al modelo est a basada principalmente en imagen, que permite aportar contexto adicional del peat on. Adem as, el uso de otras variables relacionadas con el peat on junto con mejoras en la arquitectura, permiten mejorar considerablemente los resultados en el conjunto de datos JAAD. El segundo sistema se basa en una arquitectura end-to-end basado en la combinaci on de redes neuronales convolucionales tridimensionales y/o el codi cador de la arquitectura Transformer. En este modelo, a diferencia del anterior, la mayor a de las mejoras est an centradas en transformaciones de los datos de entrada. Tras analizar dichas mejoras, una serie de modelos se han evaluado y comparado con otros m etodos utilizando tanto el conjunto de datos JAAD como PIE. Los resultados obtenidos han conseguido liderar el estado del arte, validando la arquitectura propuesta.The urban environment is one of the most complex scenarios for an autonomous vehicle, as it is shared with other types of users known as vulnerable road users, with pedestrians as their principal representative. These users are characterized by their great dynamicity. Despite a large number of interactions between vehicles and pedestrians, the safety of pedestrians has not increased at the same rate as that of vehicle occupants. For this reason, it is necessary to address this problem. One possible strategy would be anticipating pedestrian behavior to minimize risky situations, especially during the crossing. The objective of this doctoral thesis is to achieve such anticipation through the development of crosswalk action prediction techniques based on deep learning. Before the design and implementation of the prediction systems, a classi cation system has been developed to discern the pedestrians involved in the road scene. The system, based on convolutional neural networks, has been trained and validated with a customized dataset. This set has been built from several existing sets and augmented by including images obtained from the Internet. This pre-anticipation step would reduce unnecessary processing within the vehicle perception system. After this step, two systems have been developed as a proposal to solve the prediction problem. The rst system is composed of convolutional and recurrent encoder networks. It obtains a short-term prediction of the crossing action performed one second in the future. The input information to the model is mainly image-based, which provides additional pedestrian context. In addition, the use of pedestrian-related variables and architectural improvements allows better results on the JAAD dataset. The second system is an end-to-end architecture based on the combination of threedimensional convolutional neural networks and/or the Transformer architecture encoder. In this model, most of the proposed and investigated improvements are focused on transformations of the input data. After an extensive set of individual tests, several models have been trained, evaluated, and compared with other methods using both JAAD and PIE datasets. Obtained results are among the best state-of-the-art models, validating the proposed architecture

Intent prediction of vulnerable road users for trusted autonomous vehicles

This study investigated how future autonomous vehicles could be further trusted by vulnerable road users (such as pedestrians and cyclists) that they would be interacting with in urban traffic environments. It focused on understanding the behaviours of such road users on a deeper level by predicting their future intentions based solely on vehicle-based sensors and AI techniques. The findings showed that personal/body language attributes of vulnerable road users besides their past motion trajectories and physics attributes in the environment led to more accurate predictions about their intended actions

Modeling, Predicting and Capturing Human Mobility

Realistic models of human mobility are critical for modern day applications, specifically for recommendation systems, resource planning and process optimization domains. Given the rapid proliferation of mobile devices equipped with Internet connectivity and GPS functionality today, aggregating large sums of individual geolocation data is feasible. The thesis focuses on methodologies to facilitate data-driven mobility modeling by drawing parallels between the inherent nature of mobility trajectories, statistical physics and information theory. On the applied side, the thesis contributions lie in leveraging the formulated mobility models to construct prediction workflows by adopting a privacy-by-design perspective. This enables end users to derive utility from location-based services while preserving their location privacy. Finally, the thesis presents several approaches to generate large-scale synthetic mobility datasets by applying machine learning approaches to facilitate experimental reproducibility

Vulnerable road users and connected autonomous vehicles interaction: a survey

There is a group of users within the vehicular traffic ecosystem known as Vulnerable Road Users (VRUs). VRUs include pedestrians, cyclists, motorcyclists, among others. On the other hand, connected autonomous vehicles (CAVs) are a set of technologies that combines, on the one hand, communication technologies to stay always ubiquitous connected, and on the other hand, automated technologies to assist or replace the human driver during the driving process. Autonomous vehicles are being visualized as a viable alternative to solve road accidents providing a general safe environment for all the users on the road specifically to the most vulnerable. One of the problems facing autonomous vehicles is to generate mechanisms that facilitate their integration not only within the mobility environment, but also into the road society in a safe and efficient way. In this paper, we analyze and discuss how this integration can take place, reviewing the work that has been developed in recent years in each of the stages of the vehicle-human interaction, analyzing the challenges of vulnerable users and proposing solutions that contribute to solving these challenges.This work was partially funded by the Ministry of Economy, Industry, and Competitiveness of Spain under Grant: Supervision of drone fleet and optimization of commercial operations flight plans, PID2020-116377RB-C21.Peer ReviewedPostprint (published version

Conditional Behavior Prediction of Interacting Agents on Map Graphs with Neural Networks

Solange Verkehrsteilnehmer ihre Manöverabsicht und ihre geplante Trajektorie automatischen Fahrzeugen nicht mitteilen können, ist eine Verhaltensvorhersage für alle beteiligten Verkehrsteilnehmer erforderlich. Mit einer solchen Vorhersage kann das Verhalten eines automatischen Fahrzeugs vorausschauend generiert und damit komfortabler und energieeffizienter gemacht werden, was den Verkehrsfluss verbessert. Es wird ein künstliches neuronales Netz für Graphen (GNN) vorgestellt, das verschiedene probabilistische Positionsvorhersagen für interagierende Agenten zur Analyse bereitstellt. Das vorliegende Anwendungsbeispiel ist die Verkehrssituationsanalyse für das automatische Fahren, für welches ein diskretisierter Vorhersagezeitraum von einigen Sekunden als relevant angesehen wird. Das GNN propagiert einen vollvernetzten, gerichteten Agentengraphen probabilistisch durch einen dünnvernetzten, gerichteten Kartengraphen. Merkmale des Agentengraphen, der aus Verkehrsteilnehmern und deren Beziehungen besteht, sowie Merkmale des Kartengraphen, der aus Fahrbahnstücken und deren geometrischer, sowie verkehrsregelbezogenen Verbindungen besteht, können für die Vorhersage verwertet werden. Das Modell prädiziert für jeden Agenten zu jedem Prädiktionszeitpunkt eine diskrete Aufenthaltswahrscheinlichkeitsverteilung über alle Fahrbahnstücke des Kartengraphen. Eine solche Prädiktion ist in der wissenschaftlichen Literatur zwar üblich, setzt aber für deren stochastische Interpretierbarkeit und damit Anwendbarkeit statistische Unabhängigkeit des zukünftigen Verhaltens der Verkehrsteilnehmer voraus. Da diese Annahme bei interagierenden Agenten als unzulässig erachtet wird, prädiziert das Modell darüber hinaus für alle Agentenpaare diskrete Verbundwahrscheinlichkeitsverteilungen. Aus diesen können bedingte Prädiktionen gegeben möglicher zukünftiger Positionen einer der beiden Agenten berechnet werden. In der Evaluierung werden gängige Metriken für den vorliegenden Fall angepasst und verschiedene Modellierungstiefen einander gegenübergestellt. Sowohl die individuelle Prädiktion als auch die bedingte Prädiktion werden erfolgreich auf Genauigkeit und statistischer Zuverlässigkeit untersucht