Pedestrian Trajectory Prediction in Pedestrian-Vehicle Mixed Environments: A Systematic Review

Planning an autonomous vehicle's (AV) path in a space shared with pedestrians requires reasoning about pedestrians' future trajectories. A practical pedestrian trajectory prediction algorithm for the use of AVs needs to consider the effect of the vehicle's interactions with the pedestrians on pedestrians' future motion behaviours. In this regard, this paper systematically reviews different methods proposed in the literature for modelling pedestrian trajectory prediction in presence of vehicles that can be applied for unstructured environments. This paper also investigates specific considerations for pedestrian-vehicle interaction (compared with pedestrian-pedestrian interaction) and reviews how different variables such as prediction uncertainties and behavioural differences are accounted for in the previously proposed prediction models. PRISMA guidelines were followed. Articles that did not consider vehicle and pedestrian interactions or actual trajectories, and articles that only focused on road crossing were excluded. A total of 1260 unique peer-reviewed articles from ACM Digital Library, IEEE Xplore, and Scopus databases were identified in the search. 64 articles were included in the final review as they met the inclusion and exclusion criteria. An overview of datasets containing trajectory data of both pedestrians and vehicles used by the reviewed papers has been provided. Research gaps and directions for future work, such as having more effective definition of interacting agents in deep learning methods and the need for gathering more datasets of mixed traffic in unstructured environments are discussed.Comment: Published in IEEE Transactions on Intelligent Transportation System

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Crowdsensing-driven route optimisation algorithms for smart urban mobility

Urban rörlighet anses ofta vara en av de främsta möjliggörarna för en hållbar statsutveckling. Idag skulle det dock kräva ett betydande skifte mot renare och effektivare stadstransporter vilket skulle stödja ökad social och ekonomisk koncentration av resurser i städerna. En viktig prioritet för städer runt om i världen är att stödja medborgarnas rörlighet inom stadsmiljöer medan samtidigt minska trafikstockningar, olyckor och föroreningar. Att utveckla en effektivare och grönare (eller med ett ord; smartare) stadsrörlighet är en av de svåraste problemen att bemöta för stora metropoler. I denna avhandling närmar vi oss problemet från det snabba utvecklingsperspektivet av ITlandskapet i städer vilket möjliggör byggandet av rörlighetslösningar utan stora stora investeringar eller sofistikerad sensortenkik. I synnerhet föreslår vi utnyttjandet av den mobila rörlighetsavkännings, eng. Mobile Crowdsensing (MCS), paradigmen i vilken befolkningen exploaterar sin mobilkommunikation och/eller mobilasensorer med syftet att frivilligt samla, distribuera, lokalt processera och analysera geospecifik information. Rörlighetavkänningssdata (t.ex. händelser, trafikintensitet, buller och luftföroreningar etc.) inhämtad från frivilliga i befolkningen kan ge värdefull information om aktuella rörelsesförhållanden i stad vilka, med adekvata databehandlingsalgoriter, kan användas för att planera människors rörelseflöden inom stadsmiljön. Såtillvida kombineras i denna avhandling två mycket lovande smarta rörlighetsmöjliggörare, eng. Smart Mobility Enablers, nämligen MCS och rese/ruttplanering. Vi kan därmed till viss utsträckning sammanföra forskningsutmaningar från dessa två delar. Vi väljer att separera våra forskningsmål i två delar, dvs forskningssteg: (1) arkitektoniska utmaningar vid design av MCS-system och (2) algoritmiska utmaningar för tillämpningar av MCS-driven ruttplanering. Vi ämnar att visa en logisk forskningsprogression över tiden, med avstamp i mänskligt dirigerade rörelseavkänningssystem som MCS och ett avslut i automatiserade ruttoptimeringsalgoritmer skräddarsydda för specifika MCS-applikationer. Även om vi förlitar oss på heuristiska lösningar och algoritmer för NP-svåra ruttproblem förlitar vi oss på äkta applikationer med syftet att visa på fördelarna med algoritm- och infrastrukturförslagen.La movilidad urbana es considerada una de las principales desencadenantes de un desarrollo urbano sostenible. Sin embargo, hoy en día se requiere una transición hacia un transporte urbano más limpio y más eficiente que soporte una concentración de recursos sociales y económicos cada vez mayor en las ciudades. Una de las principales prioridades para las ciudades de todo el mundo es facilitar la movilidad de los ciudadanos dentro de los entornos urbanos, al mismo tiempo que se reduce la congestión, los accidentes y la contaminación. Sin embargo, desarrollar una movilidad urbana más eficiente y más verde (o en una palabra, más inteligente) es uno de los temas más difíciles de afrontar para las grandes áreas metropolitanas. En esta tesis, abordamos este problema desde la perspectiva de un panorama TIC en rápida evolución que nos permite construir movilidad sin la necesidad de grandes inversiones ni sofisticadas tecnologías de sensores. En particular, proponemos aprovechar el paradigma Mobile Crowdsensing (MCS) en el que los ciudadanos utilizan sus teléfonos móviles y dispositivos, para nosotros recopilar, procesar y analizar localmente información georreferenciada, distribuida voluntariamente. Los datos de movilidad recopilados de ciudadanos que voluntariamente quieren compartirlos (por ejemplo, eventos, intensidad del tráfico, ruido y contaminación del aire, etc.) pueden proporcionar información valiosa sobre las condiciones de movilidad actuales en la ciudad, que con el algoritmo de procesamiento de datos adecuado, pueden utilizarse para enrutar y gestionar el flujo de gente en entornos urbanos. Por lo tanto, en esta tesis combinamos dos prometedoras fuentes de movilidad inteligente: MCS y la planificación de viajes/rutas, uniendo en cierta medida los distintos desafíos de investigación. Hemos dividido nuestros objetivos de investigación en dos etapas: (1) Desafíos arquitectónicos en el diseño de sistemas MCS y (2) Desafíos algorítmicos en la planificación de rutas aprovechando la información del MCS. Nuestro objetivo es demostrar una progresión lógica de la investigación a lo largo del tiempo, comenzando desde los fundamentos de los sistemas de detección centrados en personas, como el MCS, hasta los algoritmos de optimización de rutas diseñados específicamente para la aplicación de estos. Si bien nos centramos en algoritmos y heurísticas para resolver problemas de enrutamiento de clase NP-hard, utilizamos ejemplos de aplicaciones en el mundo real para mostrar las ventajas de los algoritmos e infraestructuras propuestas

Spatial disaggregation in transport modelling -- Modelling Europe with more than 100,000 travel zones

European transport policy pursues several targets, such as reducing CO2 emissions and improving the efficiency of transport systems. In order to investigate the effectiveness as well as the welfare impacts of potential policy measures, transport models are applied in policy consulting. However, the travel zones used in recent transport models operating at European scale are often too large, mainly due to complexity and data availability. These models can provide only limited insights into regional traffic flows. In this thesis, an innovative transport modelling approach called HIPAT is introduced. HIPAT is based on disaggregated and homogeneous travel zones, and thus facilitates the modelling of European traffic flows, including long-distance, regional and short-distance passenger trips with a single, consistent transport model. This enables European policy makers to assess also regional welfare impacts when prioritising, for instance, investments in the trans-European transport network. The quantum leap from 1,500 travel zones at NUTS-3 level to more than 100,000 at LAU-2 level, while simultaneously reducing the model runtime from several days to one hour, can be facilitated by solving the trip distribution problem very efficiently at different spatial levels. In comparison to recent European transport models, the consistency of the overall modelling approach was also improved, by integrating the trip distribution, the modal split and the network assignment models. Within the last nine years most parts of this thesis were researched in the course of the author’s involvement as a transport modelling and data specialist in the two European research projects ETISplus and HIGH-TOOL. In a first step, a modelling database was established covering regional indicators at NUTS-3 level and harmonised mobility indicators from travel surveys at country level. In a second step, the currently applied four-step transport modelling approach was intensively revised and the IPAT passenger transport model was developed. Its methodology has successfully been validated in the course of publishing the IPAT model with the HIGH-TOOL policy assessment model that was awarded the German Mobility Prize in 2017. In a third step, the introduced modelling database was disaggregated from NUTS-3 to LAU-2 level for an important long-distance road transport corridor between Paris and Budapest (the ‘’Magistrale’’). The compilation of this database was a key precondition in order to realise a prototype implementation of the HIPAT approach. The model operates on about 33,000 travel zones and has a runtime of two minutes. It was intensively tested at NUTS-2, NUTS-3 and LAU-2 levels and the results clearly demonstrate the advantages of smaller travel zones. Solely LAU-2 level enables the modelling of regional and short-distance traffic flows. Hence, the implemented HIPAT model provides, for the first time, a sound basis for the assessment of regional welfare impacts of European transport policies. In a next step, the scope of this model should be increased to cover the whole of Europe, thus encompassing more than 100,000 travel zones

Long-term future prediction under uncertainty and multi-modality

Humans have an innate ability to excel at activities that involve prediction of complex object dynamics such as predicting the possible trajectory of a billiard ball after it has been hit by the player or the prediction of motion of pedestrians while on the road. A key feature that enables humans to perform such tasks is anticipation. There has been continuous research in the area of Computer Vision and Artificial Intelligence to mimic this human ability for autonomous agents to succeed in the real world scenarios. Recent advances in the field of deep learning and the availability of large scale datasets has enabled the pursuit of fully autonomous agents with complex decision making abilities such as self-driving vehicles or robots. One of the main challenges encompassing the deployment of these agents in the real world is their ability to perform anticipation tasks with at least human level efficiency. To advance the field of autonomous systems, particularly, self-driving agents, in this thesis, we focus on the task of future prediction in diverse real world settings, ranging from deterministic scenarios such as prediction of paths of balls on a billiard table to the predicting the future of non-deterministic street scenes. Specifically, we identify certain core challenges for long-term future prediction: long-term prediction, uncertainty, multi-modality, and exact inference. To address these challenges, this thesis makes the following core contributions. Firstly, for accurate long-term predictions, we develop approaches that effectively utilize available observed information in the form of image boundaries in videos or interactions in street scenes. Secondly, as uncertainty increases into the future in case of non-deterministic scenarios, we leverage Bayesian inference frameworks to capture calibrated distributions of likely future events. Finally, to further improve performance in highly-multimodal non-deterministic scenarios such as street scenes, we develop deep generative models based on conditional variational autoencoders as well as normalizing flow based exact inference methods. Furthermore, we introduce a novel dataset with dense pedestrian-vehicle interactions to further aid the development of anticipation methods for autonomous driving applications in urban environments.Menschen haben die angeborene Fähigkeit, Vorgänge mit komplexer Objektdynamik vorauszusehen, wie z. B. die Vorhersage der möglichen Flugbahn einer Billardkugel, nachdem sie vom Spieler gestoßen wurde, oder die Vorhersage der Bewegung von Fußgängern auf der Straße. Eine Schlüsseleigenschaft, die es dem Menschen ermöglicht, solche Aufgaben zu erfüllen, ist die Antizipation. Im Bereich der Computer Vision und der Künstlichen Intelligenz wurde kontinuierlich daran geforscht, diese menschliche Fähigkeit nachzuahmen, damit autonome Agenten in der realen Welt erfolgreich sein können. Jüngste Fortschritte auf dem Gebiet des Deep Learning und die Verfügbarkeit großer Datensätze haben die Entwicklung vollständig autonomer Agenten mit komplexen Entscheidungsfähigkeiten wie selbstfahrende Fahrzeugen oder Roboter ermöglicht. Eine der größten Herausforderungen beim Einsatz dieser Agenten in der realen Welt ist ihre Fähigkeit, Antizipationsaufgaben mit einer Effizienz durchzuführen, die mindestens der menschlichen entspricht. Um das Feld der autonomen Systeme, insbesondere der selbstfahrenden Agenten, voranzubringen, konzentrieren wir uns in dieser Arbeit auf die Aufgabe der Zukunftsvorhersage in verschiedenen realen Umgebungen, die von deterministischen Szenarien wie der Vorhersage der Bahnen von Kugeln auf einem Billardtisch bis zur Vorhersage der Zukunft von nicht-deterministischen Straßenszenen reichen. Insbesondere identifizieren wir bestimmte grundlegende Herausforderungen für langfristige Zukunftsvorhersagen: Langzeitvorhersage, Unsicherheit, Multimodalität und exakte Inferenz. Um diese Herausforderungen anzugehen, leistet diese Arbeit die folgenden grundlegenden Beiträge. Erstens: Für genaue Langzeitvorhersagen entwickeln wir Ansätze, die verfügbare Beobachtungsinformationen in Form von Bildgrenzen in Videos oder Interaktionen in Straßenszenen effektiv nutzen. Zweitens: Da die Unsicherheit in der Zukunft bei nicht-deterministischen Szenarien zunimmt, nutzen wir Bayes’sche Inferenzverfahren, um kalibrierte Verteilungen wahrscheinlicher zukünftiger Ereignisse zu erfassen. Drittens: Um die Leistung in hochmultimodalen, nichtdeterministischen Szenarien wie Straßenszenen weiter zu verbessern, entwickeln wir tiefe generative Modelle, die sowohl auf konditionalen Variations-Autoencodern als auch auf normalisierenden fließenden exakten Inferenzmethoden basieren. Darüber hinaus stellen wir einen neuartigen Datensatz mit dichten Fußgänger-Fahrzeug- Interaktionen vor, um Antizipationsmethoden für autonome Fahranwendungen in urbanen Umgebungen weiter zu entwickeln

Simultaneous Prediction and Planning in Crowds using Learnt Models of Social Response

The ability of autonomous mobile robots to work alongside humans and animals in real world environments has the potential to revolutionise the way in which many routine and labour intensive tasks are completed. Whilst we are seeing increasing applications in controlled environments, such as traffic and warehousing, robots are still far from ubiquitous in everyday life. In unstructured environments, such as agriculture or pedestrian crowds, where interactions between agents are not guided by infrastructure, there exist additional challenges that need to be overcome before we are likely to see the widespread adoption of mobile robots. Safe navigation in shared environments requires the accurate perception of nearby individuals using a robot's on board sensors. Additionally, the future motion of detected individuals needs to be predicted both for collision avoidance and efficient navigation. These predictions should reflect the inherent uncertainty of the individual's future, including the ways in which an individual might respond to its neighbours, including the robot itself. As such, there exists a dependency between any prediction of an individual's motion and the planned path of the robot, which needs to be accounted for both during the prediction and planning stages of navigation. This thesis focuses on how prediction and planning can be approached in a single framework to address this dependency, using learnt models of social response within a sampling based path planner for simultaneous prediction and planning (SPP). Additional challenges faced in navigating shared and unstructured environments are also addressed, including predicting the uncertain branching and multi-modal nature of agent motion during social interactions, and overcoming the on-board limitations of mobile robots --- such as resource and sensing constraints --- in order to achieve extended autonomy

Detecting Non-Line of Sight to Prevent Accidents in Vehicular Ad hoc Networks

There are still many challenges in the field of VANETs that encouraged researchers to conduct further investigation in this field to meet these challenges. The issue pertaining to routing protocols such as delivering the warning messages to the vehicles facing Non-Line of Sight (NLOS) situations without causing the storm problem and channel contention, is regarded as a serious dilemma which is required to be tackled in VANET, especially in congested environments. This requires the designing of an efficient mechanism of routing protocol that can broadcast the warning messages from the emergency vehicles to the vehicles under NLOS, reducing the overhead and increasing the packet delivery ratio with a reduced time delay and channel utilisation. The main aim of this work is to develop the novel routing protocol for a high-density environment in VANET through utilisation of its high mobility features, aid of the sensors such as Global Positioning System (GPS) and Navigation System (NS). In this work, the cooperative approach has been used to develop the routing protocol called the Co-operative Volunteer Protocol (CVP), which uses volunteer vehicles to disseminate the warning message from the source to the target vehicle under NLOS issue; this also increases the packet delivery ratio, detection of NLOS and resolution of NLOS by delivering the warning message successfully to the vehicle under NLOS, thereby causing a direct impact on the reduction of collisions between vehicles in normal mode and emergency mode on the road near intersections or on highways. The cooperative approach adopted for warning message dissemination reduced the rebroadcast rate of messages, thereby decreasing significantly the storm issue and the channel contention. A novel architecture has been developed by utilising the concept of a Context-Aware System (CAS), which clarifies the OBU components and their interaction with each other in order to collect data and take the decisions based on the sensed circumstances. The proposed architecture has been divided into three main phases: sensing, processing and acting. The results obtained from the validation of the proposed CVP protocol using the simulator EstiNet under specific conditions and parameters showed that performance of the proposed protocol is better than that of the GRANT protocol with regard to several metrics such as packet delivery ratio, neighbourhood awareness, channel utilisation, overhead and latency. It is also successfully shown that the proposed CVP could detect the NLOS situation and solves it effectively and efficiently for both the intersection scenario in urban areas and the highway scenario

AN ADAPTIVE INFORMATION DISSEMINATION MODEL FOR VANET COMMUNICATION

Vehicular ad hoc networks (VANETs) have been envisioned to be useful in road safety and many commercial applications. The growing trend to provide communication among the vehicles on the road has provided the opportunities for developing a variety of applications for VANET. The unique characteristics of VANET bring about new research challenges