Intelligent Roundabout Insertion using Deep Reinforcement Learning

An important topic in the autonomous driving research is the development of maneuver planning systems. Vehicles have to interact and negotiate with each other so that optimal choices, in terms of time and safety, are taken. For this purpose, we present a maneuver planning module able to negotiate the entering in busy roundabouts. The proposed module is based on a neural network trained to predict when and how entering the roundabout throughout the whole duration of the maneuver. Our model is trained with a novel implementation of A3C, which we will call Delayed A3C (D-A3C), in a synthetic environment where vehicles move in a realistic manner with interaction capabilities. In addition, the system is trained such that agents feature a unique tunable behavior, emulating real world scenarios where drivers have their own driving styles. Similarly, the maneuver can be performed using different aggressiveness levels, which is particularly useful to manage busy scenarios where conservative rule-based policies would result in undefined waits

From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning

Deep Reinforcement Learning has proved to be able to solve many control tasks in different fields, but the behavior of these systems is not always as expected when deployed in real-world scenarios. This is mainly due to the lack of domain adaptation between simulated and real-world data together with the absence of distinction between train and test datasets. In this work, we investigate these problems in the autonomous driving field, especially for a maneuver planning module for roundabout insertions. In particular, we present a system based on multiple environments in which agents are trained simultaneously, evaluating the behavior of the model in different scenarios. Finally, we analyze techniques aimed at reducing the gap between simulated and real-world data showing that this increased the generalization capabilities of the system both on unseen and real-world scenarios.Comment: Intelligent Vehicle Symposium 2020 (IV2020

CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.Comment: WWW 2019 Demo Pape

Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning

In this article, we demonstrate a zero-shot transfer of an autonomous driving policy from simulation to University of Delaware's scaled smart city with adversarial multi-agent reinforcement learning, in which an adversary attempts to decrease the net reward by perturbing both the inputs and outputs of the autonomous vehicles during training. We train the autonomous vehicles to coordinate with each other while crossing a roundabout in the presence of an adversary in simulation. The adversarial policy successfully reproduces the simulated behavior and incidentally outperforms, in terms of travel time, both a human-driving baseline and adversary-free trained policies. Finally, we demonstrate that the addition of adversarial training considerably improves the performance \eat{stability and robustness} of the policies after transfer to the real world compared to Gaussian noise injection.Comment: 6 pages, 4 figure

Purpose in the Machine: Do Traffic Simulators Produce Distributionally Equivalent Outcomes for Reinforcement Learning Applications?

Traffic simulators are used to generate data for learning in intelligent transportation systems (ITSs). A key question is to what extent their modelling assumptions affect the capabilities of ITSs to adapt to various scenarios when deployed in the real world. This work focuses on two simulators commonly used to train reinforcement learning (RL) agents for traffic applications, CityFlow and SUMO. A controlled virtual experiment varying driver behavior and simulation scale finds evidence against distributional equivalence in RL-relevant measures from these simulators, with the root mean squared error and KL divergence being significantly greater than 0 for all assessed measures. While granular real-world validation generally remains infeasible, these findings suggest that traffic simulators are not a deus ex machina for RL training: understanding the impacts of inter-simulator differences is necessary to train and deploy RL-based ITSs.Comment: 12 pages; accepted version, published at the 2023 Winter Simulation Conference (WSC '23

Navigating roundabouts and unprotected turns in autonomous driving

© 2024 IEEE. This is the accepted manuscript version of an article which has been published in final form at https://doi.org/10.1109/TFR.2024.3421389The development of a fully autonomous driving vehicle (AV) requires various traffic situations to be handled efficiently. One of the most common driving manoeuvres which an AV experiences in daily traffic is giving way (yielding) to other traffic participants. In this paper, we propose a simple yet efficient method of yielding that doesn’t query yielding areas of interests from map API making it hassle free to use without having to rely on digitized yielding areas. We incorporated our method into one of the well-known open-source autonomy stacks called Autoware. The proposed method makes use of high-definition (HD) map elements including lanes and stoplines for filtering vehicles which participate in yielding decision making. Our method estimates future collisions of filtered vehicles of interest with AV’s planned trajectory and outputs a binary yielding decision for ego vehicle. Our method covers different yielding areas including a roundabout and an unprotected turn. We tested and evaluated the decision making of our method on various simulated scenarios and afterwards successful real-world tests were conducted using an in-house AV. An in-depth analysis of our approach shows that the proposed yielding solution works reasonably well i.e. 87% successful yielding area navigation ratio on real data.Peer reviewe

Scenario-Transferable Semantic Graph Reasoning for Interaction-Aware Probabilistic Prediction

Accurately predicting the possible behaviors of traffic participants is an essential capability for autonomous vehicles. Since autonomous vehicles need to navigate in dynamically changing environments, they are expected to make accurate predictions regardless of where they are and what driving circumstances they encountered. A number of methodologies have been proposed to solve prediction problems under different traffic situations. However, these works either focus on one particular driving scenario (e.g. highway, intersection, or roundabout) or do not take sufficient environment information (e.g. road topology, traffic rules, and surrounding agents) into account. In fact, the limitation to certain scenario is mainly due to the lackness of generic representations of the environment. The insufficiency of environment information further limits the flexibility and transferability of the predictor. In this paper, we propose a scenario-transferable and interaction-aware probabilistic prediction algorithm based on semantic graph reasoning. We first introduce generic representations for both static and dynamic elements in driving environments. Then these representations are utilized to describe semantic goals for selected agents and incorporate them into spatial-temporal structures. Finally, we reason internal relations among these structured semantic representations using learning-based method and obtain prediction results. The proposed algorithm is thoroughly examined under several complicated real-world driving scenarios to demonstrate its flexibility and transferability, where the predictor can be directly used under unforeseen driving circumstances with different static and dynamic information.Comment: 17 pages, 11 figure

Proceedings of the 4th Symposium on Management of Future Motorway and Urban Traffic Systems 2022

The 4th Symposium on Management of Future Motorway and Urban Traffic Systems (MFTS) was held in Dresden, Germany, from November 30th to December 2nd, 2022. Organized by the Chair of Traffic Process Automation (VPA) at the “Friedrich List” Faculty of Transport and Traffic Sciences of the TU Dresden, the proceedings of this conference are published as volume 9 in the Chair’s publication series “Verkehrstelematik” and contain a large part of the presented conference extended abstracts. The focus of the MFTS conference 2022 was cooperative management of multimodal transport and reflected the vision of the professorship to be an internationally recognized group in ITS research and education with the goal of optimizing the operation of multimodal transport systems. In 14 MFTS sessions, current topics in demand and traffic management, traffic control in conventional, connected and automated transport, connected and autonomous vehicles, traffic flow modeling and simulation, new and shared mobility systems, digitization, and user behavior and safety were discussed. In addition, special sessions were organized, for example on “Human aspects in traffic modeling and simulation” and “Lesson learned from Covid19 pandemic”, whose descriptions and analyses are also included in these proceedings.:1 Connected and Automated Vehicles 1.1 Traffic-based Control of Truck Platoons on Freeways 1.2 A Lateral Positioning Strategy for Connected and Automated Vehicles in Lane-free Traffic 1.3 Simulation Methods for Mixed Legacy-Autonomous Mainline Train Operations 1.4 Can Dedicated Lanes for Automated Vehicles on Urban Roads Improve Traffic Efficiency? 1.5 GLOSA System with Uncertain Green and Red Signal Phases 2 New Mobility Systems 2.1 A New Model for Electric Vehicle Mobility and Energy Consumption in Urban Traffic Networks 2.2 Shared Autonomous Vehicles Implementation for a Disrupted Public Transport Network 3 Traffic Flow and Simulation 3.1 Multi-vehicle Stochastic Fundamental Diagram Consistent with Transportations Systems Theory 3.2 A RoundD-like Roundabout Scenario in CARLA Simulator 3.3 Multimodal Performance Evaluation of Urban Traffic Control: A Microscopic Simulation Study 3.4 A MILP Framework to Solve the Sustainable System Optimum with Link MFD Functions 3.5 On How Traffic Signals Impact the Fundamental Diagrams of Urban Roads 4 Traffic Control in Conventional Traffic 4.1 Data-driven Methods for Identifying Travel Conditions Based on Traffic and Weather Characteristics 4.2 AI-based Multi-class Traffic Model Oriented to Freeway Traffic Control 4.3 Exploiting Deep Learning and Traffic Models for Freeway Traffic Estimation 4.4 Automatic Design of Optimal Actuated Traffic Signal Control with Transit Signal Priority 4.5 A Deep Reinforcement Learning Approach for Dynamic Traffic Light Control with Transit Signal Priority 4.6 Towards Efficient Incident Detection in Real-time Traffic Management 4.7 Dynamic Cycle Time in Traffic Signal of Cyclic Max-Pressure Control 5 Traffic Control with Autonomous Vehicles 5.1 Distributed Ordering and Optimization for Intersection Management with Connected and Automated Vehicles 5.2 Prioritization of an Automated Shuttle for V2X Public Transport at a Signalized Intersection – a Real-life Demonstration 6 User Behaviour and Safety 6.1 Local Traffic Safety Analyzer (LTSA) - Improved Road Safety and Optimized Signal Control for Future Urban Intersections 7 Demand and Traffic Management 7.1 A Stochastic Programming Method for OD Estimation Using LBSN Check-in Data 7.2 Delineation of Traffic Analysis Zone for Public Transportation OD Matrix Estimation Based on Socio-spatial Practices 8 Workshops 8.1 How to Integrate Human Aspects Into Engineering Science of Transport and Traffic? - a Workshop Report about Discussions on Social Contextualization of Mobility 8.2 Learning from Covid: How Can we Predict Mobility Behaviour in the Face of Disruptive Events? – How to Investigate the Mobility of the FutureDas 4. Symposium zum Management zukünftiger Autobahn- und Stadtverkehrssysteme (MFTS) fand vom 30. November bis 2. Dezember 2022 in Dresden statt und wurde vom Lehrstuhl für Verkehrsprozessautomatisierung (VPA) an der Fakultät Verkehrswissenschaften„Friedrich List“ der TU Dresden organisiert. Der Tagungsband erscheint als Band 9 in der Schriftenreihe „Verkehrstelematik“ des Lehrstuhls und enthält einen Großteil der vorgestellten Extended-Abstracts des Symposiums. Der Schwerpunkt des MFTS-Symposiums 2022 lag auf dem kooperativen Management multimodalen Verkehrs und spiegelte die Vision der Professur wider, eine international anerkannte Gruppe in der ITS-Forschung und -Ausbildung mit dem Ziel der Optimierung des Betriebs multimodaler Transportsysteme zu sein. In 14 MFTS-Sitzungen wurden aktuelle Themen aus den Bereichen Nachfrage- und Verkehrsmanagement, Verkehrssteuerung im konventionellen, vernetzten und automatisierten Verkehr, vernetzte und autonome Fahrzeuge, Verkehrsflussmodellierung und -simulation, neue und geteilte Mobilitätssysteme, Digitalisierung sowie Nutzerverhalten und Sicherheit diskutiert. Darüber hinaus wurden Sondersitzungen organisiert, beispielsweise zu „Menschlichen Aspekten bei der Verkehrsmodellierung und -simulation“ und „Lektionen aus der Covid-19-Pandemie“, deren Beschreibungen und Analysen ebenfalls in diesen Tagungsband einfließen.:1 Connected and Automated Vehicles 1.1 Traffic-based Control of Truck Platoons on Freeways 1.2 A Lateral Positioning Strategy for Connected and Automated Vehicles in Lane-free Traffic 1.3 Simulation Methods for Mixed Legacy-Autonomous Mainline Train Operations 1.4 Can Dedicated Lanes for Automated Vehicles on Urban Roads Improve Traffic Efficiency? 1.5 GLOSA System with Uncertain Green and Red Signal Phases 2 New Mobility Systems 2.1 A New Model for Electric Vehicle Mobility and Energy Consumption in Urban Traffic Networks 2.2 Shared Autonomous Vehicles Implementation for a Disrupted Public Transport Network 3 Traffic Flow and Simulation 3.1 Multi-vehicle Stochastic Fundamental Diagram Consistent with Transportations Systems Theory 3.2 A RoundD-like Roundabout Scenario in CARLA Simulator 3.3 Multimodal Performance Evaluation of Urban Traffic Control: A Microscopic Simulation Study 3.4 A MILP Framework to Solve the Sustainable System Optimum with Link MFD Functions 3.5 On How Traffic Signals Impact the Fundamental Diagrams of Urban Roads 4 Traffic Control in Conventional Traffic 4.1 Data-driven Methods for Identifying Travel Conditions Based on Traffic and Weather Characteristics 4.2 AI-based Multi-class Traffic Model Oriented to Freeway Traffic Control 4.3 Exploiting Deep Learning and Traffic Models for Freeway Traffic Estimation 4.4 Automatic Design of Optimal Actuated Traffic Signal Control with Transit Signal Priority 4.5 A Deep Reinforcement Learning Approach for Dynamic Traffic Light Control with Transit Signal Priority 4.6 Towards Efficient Incident Detection in Real-time Traffic Management 4.7 Dynamic Cycle Time in Traffic Signal of Cyclic Max-Pressure Control 5 Traffic Control with Autonomous Vehicles 5.1 Distributed Ordering and Optimization for Intersection Management with Connected and Automated Vehicles 5.2 Prioritization of an Automated Shuttle for V2X Public Transport at a Signalized Intersection – a Real-life Demonstration 6 User Behaviour and Safety 6.1 Local Traffic Safety Analyzer (LTSA) - Improved Road Safety and Optimized Signal Control for Future Urban Intersections 7 Demand and Traffic Management 7.1 A Stochastic Programming Method for OD Estimation Using LBSN Check-in Data 7.2 Delineation of Traffic Analysis Zone for Public Transportation OD Matrix Estimation Based on Socio-spatial Practices 8 Workshops 8.1 How to Integrate Human Aspects Into Engineering Science of Transport and Traffic? - a Workshop Report about Discussions on Social Contextualization of Mobility 8.2 Learning from Covid: How Can we Predict Mobility Behaviour in the Face of Disruptive Events? – How to Investigate the Mobility of the Futur