Personalized Maneuver Prediction at Intersections

Losing V, Hammer B, Wersing H. Personalized Maneuver Prediction at Intersections. Presented at the IEEE Intelligent Transportation Systems Conference 2017, Yokohama

Personalized Maneuver Prediction at Intersections

Losing V, Hammer B, Wersing H. Personalized Maneuver Prediction at Intersections. Presented at the IEEE Intelligent Transportation Systems Conference 2017, Yokohama

Learning to Segment and Represent Motion Primitives from Driving Data for Motion Planning Applications

Developing an intelligent vehicle which can perform human-like actions requires the ability to learn basic driving skills from a large amount of naturalistic driving data. The algorithms will become efficient if we could decompose the complex driving tasks into motion primitives which represent the elementary compositions of driving skills. Therefore, the purpose of this paper is to segment unlabeled trajectory data into a library of motion primitives. By applying a probabilistic inference based on an iterative Expectation-Maximization algorithm, our method segments the collected trajectories while learning a set of motion primitives represented by the dynamic movement primitives. The proposed method utilizes the mutual dependencies between the segmentation and representation of motion primitives and the driving-specific based initial segmentation. By utilizing this mutual dependency and the initial condition, this paper presents how we can enhance the performance of both the segmentation and the motion primitive library establishment. We also evaluate the applicability of the primitive representation method to imitation learning and motion planning algorithms. The model is trained and validated by using the driving data collected from the Beijing Institute of Technology intelligent vehicle platform. The results show that the proposed approach can find the proper segmentation and establish the motion primitive library simultaneously

Driver Digital Twin for Online Prediction of Personalized Lane Change Behavior

Connected and automated vehicles (CAVs) are supposed to share the road with human-driven vehicles (HDVs) in a foreseeable future. Therefore, considering the mixed traffic environment is more pragmatic, as the well-planned operation of CAVs may be interrupted by HDVs. In the circumstance that human behaviors have significant impacts, CAVs need to understand HDV behaviors to make safe actions. In this study, we develop a Driver Digital Twin (DDT) for the online prediction of personalized lane change behavior, allowing CAVs to predict surrounding vehicles' behaviors with the help of the digital twin technology. DDT is deployed on a vehicle-edge-cloud architecture, where the cloud server models the driver behavior for each HDV based on the historical naturalistic driving data, while the edge server processes the real-time data from each driver with his/her digital twin on the cloud to predict the lane change maneuver. The proposed system is first evaluated on a human-in-the-loop co-simulation platform, and then in a field implementation with three passenger vehicles connected through the 4G/LTE cellular network. The lane change intention can be recognized in 6 seconds on average before the vehicle crosses the lane separation line, and the Mean Euclidean Distance between the predicted trajectory and GPS ground truth is 1.03 meters within a 4-second prediction window. Compared to the general model, using a personalized model can improve prediction accuracy by 27.8%. The demonstration video of the proposed system can be watched at https://youtu.be/5cbsabgIOdM

End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners

For human drivers, having rear and side-view mirrors is vital for safe driving. They deliver a more complete view of what is happening around the car. Human drivers also heavily exploit their mental map for navigation. Nonetheless, several methods have been published that learn driving models with only a front-facing camera and without a route planner. This lack of information renders the self-driving task quite intractable. We investigate the problem in a more realistic setting, which consists of a surround-view camera system with eight cameras, a route planner, and a CAN bus reader. In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and low-level driving maneuvers (e.g. steering angle and speed) by human drivers. With such a sensor setup we collect a new driving dataset, covering diverse driving scenarios and varying weather/illumination conditions. Finally, we learn a novel driving model by integrating information from the surround-view cameras and the route planner. Two route planners are exploited: 1) by representing the planned routes on OpenStreetMap as a stack of GPS coordinates, and 2) by rendering the planned routes on TomTom Go Mobile and recording the progression into a video. Our experiments show that: 1) 360-degree surround-view cameras help avoid failures made with a single front-view camera, in particular for city driving and intersection scenarios; and 2) route planners help the driving task significantly, especially for steering angle prediction.Comment: to be published at ECCV 201

Fahrerverhaltensvorhersage an Kreisverkehren

Roundabouts are considered important because converting an intersection into a roundabout has been proven to improve safety. However, the absolute number of crashes at roundabouts is still high. Many crashes occur because car drivers fail to yield. Intelligent systems can increase safety if they can prevent crashes by precisely predicting driver maneuvers. Therefore, a reliable and trustworthy predictive model of driver maneuvers is needed. A few studies analyze human behavior at roundabouts. However, they focus on an operational timescale rather than on maneuvers on a tactical timescale. Tactical maneuvers have mostly been investigated in scenarios about typical intersections and overtaking. Thus, there is still a lack of research on driver maneuver prediction at roundabouts. To fill this gap, the objective of this thesis is to develop a model that can predict driver maneuvers at single-lane roundabouts. Two types of driver maneuvers are possible in front of each exit of a roundabout: exiting the roundabout and staying in the roundabout. To predict which maneuver a driver will execute in front of an exit, a driver maneuver predictive model was developed on the basis of an analysis of driver behavior data acquired from a field study and a simulator study. Soft-classification algorithms were proposed to train the predictive model. The model consisted of four sub-models for four different scenarios, which were defined by the correlation between roundabout layouts and drivers' steering behavior. The sub-models make it possible to predict the exiting or staying maneuvers executed in the corresponding scenarios. Furthermore, a personalized predictive model was developed to adapt to individual drivers because different drivers have different driving styles. The driver maneuver predictive model shows excellent predictability: In the scenarios without traffic, the model reported prediction results for more than 97.60% of test drives at the position 10 m from the exits of the roundabouts. Of these drives, more than 97.10% were predicted correctly. The personalized predictive model provided even better prediction results for individual drivers with significantly different driving styles. Moreover, the driver maneuver predictive model also successfully predicts drivers' maneuvers in most scenarios with cyclists. The prediction results show that steering angle, steering angle speed, velocity, and acceleration of the ego car provide the most important information. With this information, the model can predict the maneuver of a driver with any type of driving style at a single-lane roundabout with any type of layout.Kreisverkehre gelten als ein wichtiger Bestandteil der Verkehrsinfrastruktur, da ihre Verwendung anstelle von traditionellen Kreuzungen einen wesentlichen Beitrag zur Verkehrssicherheit leistet. Die absolute Anzahl von Unfällen bleibt jedoch auch an Kreisverkehren noch hoch. Viele Kollisionen werden dabei durch Missachtung der Vorfahrt verursacht. Intelligente Fahrzeugassistenzsysteme könnten hier eingreifen, vorausgesetzt sie verfügen über eine zuverlässige Vorhersage des Fahrerverhaltens. Hierfür wird ein robustes und präzises Modell für die Vorhersage von Fahrmanövern im Kreisverkehr benötigt. Empirische Studien zu menschlichem Verhalten an Kreisverkehren fokussieren in der Regel auf die operationale Ebene der Fahraufgabe, also auf eine zeitlich hoch aufgelöste Zeitskala. Die taktische Ebene, auf der Manöver wie "Verlassen des Kreisverkehr" stattfinden, wurde dabei jedoch nicht ausreichend analysiert. Insbesondere fehlen Modelle, die Fahrmanöver im Kreisverkehr vorhersagen. Ziel dieser Arbeit ist es daher, ein solches Modell für einspurige Kreisverkehre zu entwickeln. Zwei Arten von Manövern sind innerhalb eines einspurigen Kreisverkehrs möglich: Im Kreisel zu bleiben, oder ihn zu verlassen. Um möglichst früh eines der beiden Manöver vorherzusagen wurden im Rahmen dieser Arbeit verschiedene Modelle entwickelt, welche auf Fahrdaten aus dem Realverkehr sowie Simulationsstudien basieren. Für das Training der jeweiligen Modelle werden Soft-Klassifikationsalgorithmen vorgeschlagen, die auf einem Quasi-Hidden-Markov-Modell basieren. Dieses Modell besteht aus vier Teilmodellen für jeweils vier verschiedene Szenarien, die durch die Korrelation zwischen Kreisverkehrlayouts und Lenkverhalten von Fahrern definiert wurden. Mit den Teilmodellen können die in den entsprechenden Szenarien ausgeführten Manöver "Verlassen" oder "Bleiben" vorhergesagt werden. Des Weiteren wurde ein personalisiertes Vorhersagemodell entwickelt, um sich an den individuellen Fahrer anzupassen, da verschiedene Fahrer unterschiedliche Fahrstile aufweisen. Das Fahrmanöver-Vorhersagemodell zeigt eine ausgezeichnete Performanz: In den Szenarien ohne Verkehr lieferte das Modell in einem Abstand von 10 m vor der Kreisverkehrsausfahrt Vorhersagen für mindestens 97,60% aller Testfahrten. Von diesen Fahrten wurden wiederum über 97,10% korrekt vorhergesagt. Personalisierte Modelle erreichen noch bessere Vorhersageergebnisse. Sind weitere Verkehrsteilnehmer in den analysierten Szenarien anwesend liegt die Vorhersagegüte etwas darunter. Die Ergebnisse zeigen, dass Lenkwinkel, Lenkwinkelgeschwindigkeit sowie Eigengeschwindigkeit und -beschleunigung die wichtigsten Informationen liefern. Hiermit kann das Modell das Manöver eines Fahrers mit jeder Art von Fahrstil an einem Kreisverkehr mit jeder Art von Layout vorhersagen

Interaction-Aware Personalized Vehicle Trajectory Prediction Using Temporal Graph Neural Networks

Accurate prediction of vehicle trajectories is vital for advanced driver assistance systems and autonomous vehicles. Existing methods mainly rely on generic trajectory predictions derived from large datasets, overlooking the personalized driving patterns of individual drivers. To address this gap, we propose an approach for interaction-aware personalized vehicle trajectory prediction that incorporates temporal graph neural networks. Our method utilizes Graph Convolution Networks (GCN) and Long Short-Term Memory (LSTM) to model the spatio-temporal interactions between target vehicles and their surrounding traffic. To personalize the predictions, we establish a pipeline that leverages transfer learning: the model is initially pre-trained on a large-scale trajectory dataset and then fine-tuned for each driver using their specific driving data. We employ human-in-the-loop simulation to collect personalized naturalistic driving trajectories and corresponding surrounding vehicle trajectories. Experimental results demonstrate the superior performance of our personalized GCN-LSTM model, particularly for longer prediction horizons, compared to its generic counterpart. Moreover, the personalized model outperforms individual models created without pre-training, emphasizing the significance of pre-training on a large dataset to avoid overfitting. By incorporating personalization, our approach enhances trajectory prediction accuracy