354 research outputs found
Human Motion Trajectory Prediction: A Survey
With growing numbers of intelligent autonomous systems in human environments,
the ability of such systems to perceive, understand and anticipate human
behavior becomes increasingly important. Specifically, predicting future
positions of dynamic agents and planning considering such predictions are key
tasks for self-driving vehicles, service robots and advanced surveillance
systems. This paper provides a survey of human motion trajectory prediction. We
review, analyze and structure a large selection of work from different
communities and propose a taxonomy that categorizes existing methods based on
the motion modeling approach and level of contextual information used. We
provide an overview of the existing datasets and performance metrics. We
discuss limitations of the state of the art and outline directions for further
research.Comment: Submitted to the International Journal of Robotics Research (IJRR),
37 page
Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review
Behaviour prediction function of an autonomous vehicle predicts the future
states of the nearby vehicles based on the current and past observations of the
surrounding environment. This helps enhance their awareness of the imminent
hazards. However, conventional behaviour prediction solutions are applicable in
simple driving scenarios that require short prediction horizons. Most recently,
deep learning-based approaches have become popular due to their superior
performance in more complex environments compared to the conventional
approaches. Motivated by this increased popularity, we provide a comprehensive
review of the state-of-the-art of deep learning-based approaches for vehicle
behaviour prediction in this paper. We firstly give an overview of the generic
problem of vehicle behaviour prediction and discuss its challenges, followed by
classification and review of the most recent deep learning-based solutions
based on three criteria: input representation, output type, and prediction
method. The paper also discusses the performance of several well-known
solutions, identifies the research gaps in the literature and outlines
potential new research directions
Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-based Convolutional Neural Networks
Lane change (LC) is one of the safety-critical manoeuvres in highway driving
according to various road accident records. Thus, reliably predicting such
manoeuvre in advance is critical for the safe and comfortable operation of
automated driving systems. The majority of previous studies rely on detecting a
manoeuvre that has been already started, rather than predicting the manoeuvre
in advance. Furthermore, most of the previous works do not estimate the key
timings of the manoeuvre (e.g., crossing time), which can actually yield more
useful information for the decision making in the ego vehicle. To address these
shortcomings, this paper proposes a novel multi-task model to simultaneously
estimate the likelihood of LC manoeuvres and the time-to-lane-change (TTLC). In
both tasks, an attention-based convolutional neural network (CNN) is used as a
shared feature extractor from a bird's eye view representation of the driving
environment. The spatial attention used in the CNN model improves the feature
extraction process by focusing on the most relevant areas of the surrounding
environment. In addition, two novel curriculum learning schemes are employed to
train the proposed approach. The extensive evaluation and comparative analysis
of the proposed method in existing benchmark datasets show that the proposed
method outperforms state-of-the-art LC prediction models, particularly
considering long-term prediction performance.Comment: 13 pages, 11 figure
Deep learning-based vehicle behaviour prediction for autonomous driving applications : a review
Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behavior prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their promising performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behavior prediction in this article. We firstly give an overview of the generic problem of vehicle behavior prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The article also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions
MixNet: Structured Deep Neural Motion Prediction for Autonomous Racing
Reliably predicting the motion of contestant vehicles surrounding an
autonomous racecar is crucial for effective and performant planning. Although
highly expressive, deep neural networks are black-box models, making their
usage challenging in safety-critical applications, such as autonomous driving.
In this paper, we introduce a structured way of forecasting the movement of
opposing racecars with deep neural networks. The resulting set of possible
output trajectories is constrained. Hence quality guarantees about the
prediction can be given. We report the performance of the model by evaluating
it together with an LSTM-based encoder-decoder architecture on data acquired
from high-fidelity Hardware-in-the-Loop simulations. The proposed approach
outperforms the baseline regarding the prediction accuracy but still fulfills
the quality guarantees. Thus, a robust real-world application of the model is
proven. The presented model was deployed on the racecar of the Technical
University of Munich for the Indy Autonomous Challenge 2021. The code used in
this research is available as open-source software at
www.github.com/TUMFTM/MixNet
PFL-LSTR: A privacy-preserving framework for driver intention inference based on in-vehicle and out-vehicle information
Intelligent vehicle anticipation of the movement intentions of other drivers
can reduce collisions. Typically, when a human driver of another vehicle
(referred to as the target vehicle) engages in specific behaviors such as
checking the rearview mirror prior to lane change, a valuable clue is therein
provided on the intentions of the target vehicle's driver. Furthermore, the
target driver's intentions can be influenced and shaped by their driving
environment. For example, if the target vehicle is too close to a leading
vehicle, it may renege the lane change decision. On the other hand, a following
vehicle in the target lane is too close to the target vehicle could lead to its
reversal of the decision to change lanes. Knowledge of such intentions of all
vehicles in a traffic stream can help enhance traffic safety. Unfortunately,
such information is often captured in the form of images/videos. Utilization of
personally identifiable data to train a general model could violate user
privacy. Federated Learning (FL) is a promising tool to resolve this conundrum.
FL efficiently trains models without exposing the underlying data. This paper
introduces a Personalized Federated Learning (PFL) model embedded a long
short-term transformer (LSTR) framework. The framework predicts drivers'
intentions by leveraging in-vehicle videos (of driver movement, gestures, and
expressions) and out-of-vehicle videos (of the vehicle's surroundings -
frontal/rear areas). The proposed PFL-LSTR framework is trained and tested
through real-world driving data collected from human drivers at Interstate 65
in Indiana. The results suggest that the PFL-LSTR exhibits high adaptability
and high precision, and that out-of-vehicle information (particularly, the
driver's rear-mirror viewing actions) is important because it helps reduce
false positives and thereby enhances the precision of driver intention
inference.Comment: Submitted for presentation only at the 2024 Annual Meeting of the
Transportation Research Boar
Driver Digital Twin for Online Prediction of Personalized Lane Change Behavior
Connected and automated vehicles (CAVs) are supposed to share the road with
human-driven vehicles (HDVs) in a foreseeable future. Therefore, considering
the mixed traffic environment is more pragmatic, as the well-planned operation
of CAVs may be interrupted by HDVs. In the circumstance that human behaviors
have significant impacts, CAVs need to understand HDV behaviors to make safe
actions. In this study, we develop a Driver Digital Twin (DDT) for the online
prediction of personalized lane change behavior, allowing CAVs to predict
surrounding vehicles' behaviors with the help of the digital twin technology.
DDT is deployed on a vehicle-edge-cloud architecture, where the cloud server
models the driver behavior for each HDV based on the historical naturalistic
driving data, while the edge server processes the real-time data from each
driver with his/her digital twin on the cloud to predict the lane change
maneuver. The proposed system is first evaluated on a human-in-the-loop
co-simulation platform, and then in a field implementation with three passenger
vehicles connected through the 4G/LTE cellular network. The lane change
intention can be recognized in 6 seconds on average before the vehicle crosses
the lane separation line, and the Mean Euclidean Distance between the predicted
trajectory and GPS ground truth is 1.03 meters within a 4-second prediction
window. Compared to the general model, using a personalized model can improve
prediction accuracy by 27.8%. The demonstration video of the proposed system
can be watched at https://youtu.be/5cbsabgIOdM
Recommended from our members
Knowledge Discovery and Data Mining for Shared Mobility and Connected and Automated Vehicle Applications
The rapid development of shared mobility and connected and automated vehicles (CAVs) has not only brought new intelligent transportation system (ITS) challenges with the new types of mobility, but also brought a huge opportunity to accelerate the connectivity and informatization of transportation systems, particularly when we consider all the new forms of data that is becoming available. The primary challenge is how to take advantage of the enormous amount of data to discover knowledge, build effective models, and develop impactful applications. With the theoretical and experimental progress being made over the last two decades, data mining and machine learning technologies have become key approaches for parsing data, understanding information, and making informed decisions, especially as the rise of deep learning algorithms bringing new levels of performance to the analysis of large datasets. The combination of data mining and ITS can greatly benefit research and advances in shared mobility and CAVs.This dissertation focuses on knowledge discovery and data mining for shared mobility and CAV applications. When considering big data associated with shared mobility operations and CAV research, data mining techniques can be customized with transportation knowledge to initially parse the data. Then machine learning methods can be used to model the parsed data to elicit hidden knowledge. Finally, the discovered knowledge and extracted information can help in the development of effective shared mobility and CAV applications to achieve the goals of a safer, faster, and more eco-friendly transportation systems.In this dissertation, there are four main sections that are addressed. First, new methodologies are introduced for extracting lane-level road features from rough crowdsourced GPS trajectories via data mining, which is subsequently used as the fundamental information for CAV applications. The proposed method results in decimeter level accuracy, which satisfies the positioning needs for many macroscopic and microscopic shared mobility and CAV applications. Second, macroscopic ride-hailing service big data has been analyzed for demand prediction, vehicle operation, and system efficiency monitoring. The proposed deep learning algorithms increase the ride-hailing demand prediction accuracy to 80% and can help the fleet dispatching system reduce 30% of vacant travel distance. Third, microscopic automated vehicle perception data has been analyzed for a real-time computer vision system that can be used for lane change behavior detection. The proposed deep learning design combines the residual neural network image input with time serious control data and reaches 95% of lane change behavior prediction accuracy. Last but not least, new ride sharing and CAV applications have been simulated in a behavior modeling framework to analyze the impact of mobility and energy consumption, which addresses key barriers by quantifying the transportation system-wide mobility, energy and behavior impacts from new mobility technologies using real-world data
- …