996 research outputs found
VRSTNN : visual-relational spatio-temporal neural network for early hazardous event detection in automated driving systems
Reliable and early detection of hazardous events is vital for the safe deployment of automated driving systems. Yet, it remains challenging as road environments can be highly complex and dynamic. State-of-the-art solutions utilise neural networks to learn visual features and temporal patterns from collision videos. However, in this paper, we show how visual features alone may not provide the essential context needed to detect early warning patterns. To address these limitations, we first propose an input encoding that captures the context of the scene. This is achieved by formulating a scene as a graph to provide a framework to represent the arrangement, relationships and behaviours of each road user. We then process the graphs using graph neural networks to identify scene context from: 1) the collective behaviour of nearby road users based on their relationships and 2) local node features that describe individual behaviour. We then propose a novel visual-relational spatiotemporal neural network (VRSTNN) that leverages multi-modal processing to understand scene context and fuse it with the visual characteristics of the scene for more reliable and early hazard detection. Our results show that our VRSTNN outperforms stateof- the-art models in terms of accuracy, F1 and false negative rate on a real and synthetic benchmark dataset: DOTA and GTAC
Recent Advances in Graph-based Machine Learning for Applications in Smart Urban Transportation Systems
The Intelligent Transportation System (ITS) is an important part of modern
transportation infrastructure, employing a combination of communication
technology, information processing and control systems to manage transportation
networks. This integration of various components such as roads, vehicles, and
communication systems, is expected to improve efficiency and safety by
providing better information, services, and coordination of transportation
modes. In recent years, graph-based machine learning has become an increasingly
important research focus in the field of ITS aiming at the development of
complex, data-driven solutions to address various ITS-related challenges. This
chapter presents background information on the key technical challenges for ITS
design, along with a review of research methods ranging from classic
statistical approaches to modern machine learning and deep learning-based
approaches. Specifically, we provide an in-depth review of graph-based machine
learning methods, including basic concepts of graphs, graph data
representation, graph neural network architectures and their relation to ITS
applications. Additionally, two case studies of graph-based ITS applications
proposed in our recent work are presented in detail to demonstrate the
potential of graph-based machine learning in the ITS domain
Review of graph-based hazardous event detection methods for autonomous driving systems
Automated and autonomous vehicles are often required to operate in complex road environments with potential hazards that may lead to hazardous events causing injury or even death. Therefore, a reliable autonomous hazardous event detection system is a key enabler for highly autonomous vehicles (e.g., Level 4 and 5 autonomous vehicles) to operate without human supervision for significant periods of time. One promising solution to the problem is the use of graph-based methods that are powerful tools for relational reasoning. Using graphs to organise heterogeneous knowledge about the operational environment, link scene entities (e.g., road users, static objects, traffic rules) and describe how they affect each other. Due to a growing interest and opportunity presented by graph-based methods for autonomous hazardous event detection, this paper provides a comprehensive review of the state-of-the-art graph-based methods that we categorise as rule-based, probabilistic, and machine learning-driven. Additionally, we present an in-depth overview of the available datasets to facilitate hazardous event training and evaluation metrics to assess model performance. In doing so, we aim to provide a thorough overview and insight into the key research opportunities and open challenges
Graph Convolutional Networks for Complex Traffic Scenario Classification
A scenario-based testing approach can reduce the time required to obtain
statistically significant evidence of the safety of Automated Driving Systems
(ADS). Identifying these scenarios in an automated manner is a challenging
task. Most methods on scenario classification do not work for complex scenarios
with diverse environments (highways, urban) and interaction with other traffic
agents. This is mirrored in their approaches which model an individual vehicle
in relation to its environment, but neglect the interaction between multiple
vehicles (e.g. cut-ins, stationary lead vehicle). Furthermore, existing
datasets lack diversity and do not have per-frame annotations to accurately
learn the start and end time of a scenario. We propose a method for complex
traffic scenario classification that is able to model the interaction of a
vehicle with the environment, as well as other agents. We use Graph
Convolutional Networks to model spatial and temporal aspects of these
scenarios. Expanding the nuScenes and Argoverse 2 driving datasets, we
introduce a scenario-labeled dataset, which covers different driving
environments and is annotated per frame. Training our method on this dataset,
we present a promising baseline for future research on per-frame complex
scenario classification.Comment: Netherlands Conference on Computer Vision (NCCV) 2023 camera-ready +
supplementary materia
Prediction of social dynamic agents and long-tailed learning challenges: a survey
Autonomous robots that can perform common tasks like driving, surveillance, and chores have the biggest potential for impact due to frequency of usage, and the biggest potential for risk due to direct interaction with humans. These tasks take place in openended environments where humans socially interact and pursue their goals in complex and diverse ways. To operate in such environments, such systems must predict this behaviour, especially when the behavior is unexpected and potentially dangerous. Therefore, we summarize trends in various types of tasks, modeling methods, datasets, and social interaction modules aimed at predicting the future location of dynamic, socially interactive agents. Furthermore, we describe long-tailed learning techniques from classification and regression problems that can be applied to prediction problems. To our knowledge this is the first work that reviews social interaction modeling within prediction, and long-tailed learning techniques within regression and prediction
- …