Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning

Although aviation accidents are rare, safety incidents occur more frequently and require a careful analysis to detect and mitigate risks in a timely manner. Analyzing safety incidents using operational data and producing event-based explanations is invaluable to airline companies as well as to governing organizations such as the Federal Aviation Administration (FAA) in the United States. However, this task is challenging because of the complexity involved in mining multi-dimensional heterogeneous time series data, the lack of time-step-wise annotation of events in a flight, and the lack of scalable tools to perform analysis over a large number of events. In this work, we propose a precursor mining algorithm that identifies events in the multidimensional time series that are correlated with the safety incident. Precursors are valuable to systems health and safety monitoring and in explaining and forecasting safety incidents. Current methods suffer from poor scalability to high dimensional time series data and are inefficient in capturing temporal behavior. We propose an approach by combining multiple-instance learning (MIL) and deep recurrent neural networks (DRNN) to take advantage of MIL's ability to learn using weakly supervised data and DRNN's ability to model temporal behavior. We describe the algorithm, the data, the intuition behind taking a MIL approach, and a comparative analysis of the proposed algorithm with baseline models. We also discuss the application to a real-world aviation safety problem using data from a commercial airline company and discuss the model's abilities and shortcomings, with some final remarks about possible deployment directions

Audio Surveillance of Roads:A System for Detecting Anomalous Sounds

In the last decades, several systems based on video analysis have been proposed for automatically detecting accidents on roads to ensure a quick intervention of emergency teams. However, in some situations, the visual information is not sufficient or sufficiently reliable, whereas the use of microphones and audio event detectors can significantly improve the overall reliability of surveillance systems. In this paper, we propose a novel method for detecting road accidents by analyzing audio streams to identify hazardous situations such as tire skidding and car crashes. Our method is based on a two-layer representation of an audio stream: at a low level, the system extracts a set of features that is able to capture the discriminant properties of the events of interest, and at a high level, a representation based on a bag-of-words approach is then exploited in order to detect both short and sustained events. The deployment architecture for using the system in real environments is discussed, together with an experimental analysis carried out on a data set made publicly available for benchmarking purposes. The obtained results confirm the effectiveness of the proposed approach.</p

Intrusion Detection System for Platooning Connected Autonomous Vehicles

The deployment of Connected Autonomous Vehicles (CAVs) in Vehicular Ad Hoc Networks (VANETs) requires secure wireless communication in order to ensure reliable connectivity and safety. However, this wireless communication is vulnerable to a variety of cyber atacks such as spoofing or jamming attacks. In this paper, we describe an Intrusion Detection System (IDS) based on Machine Learning (ML) techniques designed to detect both spoofing and jamming attacks in a CAV environment. The IDS would reduce the risk of traffic disruption and accident caused as a result of cyber-attacks. The detection engine of the presented IDS is based on the ML algorithms Random Forest (RF), k-Nearest Neighbour (k-NN) and One-Class Support Vector Machine (OCSVM), as well as data fusion techniques in a cross-layer approach. To the best of the authors’ knowledge, the proposed IDS is the first in literature that uses a cross-layer approach to detect both spoofing and jamming attacks against the communication of connected vehicles platooning. The evaluation results of the implemented IDS present a high accuracy of over 90% using training datasets containing both known and unknown attacks

Driver Distraction through Conversation Measured with Pupillometry

Assessing a driver´s mental workload during tasks that are not visualmanual is a challenging endeavor. Especially with the rapid development of speech systems, this is becoming increasingly important. Pupillometry promises to be a suitable physiological measurement method, sensitive to variations of cognitive workload. This driving simulator study shows that the pupillometry data indicate a significant increase in cognitive activity during conversation tasks regardless of the acoustic channel used

Computational driver behavior models for vehicle safety applications

The aim of this thesis is to investigate how human driving behaviors can be formally described in mathematical models intended for online personalization of advanced driver assistance systems (ADAS) or offline virtual safety evaluations. Both longitudinal (braking) and lateral (steering) behaviors in routine driving and emergencies are addressed. Special attention is paid to driver glance behavior in critical situations and the role of peripheral vision.First, a hybrid framework based on autoregressive models with exogenous input (ARX-models) is employed to predict and classify driver control in real time. Two models are suggested, one targeting steering behavior and the other longitudinal control behavior. Although the predictive performance is unsatisfactory, both models can distinguish between different driving styles.Moreover, a basic model for drivers\u27 brake initiation and modulation in critical longitudinal situations (specifically for rear-end conflicts) is constructed. The model is based on a conceptual framework of noisy evidence accumulation and predictive processing. Several model extensions related to gaze behavior are also proposed and successfully fitted to real-world crashes and near-crashes. The influence of gaze direction is further explored in a driving simulator study, showing glance response times to be independent of the glance\u27s visual eccentricity, while brake response times increase for larger gaze angles, as does the rate of missed target detections.Finally, the potential of a set of metrics to quantify subjectively perceived risk in lane departure situations to explain drivers\u27 recovery steering maneuvers was investigated. The most influential factors were the relative yaw angle and splay angle error at steering initiation. Surprisingly, it was observed that drivers often initiated the recovery steering maneuver while looking off-road.To sum up, the proposed models in this thesis facilitate the development of personalized ADASs and contribute to trustworthy virtual evaluations of current, future, and conceptual safety systems. The insights and ideas contribute to an enhanced, human-centric system development, verification, and validation process. In the long term, this will likely lead to improved vehicle safety and a reduced number of severe injuries and fatalities in traffic

SenSys: A Smartphone-Based Framework for ITS applications

Intelligent transportation systems (ITS) use different methods to collect and process traffic data. Conventional techniques suffer from different challenges, like the high installation and maintenance cost, connectivity and communication problems, and the limited set of data. The recent massive spread of smartphones among drivers encouraged the ITS community to use them to solve ITS challenges. Using smartphones in ITS is gaining an increasing interest among researchers and developers. Typically, the set of sensors that comes with smartphones is utilized to develop tools and services in order to enhance safety and driving experience. GPS, cameras, Bluetooth, inertial sensors and other embedded sensors are used to detect and analyze drivers\u27 behavior and vehicles\u27 motion. The use of smartphones made the data collection process easier because of their availability among drivers, the set of different sensors, the computation ability, and the low installation and maintenance cost. On the other hand, different smartphones sensors have diverse characteristics and accuracy and each one of them needs special fusion, processing, and filtration methods to generate more stable and accurate data. Using smartphones in ITS faces different challenges like inaccurate readings, weak GPS reception, noisy sensors and unaligned readings.These challenges waste researchers and developers time and effort, and they prevent them from building accurate ITS applications. This work proposes SenSys a smartphone framework that collects and processes traffic data and then analyzes and extracts vehicle dynamics and vehicle activities which can be used by developers and researchers to create their navigation, communication, and safety ITS applications. SenSys framework fuses and filters smartphone\u27s sensors readings which result in enhancing the accuracy of tracking and analyzing various vehicle dynamics such as vehicle\u27s stops, lane changes, turn detection, and accurate vehicle speed calculation that, in turn, will enable development of new ITS applications and services

Driving and Speaking: Revelations by the Head-Mounted Detection Response Task

The cognitive workload of speech-related activity needs to be examined in an economic and simple way. This is especially important as invehicle technology is becoming more cognitive with, for example, the use of speech-interaction and industry will need a way to keep pace with new technologies. One proposed way to measure cognitive workload is the detection response task (DRT) method. In this study, the DRT was used to assess different speech-related cognitive tasks. Three conversation tasks and the n-back task were performed together with a simulated driving task and a head-mounted DRT (HDRT). The aim was to evaluate the conversation and n-back tasks with the HDRT and to quantify the respective cognitive workload. Results show an increase in HDRT reaction times when additional cognitive tasks are performed relative to baseline measurements. In line with other research methods, the HDRT provided a reliable measurement of additional workload

ChatGPT is on the Horizon: Could a Large Language Model be Suitable for Intelligent Traffic Safety Research and Applications?

ChatGPT embarks on a new era of artificial intelligence and will revolutionize the way we approach intelligent traffic safety systems. This paper begins with a brief introduction about the development of large language models (LLMs). Next, we exemplify using ChatGPT to address key traffic safety issues. Furthermore, we discuss the controversies surrounding LLMs, raise critical questions for their deployment, and provide our solutions. Moreover, we propose an idea of multi-modality representation learning for smarter traffic safety decision-making and open more questions for application improvement. We believe that LLM will both shape and potentially facilitate components of traffic safety research.Comment: Submitted to Nature - Machine Intelligence (Revised and Extended