Prediction of drivers’ performance in highly automated vehicles

Purpose: The aim of this research was to assess the predictability of driver’s response to critical hazards during the transition from automated to manual driving in highly automated vehicles using their physiological data.Method: A driving simulator experiment was conducted to collect drivers’ physiological data before, during and after the transition from automated to manual driving. A total of 33 participants between 20 and 30 years old were recruited. Participants went through a driving scenario under the influence of different non-driving related tasks. The repeated measures approach was used to assess the effect of repeatability on the driver’s physiological data. Statistical and machine learning methods were used to assess the predictability of drivers’ response quality based on their physiological data collected before responding to a critical hazard. Findings: - The results showed that the observed physiological data that was gathered before the transition formed strong indicators of the drivers’ ability to respond successfully to a potential hazard after the transition. In addition, physiological behaviour was influenced by driver’s secondary tasks engagement and correlated with the driver’s subjective measures to the difficulty of the task. The study proposes new quality measures to assess the driver’s response to critical hazards in highly automated driving. Machine learning results showed that response time is predictable using regression methods. In addition, the classification methods were able to classify drivers into low, medium and high-risk groups based on their quality measures values. Research Implications: Proposed models help increase the safety of automated driving systems by providing insights into the drivers’ ability to respond to future critical hazards. More research is required to find the influence of age, drivers’ experience of the automated vehicles and traffic density on the stability of the proposed models. Originality: The main contribution to knowledge of this study is the feasibility of predicting drivers’ ability to respond to critical hazards using the physiological behavioural data collected before the transition from automated to manual driving. With the findings, automation systems could change the transition time based on the driver’s physiological state to allow for the safest transition possible. In addition, it provides an insight into driver’s readiness and therefore, allows the automated system to adopt the correct driving strategy and plan to enhance drivers experience and make the transition phase safer for everyone.</div

Modeling driver distraction mechanism and its safety impact in automated vehicle environment.

Automated Vehicle (AV) technology expects to enhance driving safety by eliminating human errors. However, driver distraction still exists under automated driving. The Society of Automotive Engineers (SAE) has defined six levels of driving automation from Level 0~5. Until achieving Level 5, human drivers are still needed. Therefore, the Human-Vehicle Interaction (HVI) necessarily diverts a driver’s attention away from driving. Existing research mainly focused on quantifying distraction in human-operated vehicles rather than in the AV environment. It causes a lack of knowledge on how AV distraction can be detected, quantified, and understood. Moreover, existing research in exploring AV distraction has mainly pre-defined distraction as a binary outcome and investigated the patterns that contribute to distraction from multiple perspectives. However, the magnitude of AV distraction is not accurately quantified. Moreover, past studies in quantifying distraction have mainly used wearable sensors’ data. In reality, it is not realistic for drivers to wear these sensors whenever they drive. Hence, a research motivation is to develop a surrogate model that can replace the wearable device-based data to predict AV distraction. From the safety perspective, there lacks a comprehensive understanding of how AV distraction impacts safety. Furthermore, a solution is needed for safely offsetting the impact of distracted driving. In this context, this research aims to (1) improve the existing methods in quantifying Human-Vehicle Interaction-induced (HVI-induced) driver distraction under automated driving; (2) develop a surrogate driver distraction prediction model without using wearable sensor data; (3) quantitatively reveal the dynamic nature of safety benefits and collision hazards of HVI-induced visual and cognitive distractions under automated driving by mathematically formulating the interrelationships among contributing factors; and (4) propose a conceptual prototype of an AI-driven, Ultra-advanced Collision Avoidance System (AUCAS-L3) targeting HVI-induced driver distraction under automated driving without eye-tracking and video-recording. Fixation and pupil dilation data from the eye tracking device are used to model driver distraction, focusing on visual and cognitive distraction, respectively. In order to validate the proposed methods for measuring and modeling driver distraction, a data collection was conducted by inviting drivers to try out automated driving under Level 3 automation on a simulator. Each driver went through a jaywalker scenario twice, receiving a takeover request under two types of HVI, namely “visual only” and “visual and audible”. Each driver was required to wear an eye-tracker so that the fixation and pupil dilation data could be collected when driving, along with driving performance data being recorded by the simulator. In addition, drivers’ demographical information was collected by a pre-experiment survey. As a result, the magnitude of visual and cognitive distraction was quantified, exploring the dynamic changes over time. Drivers are more concentrated and maintain a higher level of takeover readiness under the “visual and audible” warning, compared to “visual only” warning. The change of visual distraction was mathematically formulated as a function of time. In addition, the change of visual distraction magnitude over time is explained from the driving psychology perspective. Moreover, the visual distraction was also measured by direction in this research, and hotspots of visual distraction were identified with regard to driving safety. When discussing the cognitive distraction magnitude, the driver’s age was identified as a contributing factor. HVI warning type contributes to the significant difference in cognitive distraction acceleration rate. After drivers reach the maximum visual distraction, cognitive distraction tends to increase continuously. Also, this research contributes to quantitatively revealing how visual and cognitive distraction impacts the collision hazards, respectively. Moreover, this research contributes to the literature by developing deep learning-based models in predicting a driver’s visual and cognitive distraction intensity, focusing on demographics, HVI warning types, and driving performance. As a solution to safety issues caused by driver distraction, the AUCAS-L3 has been proposed. The AUCAS-L3 is validated with high accuracies in predicting (a) whether a driver is distracted and does not perform takeover actions and (b) whether crashes happen or not if taken over. After predicting the presence of driver distraction or a crash, AUCAS-L3 automatically applies the brake pedal for drivers as effective and efficient protection to driver distraction under automated driving. And finally, a conceptual prototype in predicting AV distraction and traffic conflict was proposed, which can predict the collision hazards in advance of 0.82 seconds on average

A multidisciplinary research approach for experimental applications in road-driver interaction analysis

This doctoral dissertation represents a cluster of the research activities conducted at the DICAM Department of the University of Bologna during a three years Ph.D. course. In relation to the broader research topic of “road safety”, the presented research focuses on the investigation of the interaction between the road and the drivers according to human factor principles and supported by the following strategies: 1) The multidisciplinary structure of the research team covering the following academic disciplines: Civil Engineering, Psychology, Neuroscience and Computer Science Engineering. 2) The development of several experimental real driving tests aimed to provide investigators with knowledge and insights on the relation between the driver and the surrounding road environment by focusing on the behaviour of drivers. 3) The use of innovative technologies for the experimental studies, capable to collect data of the vehicle and on the user: a GPS data recorder, for recording the kinematic parameters of the vehicle; an eye tracking device, for monitoring the drivers’ visual behaviour; a neural helmet, for the detection of drivers’ cerebral activity (electroencephalography, EEG). 4) The use of mathematical-computational methodologies (deep learning) for data analyses from experimental studies. The outcomes of this work consist of new knowledge on the casualties between drivers’ behaviour and road environment to be considered for infrastructure design. In particular, the ground-breaking results are represented by: - the reliability and effectiveness of the methodology based on human EEG signals to objectively measure driver’s mental workload with respect to different road factors; - the successful approach for extracting latent features from multidimensional driving behaviour data using a deep learning technique, obtaining driving colour maps which represent an immediate visualization with potential impacts on road safety

Predicting Driver Distraction: An Analysis of Machine Learning Algorithms and Input Measures

The research area on the detection and classification of distracted driving is growing in importance as in-vehicle information systems such as navigation and entertainment displays, which introduce sources of distraction for drivers, become more common in vehicles. To mitigate the potential consequences of distracted driving it is necessary for such systems to provide a means of detecting driver distraction and then responding appropriately. This study uses a machine-learning approach to develop classification models that detect and differentiate both cognitive and sensorimotor distraction among drivers, which were induced via secondary tasks in a simulator study. The inputs to these models are combinations of driving performance measures (e.g. brake force, lane offset, speed, and steering angle) and driver physiological measures (e.g. breathing rate, heart rate, and perinasal electrodermal activity), and the outputs are predictions of driver distraction (e.g. cognitive distraction, sensorimotor distraction, or normal driving). Various combinations of driving performance and driver physiological measures, multiple types of machine-learning algorithms, and a systematic feature extraction and reduction method called TSFRESH were used to develop the classification models. Results showed that the physiological measures did not provide significant information for detecting and classifying driver distraction. Furthermore, no significant differences were found between the different machine-learning algorithms. Analyses on feature importance also revealed that driving performance measures including steering angle, lane offset, and speed were the most important indicators of distracted driving, and that features characterizing the extreme values, the variance and fluctuation, and the non linearity and complexity of time series input were more informative for classifying driver distraction than other features. Conclusions suggest that distraction detection models gain more information from driving performance measures than physiological measures and that using features that characterize specific aspects of time series input is useful for classifying driver distraction

Cross-correlation based performance measures for characterizing the influence of in-vehicle interfaces on driving and cognitive workload

Driving is a cognitively loading task which requires drivers\u27 full attention and coordination of both mind and body. However, drivers often engage in side activities which can negatively impact safety. A typical approach for analyzing the influences of side activities on driving is to conduct experiments in which various driving performance measures are collected, such as steering wheel angle and lane position. Those measures are then transformed, typically using means and variances, before being analyzed statistically. However, the problem is that those transformations perform averaging of the acquired data, which can result in missing short, but important events (such as glances directed off-road). As a consequence, statistically significant differences may not be observed between the tested conditions. Nevertheless, just because the influences of in-vehicle interactions do not show in the averages, it does not mean that they do not exist or should be neglected, especially if the nature of the interactions is such that they can be performed frequently (for example, with an infotainment system). This can create a false conclusion about the lack of influence of the tested side activity on driving. The main contribution of this research is in developing two new performance measures inspired by the mathematical function of cross-correlation: one which evaluates the cumulative effect and the other which evaluates the effects of individual instances of in-vehicle interactions on driving and cognitive load. The results from three driving simulator studies demonstrate that our cumulative measure provides more sensitivity to the effects of in-vehicle interactions, even when they are not detected through average-based measures. Additionally, our instance-based measure provides a low-level insight into the nature of the influence of individual in-vehicle interactions. Both measures produce results that can be ranked, which allows determining the relative size of the effect that various in-vehicle interactions have on driving. Finally, we demonstrate a set of variables which can be used for predicting the cumulative and instance-based results. This predictive ability is important, because it may allow obtaining quick simulation results without performing actual experiments, which can be used in the early stages of an interface or experiment design process

Driver behaviour characterization using artificial intelligence techniques in level 3 automated vehicle.

Brighton, James L. - Associate SupervisorAutonomous vehicles free drivers from driving and allow them to engage in some non-driving related activities. However, the engagement in such activities could reduce their awareness of the driving environment, which could bring a potential risk for the takeover process in the current automation level of the intelligent vehicle. Therefore, it is of great importance to monitor the driver's behaviour when the vehicle is in automated driving mode. This research aims to develop a computer vision-based driver monitoring system for autonomous vehicles, which characterises driver behaviour inside the vehicle cabin by their visual attention and hand movement and proves the feasibility of using such features to identify the driver's non-driving related activities. This research further proposes a system, which employs both information to identify driving related activities and non-driving related activities. A novel deep learning- based model has been developed for the classification of such activities. A lightweight model has also been developed for the edge computing device, which compromises the recognition accuracy but is more suitable for further in-vehicle applications. The developed models outperform the state-of-the-art methods in terms of classification accuracy. This research also investigates the impact of the engagement in non-driving related activities on the takeover process and proposes a category method to group the activities to improve the extendibility of the driving monitoring system for unevaluated activities. The finding of this research is important for the design of the takeover strategy to improve driving safety during the control transition in Level 3 automated vehicles.PhD in Manufacturin

Detecting fatigue in car drivers and aircraft pilots by using eye-motion metrics

Fatigue is widely recognised for risking the safety of aviation and ground transportation. To enhance transport safety, fatigue detection systems based on psychophysiological measures have been under development for many years. However, a reliable and robust fatigue detection system is still missing. This thesis starts with a literature review of fatigue concepts in the transportation field and the current psychophysiological measures to fatigue, and narrows down the focus to improving fatigue detection systems using eye-motion measures. A research gap was identified between current fatigue systems only focusing on part of sleepy symptoms and a comprehensive fatigue detection system including mental fatigue needed. To address this gap, four studies were conducted to reshape the understanding of fatigue in transportation and explore effective eye-motion metrics for indicating fatigue considering different causal factors. Studies 1 and 2 investigated the influence of two types of task-related fatigue on eye movement. Twenty participants completed a vigilance task before and after a 1-h simulator-based drive with a secondary task. Forty participants, divided equally into two groups, finished the same task before and after a 1-h and 1.5-h monotonous driving task. The results demonstrated that two types of task-related fatigue caused by cognitive overload and prolonged underload induced different physiological responses to eye-motion metrics. The results also proved that the increased mental fatigue decreased driver’s vigilance. Studies 3 and 4 simulated two hazardous fatigue scenarios for pilots. Study 3 explored the relationship between eye-motion metrics and pilot fatigue in an underload flight condition with sleep deprivation (low workload and sleep pressure). Study 4 explored the effective eye-motion metrics to estimate pilot’s cognitive fatigue imposed by time on task and high workload. The results suggested different eye-motion metrics to indicate sleepiness and mental fatigue. In addition, based on the sleepiness and mental fatigue indicators in Studies 3 and 4, several classifiers were built and evaluated to accurately detect sleepiness and mental fatigue. These findings show that considering casual factors such as sleep pressure, time on task and workload when using eye-motion metrics to detect fatigue can improve the accuracy and face validity of the current fatigue detection systems

Recent developments in vision, aging, and driving: 1988-1994

American Automobile Manufacturers Association, Detroit, Mich.http://deepblue.lib.umich.edu/bitstream/2027.42/1075/2/86404.0001.001.pd

The development of improvements to drivers' direct and indirect vision from vehicles - phase 1

This research project concerning "The development of improvements to drivers' direct and indirect vision from vehicles" has been designed to be conducted in two phases: . Phase 1 whose aim is to scope the existing knowledge base in order to prioritise and direct activities within Phase 2; . Phase 2 whose aim is to investigate specific driver vision problems prioritised in Phase 1 and determine solutions to them. This report details the activities, findings and conclusions resulting from the Phase 1 tasks undertaken