Predicting Takeover Performance in Conditionally Automated Driving

In conditionally automated driving, drivers decoupled from operational control of the vehicle have difficulty taking over control when requested. To address this challenge, we conducted a human-in-the-loop experiment wherein the drivers needed to take over control from an automated vehicle. We collected drivers’ physiological data and data from the driving environment, and based on which developed random forest models for predicting drivers’ takeover performance in real time. Drivers’ subjective ratings of their takeover performance were treated as the ground truth. The best random forest model had an accuracy of 70.2% and an F1-score of 70.1%. We also discussed the implications on the design of an adaptive in-vehicle alert system.University of Michigan McityNational Science FoundationPeer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/153789/1/Du et al. 2020.pdfDescription of Du et al. 2020.pdf : Main Fil

Predicting Driver Fatigue in Automated Driving with Explainability

Research indicates that monotonous automated driving increases the incidence of fatigued driving. Although many prediction models based on advanced machine learning techniques were proposed to monitor driver fatigue, especially in manual driving, little is known about how these black-box machine learning models work. In this paper, we proposed a combination of eXtreme Gradient Boosting (XGBoost) and SHAP (SHapley Additive exPlanations) to predict driver fatigue with explanations due to their efficiency and accuracy. First, in order to obtain the ground truth of driver fatigue, PERCLOS (percentage of eyelid closure over the pupil over time) between 0 and 100 was used as the response variable. Second, we built a driver fatigue regression model using both physiological and behavioral measures with XGBoost and it outperformed other selected machine learning models with 3.847 root-mean-squared error (RMSE), 1.768 mean absolute error (MAE) and 0.996 adjusted $R^2$. Third, we employed SHAP to identify the most important predictor variables and uncovered the black-box XGBoost model by showing the main effects of most important predictor variables globally and explaining individual predictions locally. Such an explainable driver fatigue prediction model offered insights into how to intervene in automated driving when necessary, such as during the takeover transition period from automated driving to manual driving

Psychophysiological responses to takeover requests in conditionally automated driving

In SAE Level 3 automated driving, taking over control from automation raises significant safety concerns because drivers out of the vehicle control loop have difficulty negotiating takeover transitions. Existing studies on takeover transitions have focused on drivers' behavioral responses to takeover requests (TORs). As a complement, this exploratory study aimed to examine drivers' psychophysiological responses to TORs as a result of varying non-driving-related tasks (NDRTs), traffic density and TOR lead time. A total number of 102 drivers were recruited and each of them experienced 8 takeover events in a high fidelity fixed-base driving simulator. Drivers' gaze behaviors, heart rate (HR) activities, galvanic skin responses (GSRs), and facial expressions were recorded and analyzed during two stages. First, during the automated driving stage, we found that drivers had lower heart rate variability, narrower horizontal gaze dispersion, and shorter eyes-on-road time when they had a high level of cognitive load relative to a low level of cognitive load. Second, during the takeover transition stage, 4s lead time led to inhibited blink numbers and larger maximum and mean GSR phasic activation compared to 7s lead time, whilst heavy traffic density resulted in increased HR acceleration patterns than light traffic density. Our results showed that psychophysiological measures can indicate specific internal states of drivers, including their workload, emotions, attention, and situation awareness in a continuous, non-invasive and real-time manner. The findings provide additional support for the value of using psychophysiological measures in automated driving and for future applications in driver monitoring systems and adaptive alert systems.University of Michigan McityPeer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/162593/1/AAP_physiological_responses_HF_template.pdfSEL

Using Eye-tracking Data to Predict Situation Awareness in Real Time during Takeover Transitions in Conditionally Automated Driving

Situation awareness (SA) is critical to improving takeover performance during the transition period from automated driving to manual driving. Although many studies measured SA during or after the driving task, few studies have attempted to predict SA in real time in automated driving. In this work, we propose to predict SA during the takeover transition period in conditionally automated driving using eye-tracking and self-reported data. First, a tree ensemble machine learning model, named LightGBM (Light Gradient Boosting Machine), was used to predict SA. Second, in order to understand what factors influenced SA and how, SHAP (SHapley Additive exPlanations) values of individual predictor variables in the LightGBM model were calculated. These SHAP values explained the prediction model by identifying the most important factors and their effects on SA, which further improved the model performance of LightGBM through feature selection. We standardized SA between 0 and 1 by aggregating three performance measures (i.e., placement, distance, and speed estimation of vehicles with regard to the ego-vehicle) of SA in recreating simulated driving scenarios, after 33 participants viewed 32 videos with six lengths between 1 and 20 s. Using only eye-tracking data, our proposed model outperformed other selected machine learning models, having a root-mean-squared error (RMSE) of 0.121, a mean absolute error (MAE) of 0.096, and a 0.719 correlation coefficient between the predicted SA and the ground truth. The code is available at https://github.com/refengchou/Situation-awareness-prediction. Our proposed model provided important implications on how to monitor and predict SA in real time in automated driving using eye-tracking data.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/167003/1/hkwggmgngbsqcmmqkbffywbrjtcmhhxx.pdfDescription of hkwggmgngbsqcmmqkbffywbrjtcmhhxx.pdf : Mian articleSEL

Predicting Driver Takeover Performance and Designing Alert Systems in Conditionally Automated Driving

With the Society of Automotive Engineers Level 3 automation, drivers are no longer required to actively monitor driving environments, and can potentially engage in non-driving related tasks. Nevertheless, when the automation reaches its operational limits, drivers will have to take over control of vehicles at a moment’s notice. Drivers have difficulty with takeover transitions, as they become increasingly decoupled from the operational level of driving. In response to the takeover difficulty, existing literature has investigated various factors affecting takeover performance. However, not all the factors were studied comprehensively, and the results of some factors were mixed. Meanwhile, there is a lack of research on the development of computational models that predict drivers’ takeover performance using their physiological and driving environment data. Furthermore, current research on the design of in-vehicle alert systems suffers from methodological shortcomings and presents identical takeover warnings regardless of event criticality. To address these shortcomings, the goals of this dissertation were to (1) examine the effects of drivers' cognitive load, emotions, traffic density, and takeover request lead time on their driving behavioral (takeover timeliness and quality) and psychophysiological responses (eye movements, galvanic skin responses, and heart rate activities) to takeover requests; (2) develop computational models to predict drivers’ takeover performance using their physiological and driving environment data via machine learning algorithms; and (3) design in-vehicle alert systems with different display modalities and information types and evaluate the displays in different event criticality conditions via human-subject experiments. The results of three human-subject experiments showed that positive emotional valence led to smoother takeover behaviors. Only when drivers had low cognitive load, they had shorter takeover reaction time in high oncoming traffic conditions. High oncoming traffic led to higher collision risk. High speed led to higher collision risk and harsher takeover behaviors in lane changing scenarios, but engendered longer takeover reaction time and smoother takeover behaviors in lane keeping scenarios. Meanwhile, we developed a random forest model to predict drivers' takeover performance with an accuracy of 84.3% and an F1-score of 64.0%. Our model had finer granularity than and outperformed other machine learning models used in prior studies. The findings of alert system design studies showed that drivers had more anxiety with the why only information compared to the why + what will information when information was presented in the speech modality. They felt more prepared to take over control of the vehicle and had more preference for the combination of augmented reality and speech conditions than others when drivers were in high event criticality situations. This dissertation can add to the knowledge base about takeover response investigation, takeover performance prediction, and in-vehicle alert system design. The results will enhance the understanding of how drivers’ emotions, cognitive load, traffic density, and scenario type influence their takeover responses. The computational models for takeover performance prediction are underlying algorithms of in-vehicle monitoring systems in real-world applications. The findings will provide design recommendations to automated vehicle manufacturers on in-vehicle alert systems. This will ultimately enhance the interaction between drivers and automated vehicles and improve driving safety in intelligent transportation systems.PHDIndustrial & Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169727/1/nadu_1.pd