Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving

In conditionally automated driving, drivers have difficulty in takeover transitions as they become increasingly decoupled from the operational level of driving. Factors influencing takeover performance, such as takeover lead time and the engagement of non-driving-related tasks, have been studied in the past. However, despite the important role emotions play in human-machine interaction and in manual driving, little is known about how emotions influence drivers’ takeover performance. This study, therefore, examined the effects of emotional valence and arousal on drivers’ takeover timeliness and quality in conditionally automated driving. We conducted a driving simulation experiment with 32 participants. Movie clips were played for emotion induction. Participants with different levels of emotional valence and arousal were required to take over control from automated driving, and their takeover time and quality were analyzed. Results indicate that positive valence led to better takeover quality in the form of a smaller maximum resulting acceleration and a smaller maximum resulting jerk. However, high arousal did not yield an advantage in takeover time. This study contributes to the literature by demonstrating how emotional valence and arousal affect takeover performance. The benefits of positive emotions carry over from manual driving to conditionally automated driving while the benefits of arousal do not

Adaptive multi-modal interface model concerning mental workload in take-over request during semi-autonomous driving

With the development of automated driving technologies, human factors involved in automated driving are gaining increasing attention for a balanced implementation of the convenience brought by the technology and safety risk in commercial vehicle models. One influential human factor is mental workload. In the take-over request (TOR) from autonomous to manual driving at level 3 of International Society of Automotive Engineers' (SAE) Levels of Driving Automation, the time window for the driver to have full comprehension of the driving environment is extremely short, which means the driver is under high mental workload. To support the driver during a TOR, we propose an adaptive multi-modal interface model concerning mental workload. In this study, we evaluated the reliability of only part of the proposed model in a driving-simulator experiment as well as using the experimental data from a previous study

Design of Single-modal Take-over Request in SAE Level 2 & 3 Automated Vehicle

Recently, cutting-edge technology has led to the development of automated vehicles, but the limitations of the related technology may lead to hazardous situations. This resulted in the remarkable significance of the interaction between automated vehicles and drivers. In particular, the transition between the driver and the automated vehicle in accordance with Level 3 of SAE J3016 is inevitable, and guidelines or standards regarding the takeover should be provided. Therefore, we aim to prepare the safety guidelines for the takeover and to conduct a comparative test. First, guidelines for visual, auditory, and haptic displays in existing vehicles were examined. Second, preliminary research was conducted on the modality of automated vehicles. Third, we carried out a modality investigation regarding the partially automated vehicle. Based on this, we proposed visual, auditory, and haptic signals for each modality. This will serve as a significant starting point for future research based on multimodal methods

Comparing eye-tracking metrics of mental workload caused by NDRTs in semi-autonomous driving

The objective of this study was to verify the effectiveness of eye-tacking metrics in indicating driver’s mental workload in semi-autonomous driving when the driver is engaged in different non-driving related tasks (NDRTs). A driving simulator was developed for three scenarios (high-, medium-, and low-mental workload presented by SAE (Society of Automotive Engineers) Levels 0, 1, and 2) and three uni-modality secondary tasks. Thirty-six individuals participated in the driving simulation experiment. NASA-TLX (Task Load Index), secondary task performance, and eye-tracking metrics were used as indicators of mental workload. The subjective rating using the NASA-TLX showed a main effect of autonomous level on mental workload in both visual and auditory tasks. Correlation-matrix calculation and principal-component extraction indicated that pupil diameter change, number of saccades, saccade duration, fixation duration, and 3D gaze entropy were effective indicators of a driver’s mental workload in the visual and auditory multi-tasking situations of semi-autonomous driving. The accuracy of predicting the mental-workload level using the K-Nearest Neighbor (KNN) classifier was 88.9% with bootstrapped data. These results can be used to develop an adaptive multi-modal interface that issues efficient and safe takeover requests

Modeling driver distraction mechanism and its safety impact in automated vehicle environment.

Automated Vehicle (AV) technology expects to enhance driving safety by eliminating human errors. However, driver distraction still exists under automated driving. The Society of Automotive Engineers (SAE) has defined six levels of driving automation from Level 0~5. Until achieving Level 5, human drivers are still needed. Therefore, the Human-Vehicle Interaction (HVI) necessarily diverts a driver’s attention away from driving. Existing research mainly focused on quantifying distraction in human-operated vehicles rather than in the AV environment. It causes a lack of knowledge on how AV distraction can be detected, quantified, and understood. Moreover, existing research in exploring AV distraction has mainly pre-defined distraction as a binary outcome and investigated the patterns that contribute to distraction from multiple perspectives. However, the magnitude of AV distraction is not accurately quantified. Moreover, past studies in quantifying distraction have mainly used wearable sensors’ data. In reality, it is not realistic for drivers to wear these sensors whenever they drive. Hence, a research motivation is to develop a surrogate model that can replace the wearable device-based data to predict AV distraction. From the safety perspective, there lacks a comprehensive understanding of how AV distraction impacts safety. Furthermore, a solution is needed for safely offsetting the impact of distracted driving. In this context, this research aims to (1) improve the existing methods in quantifying Human-Vehicle Interaction-induced (HVI-induced) driver distraction under automated driving; (2) develop a surrogate driver distraction prediction model without using wearable sensor data; (3) quantitatively reveal the dynamic nature of safety benefits and collision hazards of HVI-induced visual and cognitive distractions under automated driving by mathematically formulating the interrelationships among contributing factors; and (4) propose a conceptual prototype of an AI-driven, Ultra-advanced Collision Avoidance System (AUCAS-L3) targeting HVI-induced driver distraction under automated driving without eye-tracking and video-recording. Fixation and pupil dilation data from the eye tracking device are used to model driver distraction, focusing on visual and cognitive distraction, respectively. In order to validate the proposed methods for measuring and modeling driver distraction, a data collection was conducted by inviting drivers to try out automated driving under Level 3 automation on a simulator. Each driver went through a jaywalker scenario twice, receiving a takeover request under two types of HVI, namely “visual only” and “visual and audible”. Each driver was required to wear an eye-tracker so that the fixation and pupil dilation data could be collected when driving, along with driving performance data being recorded by the simulator. In addition, drivers’ demographical information was collected by a pre-experiment survey. As a result, the magnitude of visual and cognitive distraction was quantified, exploring the dynamic changes over time. Drivers are more concentrated and maintain a higher level of takeover readiness under the “visual and audible” warning, compared to “visual only” warning. The change of visual distraction was mathematically formulated as a function of time. In addition, the change of visual distraction magnitude over time is explained from the driving psychology perspective. Moreover, the visual distraction was also measured by direction in this research, and hotspots of visual distraction were identified with regard to driving safety. When discussing the cognitive distraction magnitude, the driver’s age was identified as a contributing factor. HVI warning type contributes to the significant difference in cognitive distraction acceleration rate. After drivers reach the maximum visual distraction, cognitive distraction tends to increase continuously. Also, this research contributes to quantitatively revealing how visual and cognitive distraction impacts the collision hazards, respectively. Moreover, this research contributes to the literature by developing deep learning-based models in predicting a driver’s visual and cognitive distraction intensity, focusing on demographics, HVI warning types, and driving performance. As a solution to safety issues caused by driver distraction, the AUCAS-L3 has been proposed. The AUCAS-L3 is validated with high accuracies in predicting (a) whether a driver is distracted and does not perform takeover actions and (b) whether crashes happen or not if taken over. After predicting the presence of driver distraction or a crash, AUCAS-L3 automatically applies the brake pedal for drivers as effective and efficient protection to driver distraction under automated driving. And finally, a conceptual prototype in predicting AV distraction and traffic conflict was proposed, which can predict the collision hazards in advance of 0.82 seconds on average

Strategies for dynamic appointment making by container terminals

We consider a container terminal that has to make appointments with barges dynamically, in real-time, and partly automatic. The challenge for the terminal is to make appointments with only limited knowledge about future arriving barges, and in the view of uncertainty and disturbances, such as uncertain arrival and handling times, as well as cancellations and no-shows. We illustrate this problem using an innovative implementation project which is currently running in the Port of Rotterdam. This project aims to align barge rotations and terminal quay schedules by means of a multi-agent system. In this\ud paper, we take the perspective of a single terminal that will participate in this planning system, and focus on the decision making capabilities of its intelligent agent. We focus on the question how the terminal operator can optimize, on an operational level, the utilization of its quay resources, while making reliable appointments with barges, i.e., with a guaranteed departure time. We explore two approaches: (i) an analytical approach based on the value of having certain intervals within the schedule and (ii) an approach based on sources of exibility that are naturally available to the terminal. We use simulation to get insight in the benefits of these approaches. We conclude that a major increase in utilization degree could be achieved only by deploying the sources of exibility, without harming the waiting time of barges too much

Automation-driven transformation of road infrastructure: a multi-perspective case study

Automated driving is widely assumed to play a major role in future mobility. In this paper, we focus on “high driving automation” (SAE level 4) and analyze potentials in terms of more efficient traffic flows, travel times, and user benefits as well as potential impacts on urban neighborhoods and potentials for sustainable urban development. Along selected use cases of automated vehicles in the region of Karlsruhe, Germany, we show that at least moderate user benefits can be expected from travel time savings, with the extent depending on the defined operational design domain of the vehicles and the routes taken. With regard to residential development of urban neighborhoods, there are opportunities for repurposing public space. However, these are limited and require parallel regulatory measures to become effective