IS THE MAGIC IN THE HANDS OR EYES? STUDYING THE EFFECTS OF DRIVER MONITORING STRATEGIES ON SITUATION AWARENESS, MIND WANDERING, AND CHANGE DETECTION BEHAVIOR IN LOW AND MEDIUM FIDELITY SEMI-AUTOMATED DRIVING ENVIRONMENTS

Advanced Driver Assist Systems (ADAS) are SAE level 2 features that require active driver control while engaged. Although drivers can have their feet off the pedals and hands off the steering wheel, they must maintain supervisory control of the vehicle. However, when these features are in use, drivers may become less aware of their surroundings, increasing the risk of accidents. To counter this problem, vehicle manufacturers use driver monitoring strategies to ensure drivers remain attentive while ADAS features are active. These monitoring strategies employ instructions to ensure drivers are engaged in the driving task. These instructions are broadly classified into hands-on-the-wheel and hands-free (eyes-on-the-road). The hands-on-the-wheel strategy measures driver engagement by examining steering wheel torque, while the hands-free strategy tracks the driver\u27s eyes to ensure they remain on the road. Although both strategies are commonly used in vehicles with SAE level 2 automation, there is a lack of publicly available data on their effectiveness and impact on takeover performance. In this dissertation, three studies were conducted to measure the effects of the hands-on-the-wheel and eyes-on-the-road driver monitoring strategies on situation awareness, change detection, mind-wandering, and gaze behavior. Study 1 was exploratory and utilized a low-fidelity semi-automated driving task to examine the effects of the two engagement strategies on driver attention during level 2 ADAS driving. Study 2 was an extension of Study 1 and moved to more naturalistic automation-related change detection in addition to a SAGAT freeze-probe protocol and comfort, fatigue, engagement, and takeover readiness measures in addition to the ones measured in Study 1. Study 3 extended Study 2 in a medium-fidelity driving simulator to investigate the effects of the two driver engagement strategies on driving performance variables and driver attention. Study 1 found that the hands-on-the-wheel strategy promoted less mind wandering during level 2 automated driving. Study 2 found that while the hands-on-the-wheel strategy also promoted less mind wandering, it promoted higher situation awareness, more perceived engagement with automated driving, less self-reported fatigue, and faster response to takeover requests. On the contrary, Study 3 found that the eyes-on-the-road strategy exhibited higher SA, faster responses to takeover requests, and less steering wheel variability but closer following distances post-takeover. Although the three studies have mixed findings, the hands-on-the-wheel strategy appears more promising because it engages drivers physically with the driving task, potentially leading to safer driving behaviors. This work has broader implications for SAE level 2 and 3 ADAS features, reinforcing the need for an engagement strategy with driver monitoring systems. Even as level 3 and higher technologies are developed, the results here inform strategies for automation-level step-downs as the drivers are brought back into actively controlling the vehicle

Safety Evaluation Using Counterfactual Simulations: The use of computational driver behavior models in crash avoidance systems and virtual simulations with optimal subsampling

Traffic safety is a problem worldwide. In-vehicle conflict and crash avoidance systems have been under development and assessment for some time, as integral parts of Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS). Among the methods used to assess conflict and crash avoidance systems developed by the automotive industry, virtual safety assessment methods have been shown to have great potential and efficiency. In fact, scenario generation-based virtual safety assessments play—and are likely to continue to play—a very important role in the assessments of vehicles of all levels of automation. The ultimate aim of this thesis is to improve the safety performance of conflict and crash avoidance systems. This aim is addressed through the use of computational driver models in two different ways. First, by using comfort-zone boundaries in system design, and second, by using a behavior-based crash-causation model together with a novel optimized scenario generation method for virtual safety assessment.The first objective of this thesis is to investigate how a driver model which includes road users’ comfortable behaviors in crash avoidance algorithms impacts the systems’ safety performance and the residual crash characteristics. Chinese car-to-two-wheeler crashes were targeted; Automated Emergency Braking (AEB) algorithms, which comprised the proposed crash avoidance systems, were compared to a traditional AEB algorithm. The proposed algorithms showed larger safety performance benefits. In addition, the similarities in residual crash characteristics regarding impact speed and location after different AEB implementations can potentially simplify the designs of in-crash protection system in future.The second objective is to develop and apply a method for efficient subsampling in crash-causation-model-based scenario generation for virtual safety assessment. The method, which is machine-learning-assisted, actively and iteratively updates the sampling probability based on new simulation results. The crash-causation model is based on off-road glances and a distribution of driver maximum decelerations in critical situations. A simple time-to-collision-based AEB algorithm was used to demonstrate the assessment process as well as the benefits of combining crash-causation-model-based scenario generation and optimal subsampling. The sampling methods are designed to target specific safety benefit indicators, such as impact speed reduction and crash avoidance rate. The results of the study show that the proposed sampling method requires almost 50% fewer simulations than traditional importance sampling.Future work aims to focus on applying the active sampling method to driver-model-based car-to-vulnerable road user (VRU) scenario generation. In addition to assessing conflict and crash avoidance system performance, a novel stopping criterion based on Bayesian future prediction will be further developed and demonstrated for use in experiments (e.g., as part of developing driver models) and virtual simulations (e.g., using driver-behavior-based crash-causation models). This criterion will be able to indicate when studies are unlikely to yield actionable results within the budget available, facilitating the decision to discontinue them while they are being run

Advances in Automated Driving Systems

Electrification, automation of vehicle control, digitalization and new mobility are the mega-trends in automotive engineering, and they are strongly connected. While many demonstrations for highly automated vehicles have been made worldwide, many challenges remain in bringing automated vehicles to the market for private and commercial use. The main challenges are as follows: reliable machine perception; accepted standards for vehicle-type approval and homologation; verification and validation of the functional safety, especially at SAE level 3+ systems; legal and ethical implications; acceptance of vehicle automation by occupants and society; interaction between automated and human-controlled vehicles in mixed traffic; human–machine interaction and usability; manipulation, misuse and cyber-security; the system costs of hard- and software and development efforts. This Special Issue was prepared in the years 2021 and 2022 and includes 15 papers with original research related to recent advances in the aforementioned challenges. The topics of this Special Issue cover: Machine perception for SAE L3+ driving automation; Trajectory planning and decision-making in complex traffic situations; X-by-Wire system components; Verification and validation of SAE L3+ systems; Misuse, manipulation and cybersecurity; Human–machine interactions, driver monitoring and driver-intention recognition; Road infrastructure measures for the introduction of SAE L3+ systems; Solutions for interactions between human- and machine-controlled vehicles in mixed traffic

Modelling drivers’ braking behaviour and comfort under normal driving

The increasing growth of population and a rising number of vehicles, connected to an individual, demand new solutions to reduce traffic delays and enhance road safety. Autonomous Vehicles (AVs) have been considered as an optimal solution to overcome those problems. Despite the remarkable research and development progress in the area of (semi) AVs over the last decades, there is still concern that occupants may not feel safe and comfortable due to the robot-like driving behaviour of the current technology. In order to facilitate their rapid uptake and market penetration, ride comfort in AVs must be ensured.Braking behaviour has been identified to be a crucial factor in ride comfort. There is a dearth of research on which factors affect the braking behaviour and the comfort level while braking and which braking profiles make the occupants feel safe and comfortable. Therefore, the primary aim of this thesis is to model the deceleration events of drivers under normal driving conditions to guide comfortable braking design. The aim was achieved by exploiting naturalistic driving data from three projects: (1) the Pan-European TeleFOT (Field Operational Tests of Aftermarket and Nomadic Devices in Vehicles) project, (2) the Field Operational Test (FOT) conducted by Loughborough University and Original Equipment Manufacturer (OEM), and (3) the UDRIVE Naturalistic Driving Study.A total of about 35 million observations were examined from 86 different drivers and 644 different trips resulting in almost 10,000 deceleration events for the braking features analysis and 21,600 deceleration events for the comfort level analysis. Since deceleration events are nested within trips and trips within drivers, multilevel mixed-effects linear models were employed to develop relationships between deceleration value and duration and the factors influencing them. The examined factors were kinematics, situational, driver and trip characteristics with the first two categories to affect the most the deceleration features. More specifically, the initial speed and the reason for braking play a significant role, whereas the driver’s characteristics, i.e. the age and gender do not affect the deceleration features, except for driver’s experience which significantly affects the deceleration duration.An algorithm was developed to calculate the braking profiles, indicating that the most used profile follows smooth braking at the beginning followed by a harder one. Moreover, comfort levels of drivers were analysed using the Mixed Multinomial Logit models to identify the effect of the explanatory factors on the comfort category of braking events. Kinematic factors and especially TTC and time headway (THW) were found to affect the most the comfort level. Particularly, when TTC or THW are increased by 1 second, the odds of the event to be “very comfortable” are respectively 1.03 and 4.5 times higher than being “very uncomfortable”. Moreover, the driver’s characteristic, i.e. age and gender affect significantly the comfort level of the deceleration event. Findings from this thesis can support vehicle manufacturers to ensure comfortable and safe braking operations of AVs.</div

Prediction of drivers’ performance in highly automated vehicles

Purpose: The aim of this research was to assess the predictability of driver’s response to critical hazards during the transition from automated to manual driving in highly automated vehicles using their physiological data.Method: A driving simulator experiment was conducted to collect drivers’ physiological data before, during and after the transition from automated to manual driving. A total of 33 participants between 20 and 30 years old were recruited. Participants went through a driving scenario under the influence of different non-driving related tasks. The repeated measures approach was used to assess the effect of repeatability on the driver’s physiological data. Statistical and machine learning methods were used to assess the predictability of drivers’ response quality based on their physiological data collected before responding to a critical hazard. Findings: - The results showed that the observed physiological data that was gathered before the transition formed strong indicators of the drivers’ ability to respond successfully to a potential hazard after the transition. In addition, physiological behaviour was influenced by driver’s secondary tasks engagement and correlated with the driver’s subjective measures to the difficulty of the task. The study proposes new quality measures to assess the driver’s response to critical hazards in highly automated driving. Machine learning results showed that response time is predictable using regression methods. In addition, the classification methods were able to classify drivers into low, medium and high-risk groups based on their quality measures values. Research Implications: Proposed models help increase the safety of automated driving systems by providing insights into the drivers’ ability to respond to future critical hazards. More research is required to find the influence of age, drivers’ experience of the automated vehicles and traffic density on the stability of the proposed models. Originality: The main contribution to knowledge of this study is the feasibility of predicting drivers’ ability to respond to critical hazards using the physiological behavioural data collected before the transition from automated to manual driving. With the findings, automation systems could change the transition time based on the driver’s physiological state to allow for the safest transition possible. In addition, it provides an insight into driver’s readiness and therefore, allows the automated system to adopt the correct driving strategy and plan to enhance drivers experience and make the transition phase safer for everyone.</div

Vehicle and Traffic Safety

The book is devoted to contemporary issues regarding the safety of motor vehicles and road traffic. It presents the achievements of scientists, specialists, and industry representatives in the following selected areas of road transport safety and automotive engineering: active and passive vehicle safety, vehicle dynamics and stability, testing of vehicles (and their assemblies), including electric cars as well as autonomous vehicles. Selected issues from the area of accident analysis and reconstruction are discussed. The impact on road safety of aspects such as traffic control systems, road infrastructure, and human factors is also considered

Exploration of smart infrastructure for drivers of autonomous vehicles

The connection between vehicles and infrastructure is an integral part of providing autonomous vehicles information about the environment. Autonomous vehicles need to be safe and users need to trust their driving decision. When smart infrastructure information is integrated into the vehicle, the driver needs to be informed in an understandable manner what the smart infrastructure detected. Nevertheless, interactions that benefit from smart infrastructure have not been the focus of research, leading to knowledge gaps in the integration of smart infrastructure information in the vehicle. For example, it is unclear, how the information from two complex systems can be presented, and if decisions are made, how these can be explained. Enriching the data of vehicles with information from the infrastructure opens unexplored opportunities. Smart infrastructure provides vehicles with information to predict traffic flow and traffic events. Additionally, it has information about traffic events in several kilometers distance and thus enables a look ahead on a traffic situation, which is not in the immediate view of drivers. We argue that this smart infrastructure information can be used to enhance the driving experience. To achieve this, we explore designing novel interactions, providing warnings and visualizations about information that is out of the view of the driver, and offering explanations for the cause of changed driving behavior of the vehicle. This thesis focuses on exploring the possibilities of smart infrastructure information with a focus on the highway. The first part establishes a design space for 3D in-car augmented reality applications that profit from smart infrastructure information. Through the input of two focus groups and a literature review, use cases are investigated that can be introduced in the vehicle's interaction interface which, among others, rely on environment information. From those, a design space that can be used to design novel in-car applications is derived. The second part explores out-of-view visualizations before and during take over requests to increase situation awareness. With three studies, different visualizations for out-of-view information are implemented in 2D, stereoscopic 3D, and augmented reality. Our results show that visualizations improve the situation awareness about critical events in larger distances during take over request situations. In the third part, explanations are designed for situations in which the vehicle drives unexpectedly due to unknown reasons. Since smart infrastructure could provide connected vehicles with out-of-view or cloud information, the driving maneuver of the vehicle might remain unclear to the driver. Therefore, we explore the needs of drivers in those situations and derive design recommendations for an interface which displays the cause for the unexpected driving behavior. This thesis answers questions about the integration of environment information in vehicles'. Three important aspects are explored, which are essential to consider when implementing use cases with smart infrastructure in mind. It enables to design novel interactions, provides insights on how out-of-view visualizations can improve the drivers' situation awareness and explores unexpected driving situations and the design of explanations for them. Overall, we have shown how infrastructure and connected vehicle information can be introduced in vehicles' user interface and how new technology such as augmented reality glasses can be used to improve the driver's perception of the environment.Autonome Fahrzeuge werden immer mehr in den alltäglichen Verkehr integriert. Die Verbindung von Fahrzeugen mit der Infrastruktur ist ein wesentlicher Bestandteil der Bereitstellung von Umgebungsinformationen in autonome Fahrzeugen. Die Erweiterung der Fahrzeugdaten mit Informationen der Infrastruktur eröffnet ungeahnte Möglichkeiten. Intelligente Infrastruktur übermittelt verbundenen Fahrzeugen Informationen über den prädizierten Verkehrsfluss und Verkehrsereignisse. Zusätzlich können Verkehrsgeschehen in mehreren Kilometern Entfernung übermittelt werden, wodurch ein Vorausblick auf einen Bereich ermöglicht wird, der für den Fahrer nicht unmittelbar sichtbar ist. Mit dieser Dissertation wird gezeigt, dass Informationen der intelligenten Infrastruktur benutzt werden können, um das Fahrerlebnis zu verbessern. Dies kann erreicht werden, indem innovative Interaktionen gestaltet werden, Warnungen und Visualisierungen über Geschehnisse außerhalb des Sichtfelds des Fahrers vermittelt werden und indem Erklärungen über den Grund eines veränderten Fahrzeugverhaltens untersucht werden. Interaktionen, welche von intelligenter Infrastruktur profitieren, waren jedoch bisher nicht im Fokus der Forschung. Dies führt zu Wissenslücken bezüglich der Integration von intelligenter Infrastruktur in das Fahrzeug. Diese Dissertation exploriert die Möglichkeiten intelligenter Infrastruktur, mit einem Fokus auf die Autobahn. Der erste Teil erstellt einen Design Space für Anwendungen von augmentierter Realität (AR) in 3D innerhalb des Autos, die unter anderem von Informationen intelligenter Infrastruktur profitieren. Durch das Ergebnis mehrerer Studien werden Anwendungsfälle in einem Katalog gesammelt, welche in die Interaktionsschnittstelle des Autos einfließen können. Diese Anwendungsfälle bauen unter anderem auf Umgebungsinformationen. Aufgrund dieser Anwendungen wird der Design Space entwickelt, mit Hilfe dessen neuartige Anwendungen für den Fahrzeuginnenraum entwickelt werden können. Der zweite Teil exploriert Visualisierungen für Verkehrssituationen, die außerhalb des Sichtfelds des Fahrers sind. Es wird untersucht, ob durch diese Visualisierungen der Fahrer besser auf ein potentielles Übernahmeszenario vorbereitet wird. Durch mehrere Studien wurden verschiedene Visualisierungen in 2D, stereoskopisches 3D und augmentierter Realität implementiert, die Szenen außerhalb des Sichtfelds des Fahrers darstellen. Diese Visualisierungen verbessern das Situationsbewusstsein über kritische Szenarien in einiger Entfernung während eines Übernahmeszenarios. Im dritten Teil werden Erklärungen für Situationen gestaltet, in welchen das Fahrzeug ein unerwartetes Fahrmanöver ausführt. Der Grund des Fahrmanövers ist dem Fahrer dabei unbekannt. Mit intelligenter Infrastruktur verbundene Fahrzeuge erhalten Informationen, die außerhalb des Sichtfelds des Fahrers liegen oder von der Cloud bereit gestellt werden. Dadurch könnte der Grund für das unerwartete Fahrverhalten unklar für den Fahrer sein. Daher werden die Bedürfnisse des Fahrers in diesen Situationen erforscht und Empfehlungen für die Gestaltung einer Schnittstelle, die Erklärungen für das unerwartete Fahrverhalten zur Verfügung stellt, abgeleitet. Zusammenfassend wird gezeigt wie Daten der Infrastruktur und Informationen von verbundenen Fahrzeugen in die Nutzerschnittstelle des Fahrzeugs implementiert werden können. Zudem wird aufgezeigt, wie innovative Technologien wie AR Brillen, die Wahrnehmung der Umgebung des Fahrers verbessern können. Durch diese Dissertation werden Fragen über Anwendungsfälle für die Integration von Umgebungsinformationen in Fahrzeugen beantwortet. Drei wichtige Themengebiete wurden untersucht, welche bei der Betrachtung von Anwendungsfällen der intelligenten Infrastruktur essentiell sind. Durch diese Arbeit wird die Gestaltung innovativer Interaktionen ermöglicht, Einblicke in Visualisierungen von Informationen außerhalb des Sichtfelds des Fahrers gegeben und es wird untersucht, wie Erklärungen für unerwartete Fahrsituationen gestaltet werden können