Human-Centric Detection and Mitigation Approach for Various Levels of Cell Phone-Based Driver Distractions

abstract: Driving a vehicle is a complex task that typically requires several physical interactions and mental tasks. Inattentive driving takes a driver’s attention away from the primary task of driving, which can endanger the safety of driver, passenger(s), as well as pedestrians. According to several traffic safety administration organizations, distracted and inattentive driving are the primary causes of vehicle crashes or near crashes. In this research, a novel approach to detect and mitigate various levels of driving distractions is proposed. This novel approach consists of two main phases: i.) Proposing a system to detect various levels of driver distractions (low, medium, and high) using a machine learning techniques. ii.) Mitigating the effects of driver distractions through the integration of the distracted driving detection algorithm and the existing vehicle safety systems. In phase- 1, vehicle data were collected from an advanced driving simulator and a visual based sensor (webcam) for face monitoring. In addition, data were processed using a machine learning algorithm and a head pose analysis package in MATLAB. Then the model was trained and validated to detect different human operator distraction levels. In phase 2, the detected level of distraction, time to collision (TTC), lane position (LP), and steering entropy (SE) were used as an input to feed the vehicle safety controller that provides an appropriate action to maintain and/or mitigate vehicle safety status. The integrated detection algorithm and vehicle safety controller were then prototyped using MATLAB/SIMULINK for validation. A complete vehicle power train model including the driver’s interaction was replicated, and the outcome from the detection algorithm was fed into the vehicle safety controller. The results show that the vehicle safety system controller reacted and mitigated the vehicle safety status-in closed loop real-time fashion. The simulation results show that the proposed approach is efficient, accurate, and adaptable to dynamic changes resulting from the driver, as well as the vehicle system. This novel approach was applied in order to mitigate the impact of visual and cognitive distractions on the driver performance.Dissertation/ThesisDoctoral Dissertation Applied Psychology 201

Detection of Driver Drowsiness and Distraction Using Computer Vision and Machine Learning Approaches

Drowsiness and distracted driving are leading factor in most car crashes and near-crashes. This research study explores and investigates the applications of both conventional computer vision and deep learning approaches for the detection of drowsiness and distraction in drivers. In the first part of this MPhil research study conventional computer vision approaches was studied to develop a robust drowsiness and distraction system based on yawning detection, head pose detection and eye blinking detection. These algorithms were implemented by using existing human crafted features. Experiments were performed for the detection and classification with small image datasets to evaluate and measure the performance of system. It was observed that the use of human crafted features together with a robust classifier such as SVM gives better performance in comparison to previous approaches. Though, the results were satisfactorily, there are many drawbacks and challenges associated with conventional computer vision approaches, such as definition and extraction of human crafted features, thus making these conventional algorithms to be subjective in nature and less adaptive in practice. In contrast, deep learning approaches automates the feature selection process and can be trained to learn the most discriminative features without any input from human. In the second half of this research study, the use of deep learning approaches for the detection of distracted driving was investigated. It was observed that one of the advantages of the applied methodology and technique for distraction detection includes and illustrates the contribution of CNN enhancement to a better pattern recognition accuracy and its ability to learn features from various regions of a human body simultaneously. The comparison of the performance of four convolutional deep net architectures (AlexNet, ResNet, MobileNet and NASNet) was carried out, investigated triplet training and explored the impact of combining a support vector classifier (SVC) with a trained deep net. The images used in our experiments with the deep nets are from the State Farm Distracted Driver Detection dataset hosted on Kaggle, each of which captures the entire body of a driver. The best results were obtained with the NASNet trained using triplet loss and combined with an SVC. It was observed that one of the advantages of deep learning approaches are their ability to learn discriminative features from various regions of a human body simultaneously. The ability has enabled deep learning approaches to reach accuracy at human level.

Application of big data in transportation safety analysis using statistical and deep learning methods

The emergence of new sensors and data sources provides large scale high-resolution big data from instantaneous vehicular movements, driver decision and states, surrounding environment, roadway characteristics, weather condition, etc. Such a big data can be served to expand our understanding regarding the current state of the transportation and help us to proactively evaluate and monitor the system performance. The key idea behind this dissertation is to identify the moments and locations where drivers are exhibiting different behavior comparing to the normal behavior. The concept of driving volatility is utilized which quantifies deviation from normal driving in terms of variations in speed, acceleration/deceleration, and vehicular jerk. This idea is utilized to explore the association of volatility in different hierarchies of transportation system, i.e.: 1) Instance level; 2) Event level; 3) Driver level; 4) Intersection level; and 5) Network level. In summary, the main contribution of this dissertation is exploring the association of variations in driving behavior in terms of driving volatility at different levels by harnessing big data generated from emerging data sources under real-world condition, which is applicable to the intelligent transportation systems and smart cities. By analyzing real-world crashes/near-crashes and predicting occurrence of extreme event, proactive warnings and feedback can be generated to warn drivers and adjacent vehicles regarding potential hazard. Furthermore, the results of this study help agencies to proactively monitor and evaluate safety performance of the network and identify locations where crashes are waiting to happen. The main objective of this dissertation is to integrate big data generated from emerging sources into safety analysis by considering different levels in the system. To this end, several data sources including Connected Vehicles data (with more than 2.2 billion seconds of observations), naturalistic driving data (with more than 2 million seconds of observations from vehicular kinematics and driver behavior), conventional data on roadway factors and crash data are integrated

SAFECAR: A Brain–Computer Interface and intelligent framework to detect drivers’ distractions

As recently reported by the World Health Organization (WHO), the high use of intelligent devices such as smartphones, multimedia systems, or billboards causes an increase in distraction and, consequently, fatal accidents while driving. The use of EEG-based Brain–Computer Interfaces (BCIs) has been proposed as a promising way to detect distractions. However, existing solutions are not well suited for driving scenarios. They do not consider complementary data sources, such as contextual data, nor guarantee realistic scenarios with real-time communications between components. This work proposes an automatic framework for detecting distractions using BCIs and a realistic driving simulator. The framework employs different supervised Machine Learning (ML)-based models on classifying the different types of distractions using Electroencephalography (EEG) and contextual driving data collected by car sensors, such as line crossings or objects detection. This framework has been evaluated using a driving scenario without distractions and a similar one where visual and cognitive distractions are generated for ten subjects. The proposed framework achieved 83.9% -score with a binary model and 73% with a multiclass model using EEG, improving 7% in binary classification and 8% in multi-class classification by incorporating contextual driving into the training dataset. Finally, the results were confirmed by a neurophysiological study, which revealed significantly higher voltage in selective attention and multitasking

Prediction of drivers’ performance in highly automated vehicles

Purpose: The aim of this research was to assess the predictability of driver’s response to critical hazards during the transition from automated to manual driving in highly automated vehicles using their physiological data.Method: A driving simulator experiment was conducted to collect drivers’ physiological data before, during and after the transition from automated to manual driving. A total of 33 participants between 20 and 30 years old were recruited. Participants went through a driving scenario under the influence of different non-driving related tasks. The repeated measures approach was used to assess the effect of repeatability on the driver’s physiological data. Statistical and machine learning methods were used to assess the predictability of drivers’ response quality based on their physiological data collected before responding to a critical hazard. Findings: - The results showed that the observed physiological data that was gathered before the transition formed strong indicators of the drivers’ ability to respond successfully to a potential hazard after the transition. In addition, physiological behaviour was influenced by driver’s secondary tasks engagement and correlated with the driver’s subjective measures to the difficulty of the task. The study proposes new quality measures to assess the driver’s response to critical hazards in highly automated driving. Machine learning results showed that response time is predictable using regression methods. In addition, the classification methods were able to classify drivers into low, medium and high-risk groups based on their quality measures values. Research Implications: Proposed models help increase the safety of automated driving systems by providing insights into the drivers’ ability to respond to future critical hazards. More research is required to find the influence of age, drivers’ experience of the automated vehicles and traffic density on the stability of the proposed models. Originality: The main contribution to knowledge of this study is the feasibility of predicting drivers’ ability to respond to critical hazards using the physiological behavioural data collected before the transition from automated to manual driving. With the findings, automation systems could change the transition time based on the driver’s physiological state to allow for the safest transition possible. In addition, it provides an insight into driver’s readiness and therefore, allows the automated system to adopt the correct driving strategy and plan to enhance drivers experience and make the transition phase safer for everyone.</div

Improving Drivers’ Behaviour When Partial Driving Automation Fails

With the advent of automated vehicle systems, the role of drivers has changed to a more supervisory role. However, it is known that all vehicles with Level 2 (L2) systems have a very specific operational design domain (ODD) and can only function on limited conditions. Hence, it is important for drivers to perceive the situations properly and regain the control from the L2 system when needed. As suggested by past research, designing an informative interface could help drivers in their new supervision and intervention role while driving with L2 vehicles by providing feedback to drivers when hazards or event that may cause system failure are detected. On the other hand there are many situations where these vehicles cannot detect hazards and provide any feedback prior to the event. In these cases, training programs which provide drivers with an experience of these system limitations and allow them to practice dealing with such limitations can prove to be effective countermeasures. The objective of the current study is to employ different methods (designing HMI and training drivers) to increase drivers’ situational awareness regarding operational design domain (ODD) and improve drivers performance in transfer of control situations while driving with level 2 (L2) automation features. This study includes two experiments- in first experiment, an informative dashboard interface was designed and tested through three phases (observation, prototyping, testing). Results from the testing phase showed that drivers who received the newly designed dashboards took back control more effectively and had more situational awareness compared to the control group. In the second experiment, a PC-based training program was designed and tested to improve drivers takeover response and situational awareness when L2 systems reach their ODD limits. Results showed drivers in the PC-based training group took back control more effectively when L2 systems reached their ODD limits and had more situational awareness compared to the drivers who received user manual or placebo training

Modeling driver distraction mechanism and its safety impact in automated vehicle environment.

Automated Vehicle (AV) technology expects to enhance driving safety by eliminating human errors. However, driver distraction still exists under automated driving. The Society of Automotive Engineers (SAE) has defined six levels of driving automation from Level 0~5. Until achieving Level 5, human drivers are still needed. Therefore, the Human-Vehicle Interaction (HVI) necessarily diverts a driver’s attention away from driving. Existing research mainly focused on quantifying distraction in human-operated vehicles rather than in the AV environment. It causes a lack of knowledge on how AV distraction can be detected, quantified, and understood. Moreover, existing research in exploring AV distraction has mainly pre-defined distraction as a binary outcome and investigated the patterns that contribute to distraction from multiple perspectives. However, the magnitude of AV distraction is not accurately quantified. Moreover, past studies in quantifying distraction have mainly used wearable sensors’ data. In reality, it is not realistic for drivers to wear these sensors whenever they drive. Hence, a research motivation is to develop a surrogate model that can replace the wearable device-based data to predict AV distraction. From the safety perspective, there lacks a comprehensive understanding of how AV distraction impacts safety. Furthermore, a solution is needed for safely offsetting the impact of distracted driving. In this context, this research aims to (1) improve the existing methods in quantifying Human-Vehicle Interaction-induced (HVI-induced) driver distraction under automated driving; (2) develop a surrogate driver distraction prediction model without using wearable sensor data; (3) quantitatively reveal the dynamic nature of safety benefits and collision hazards of HVI-induced visual and cognitive distractions under automated driving by mathematically formulating the interrelationships among contributing factors; and (4) propose a conceptual prototype of an AI-driven, Ultra-advanced Collision Avoidance System (AUCAS-L3) targeting HVI-induced driver distraction under automated driving without eye-tracking and video-recording. Fixation and pupil dilation data from the eye tracking device are used to model driver distraction, focusing on visual and cognitive distraction, respectively. In order to validate the proposed methods for measuring and modeling driver distraction, a data collection was conducted by inviting drivers to try out automated driving under Level 3 automation on a simulator. Each driver went through a jaywalker scenario twice, receiving a takeover request under two types of HVI, namely “visual only” and “visual and audible”. Each driver was required to wear an eye-tracker so that the fixation and pupil dilation data could be collected when driving, along with driving performance data being recorded by the simulator. In addition, drivers’ demographical information was collected by a pre-experiment survey. As a result, the magnitude of visual and cognitive distraction was quantified, exploring the dynamic changes over time. Drivers are more concentrated and maintain a higher level of takeover readiness under the “visual and audible” warning, compared to “visual only” warning. The change of visual distraction was mathematically formulated as a function of time. In addition, the change of visual distraction magnitude over time is explained from the driving psychology perspective. Moreover, the visual distraction was also measured by direction in this research, and hotspots of visual distraction were identified with regard to driving safety. When discussing the cognitive distraction magnitude, the driver’s age was identified as a contributing factor. HVI warning type contributes to the significant difference in cognitive distraction acceleration rate. After drivers reach the maximum visual distraction, cognitive distraction tends to increase continuously. Also, this research contributes to quantitatively revealing how visual and cognitive distraction impacts the collision hazards, respectively. Moreover, this research contributes to the literature by developing deep learning-based models in predicting a driver’s visual and cognitive distraction intensity, focusing on demographics, HVI warning types, and driving performance. As a solution to safety issues caused by driver distraction, the AUCAS-L3 has been proposed. The AUCAS-L3 is validated with high accuracies in predicting (a) whether a driver is distracted and does not perform takeover actions and (b) whether crashes happen or not if taken over. After predicting the presence of driver distraction or a crash, AUCAS-L3 automatically applies the brake pedal for drivers as effective and efficient protection to driver distraction under automated driving. And finally, a conceptual prototype in predicting AV distraction and traffic conflict was proposed, which can predict the collision hazards in advance of 0.82 seconds on average