Human-Centric Detection and Mitigation Approach for Various Levels of Cell Phone-Based Driver Distractions

abstract: Driving a vehicle is a complex task that typically requires several physical interactions and mental tasks. Inattentive driving takes a driver’s attention away from the primary task of driving, which can endanger the safety of driver, passenger(s), as well as pedestrians. According to several traffic safety administration organizations, distracted and inattentive driving are the primary causes of vehicle crashes or near crashes. In this research, a novel approach to detect and mitigate various levels of driving distractions is proposed. This novel approach consists of two main phases: i.) Proposing a system to detect various levels of driver distractions (low, medium, and high) using a machine learning techniques. ii.) Mitigating the effects of driver distractions through the integration of the distracted driving detection algorithm and the existing vehicle safety systems. In phase- 1, vehicle data were collected from an advanced driving simulator and a visual based sensor (webcam) for face monitoring. In addition, data were processed using a machine learning algorithm and a head pose analysis package in MATLAB. Then the model was trained and validated to detect different human operator distraction levels. In phase 2, the detected level of distraction, time to collision (TTC), lane position (LP), and steering entropy (SE) were used as an input to feed the vehicle safety controller that provides an appropriate action to maintain and/or mitigate vehicle safety status. The integrated detection algorithm and vehicle safety controller were then prototyped using MATLAB/SIMULINK for validation. A complete vehicle power train model including the driver’s interaction was replicated, and the outcome from the detection algorithm was fed into the vehicle safety controller. The results show that the vehicle safety system controller reacted and mitigated the vehicle safety status-in closed loop real-time fashion. The simulation results show that the proposed approach is efficient, accurate, and adaptable to dynamic changes resulting from the driver, as well as the vehicle system. This novel approach was applied in order to mitigate the impact of visual and cognitive distractions on the driver performance.Dissertation/ThesisDoctoral Dissertation Applied Psychology 201

Efficient and Robust Driver Fatigue Detection Framework Based on the Visual Analysis of Eye States

Fatigue detection based on vision is widely employed in vehicles due to its real-time and reliable detection results. With the coronavirus disease (COVID-19) outbreak, many proposed detection systems based on facial characteristics would be unreliable due to the face covering with the mask. In this paper, we propose a robust visual-based fatigue detection system for monitoring drivers, which is robust regarding the coverings of masks, changing illumination and head movement of drivers. Our system has three main modules: face key point alignment, fatigue feature extraction and fatigue measurement based on fused features. The innovative core techniques are described as follows: (1) a robust key point alignment algorithm by fusing global face information and regional eye information, (2) dynamic threshold methods to extract fatigue characteristics and (3) a stable fatigue measurement based on fusing percentage of eyelid closure (PERCLOS) and proportion of long closure duration blink (PLCDB). The excellent performance of our proposed algorithm and methods are verified in experiments. The experimental results show that our key point alignment algorithm is robust to different scenes, and the performance of our proposed fatigue measurement is more reliable due to the fusion of PERCLOS and PLCDB

Computing driver tiredness and fatigue in automobile via eye tracking and body movements

The aim of this paper is to classify the driver tiredness and fatigue in automobile via eye tracking and body movements using deep learning based Convolutional Neural Network (CNN) algorithm. Vehicle driver face localization serves as one of the most widely used real-world applications in fields like toll control, traffic accident scene analysis, and suspected vehicle tracking. The research proposed a CNN classifier for simultaneously localizing the region of human face and eye positioning. The classifier, rather than bounding rectangles, gives bounding quadrilaterals, which gives a more precise indication for vehicle driver face localization. The adjusted regions are preprocessed to remove noise and passed to the CNN classifier for real time processing. The preprocessing of the face features extracts connected components, filters them by size, and groups them into face expressions. The employed CNN is the well-known technology for human face recognition. One we aim to extract the facial landmarks from the frames, we will then leverage classification models and deep learning based convolutional neural networks that predict the state of the driver as 'Alert' or 'Drowsy' for each of the frames extracted. The CNN model could predict the output state labels (Alert/Drowsy) for each frame, but we wanted to take care of sequential image frames as that is extremely important while predicting the state of an individual. The process completes, if all regions have a sufficiently high score or a fixed number of retries are exhausted. The output consists of the detected human face type, the list of regions including the extracted mouth and eyes with recognition reliability through CNN with an accuracy of 98.57% with 100 epochs of training and testing

DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis

Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS), especially after the recent success of Deep Learning (DL) methods. The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development, crucial for the transition of automated driving from SAE Level-2 to SAE Level-3. In this paper, we introduce the Driver Monitoring Dataset (DMD), an extensive dataset which includes real and simulated driving scenarios: distraction, gaze allocation, drowsiness, hands-wheel interaction and context data, in 41 hours of RGB, depth and IR videos from 3 cameras capturing face, body and hands of 37 drivers. A comparison with existing similar datasets is included, which shows the DMD is more extensive, diverse, and multi-purpose. The usage of the DMD is illustrated by extracting a subset of it, the dBehaviourMD dataset, containing 13 distraction activities, prepared to be used in DL training processes. Furthermore, we propose a robust and real-time driver behaviour recognition system targeting a real-world application that can run on cost-efficient CPU-only platforms, based on the dBehaviourMD. Its performance is evaluated with different types of fusion strategies, which all reach enhanced accuracy still providing real-time response.Comment: Accepted to ECCV 2020 workshop - Assistive Computer Vision and Robotic

Driver behavior classification and lateral control for automobile safety systems

Advanced driver assistance systems (ADAS) have been developed to help drivers maintain stability, improve road safety, and avoid potential collision. The data acquisition equipment that can be used to measure the state and parameter information of the vehicle may not be available for a standard passenger car due to economical and technical limitations. This work focuses on developing three technologies (longitudinal tire force estimation, driver behavior classification and lateral control) using low-cost sensors that can be utilized in ADAS. For the longitudinal tire force estimation, a low cost 1Hz positioning global system (GPS) and a steering angle sensor are used as the vehicle data acquisition equipment. A nonlinear extended two-wheel vehicle dynamic model is employed. The sideslip angle and the yaw rate are estimated by discrete Kalman Filter. A time independent piecewise optimization scheme is proposed to provide time-continuous estimates of longitude tire force, which can be transferred to the throttle/brake pedal position. The proposed method can be validated by the estimation results. Driver behavior classification systems can detect unsafe driver behavior and avoid potentially dangerous situations. To realize this strategy, a machine learning classification method, Gaussian Mixture model (GMM), is applied to classify driver behavior. In this application, a low cost 1Hz GPS receiver is considered as the vehicle data acquisition equipment instead of other more costly sensors (such as steering angle sensor, throttle/brake position sensor, and etc.). Since the driving information is limited, the nonlinear extended two-wheel vehicle dynamic model is adopted to reconstruct the driver behavior. Firstly, the sideslip angle and the yaw rate are calculated since they are not available from the GPS measurements. Secondly, a piecewise optimization scheme is proposed to reproduce the steering angle and the longitudinal force. Finally, a GMM classifier is trained to identify abnormal driver behavior. The simulation results demonstrated that the proposed scenario can detect the unsafe driver behavior effectively. The lateral control system developed in this study is a look-down reference system which uses a magnetic sensor at the front bumper to measure the front lateral displacement and a GPS to measure the vehicle\u27s heading orientation. Firstly, the steering angles can be estimated by using the data provided by the front magnetic sensor and GPS. The estimation algorithm is an observer for a new extended single-track model, in which the steering angle and its derivative are viewed as two state variables. Secondly, the road curvature is determined based on the linear relationship with respect to the steering angle. Thirdly, an accurate and real-time estimation of the vehicle\u27s lateral displacements can be accomplished according to a state observer. Finally, the closed loop controller is used as a compensator for automated steering. The proposed estimation and control algorithms are validated by simulation results. The results showed that this lateral steering control system achieved a good and robust performance for vehicles following or tracking a reference path

CAT-CAD: A Computer-Aided Diagnosis Tool for Cataplexy

Narcolepsy with cataplexy is a severe lifelong disorder characterized, among others, by sudden loss of bilateral face muscle tone triggered by emotions (cataplexy). A recent approach for the diagnosis of the disease is based on a completely manual analysis of video recordings of patients undergoing emotional stimulation made on-site by medical specialists, looking for specific facial behavior motor phenomena. We present here the CAT-CAD tool for automatic detection of cataplexy symptoms, with the double aim of (1) supporting neurologists in the diagnosis/monitoring of the disease and (2) facilitating the experience of patients, allowing them to conduct video recordings at home. CAT-CAD includes a front-end medical interface (for the playback/inspection of patient recordings and the retrieval of videos relevant to the one currently played) and a back-end AI-based video analyzer (able to automatically detect the presence of disease symptoms in the patient recording). Analysis of patients’ videos for discovering disease symptoms is based on the detection of facial landmarks, and an alternative implementation of the video analyzer, exploiting deep-learning techniques, is introduced. Performance of both approaches is experimentally evaluated using a benchmark of real patients’ recordings, demonstrating the effectiveness of the proposed solutions