Computing driver tiredness and fatigue in automobile via eye tracking and body movements

The aim of this paper is to classify the driver tiredness and fatigue in automobile via eye tracking and body movements using deep learning based Convolutional Neural Network (CNN) algorithm. Vehicle driver face localization serves as one of the most widely used real-world applications in fields like toll control, traffic accident scene analysis, and suspected vehicle tracking. The research proposed a CNN classifier for simultaneously localizing the region of human face and eye positioning. The classifier, rather than bounding rectangles, gives bounding quadrilaterals, which gives a more precise indication for vehicle driver face localization. The adjusted regions are preprocessed to remove noise and passed to the CNN classifier for real time processing. The preprocessing of the face features extracts connected components, filters them by size, and groups them into face expressions. The employed CNN is the well-known technology for human face recognition. One we aim to extract the facial landmarks from the frames, we will then leverage classification models and deep learning based convolutional neural networks that predict the state of the driver as 'Alert' or 'Drowsy' for each of the frames extracted. The CNN model could predict the output state labels (Alert/Drowsy) for each frame, but we wanted to take care of sequential image frames as that is extremely important while predicting the state of an individual. The process completes, if all regions have a sufficiently high score or a fixed number of retries are exhausted. The output consists of the detected human face type, the list of regions including the extracted mouth and eyes with recognition reliability through CNN with an accuracy of 98.57% with 100 epochs of training and testing

Drowsy Driver Detection System

Driver weariness is one of the key causes of road mishaps in the world. Detecting the drowsiness of the driver can be one of the surest ways of quantifying driver fatigue. In this project we aim to develop an archetype drowsiness detection system. This mechanism works by monitoring the eyes of the driver and sounding an alarm when he/she feels heavy eyed. The system so constructed is a non-intrusive real-time observing system. The primacy is on improving the safety of the driver. In this mechanism the eye blink of the driver is detected. If the driver’s eyes remain closed for more than a certain span of time, the driver is believed to be tired and an alarm is sounded. The programming for this is carried out in OpenCV using the Haar cascade library for the detection of facial features

A systematic review of physiological signals based driver drowsiness detection systems.

Driving a vehicle is a complex, multidimensional, and potentially risky activity demanding full mobilization and utilization of physiological and cognitive abilities. Drowsiness, often caused by stress, fatigue, and illness declines cognitive capabilities that affect drivers' capability and cause many accidents. Drowsiness-related road accidents are associated with trauma, physical injuries, and fatalities, and often accompany economic loss. Drowsy-related crashes are most common in young people and night shift workers. Real-time and accurate driver drowsiness detection is necessary to bring down the drowsy driving accident rate. Many researchers endeavored for systems to detect drowsiness using different features related to vehicles, and drivers' behavior, as well as, physiological measures. Keeping in view the rising trend in the use of physiological measures, this study presents a comprehensive and systematic review of the recent techniques to detect driver drowsiness using physiological signals. Different sensors augmented with machine learning are utilized which subsequently yield better results. These techniques are analyzed with respect to several aspects such as data collection sensor, environment consideration like controlled or dynamic, experimental set up like real traffic or driving simulators, etc. Similarly, by investigating the type of sensors involved in experiments, this study discusses the advantages and disadvantages of existing studies and points out the research gaps. Perceptions and conceptions are made to provide future research directions for drowsiness detection techniques based on physiological signals. [Abstract copyright: © The Author(s), under exclusive licence to Springer Nature B.V. 2022. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Detecting Drowsy Learners at the Wheel of e-Learning Platforms with Multimodal Learning Analytics

Learners are expected to stay wakeful and focused while interacting with e-learning platforms. Although wakefulness of learners strongly relates to educational outcomes, detecting drowsy learning behaviors only from log data is not an easy task. In this study, we describe the results of our research to model learners’ wakefulness based on multimodal data generated from heart rate, seat pressure, and face recognition. We collected multimodal data from learners in a blended course of informatics and conducted two types of analysis on them. First, we clustered features based on learners’ wakefulness labels as generated by human raters and ran a statistical analysis. This analysis helped us generate insights from multimodal data that can be used to inform learner and teacher feedback in multimodal learning analytics. Second, we trained machine learning models with multiclass-Support Vector Machine (SVM), Random Forest (RF) and CatBoost Classifier (CatBoost) algorithms to recognize learners’ wakefulness states automatically. We achieved an average macro-F1 score of 0.82 in automated user-dependent models with CatBoost. We also showed that compared to unimodal data from each sensor, the multimodal sensor data can improve the accuracy of models predicting the wakefulness states of learners while they are interacting with e-learning platforms

Contactless privacy-preserving head movement recognition using deep learning for driver fatigue detection

Head movement holds significant importance in con-veying body language, expressing specific gestures, and reflecting emotional and character aspects. The detection of head movement in smart or assistive driving applications can play an important role in preventing major accidents and potentially saving lives. Additionally, it aids in identifying driver fatigue, a significant contributor to deadly road accidents worldwide. However, most existing head movement detection systems rely on cameras, which raise privacy concerns, face challenges with lighting conditions, and require complex training with long video sequences. This novel privacy-preserving system utilizes UWB-radar technology and leverages Deep Learning (DL) techniques to address the mentioned issues. The system focuses on classifying the five most common head gestures: Head 45L (HL45), Head 45R (HR45), Head 90L (HL90), Head 90R (HR90), and Head Down (HD). By processing the recorded data as spectrograms and leveraging the advanced DL model VGG16, the proposed system accurately detects these head gestures, achieving a maximum classification accuracy of 84.00% across all classes. This study presents a proof of concept for an effective and privacy-conscious approach to head position classification.</p

Drivers’ drowsiness detection based on an optimized random forest classification and single-channel electroencephalogram

The state of functioning (posture) of a driver at the wheel of a car involves a complex set of psychological, physiological, and physical parameters. This combination induces fatigue, which manifests itself in repeated yawning, stinging eyes, a frozen gaze, a stiff and painful neck, back pain, and other signs. The driver may fight fatigue for a few moments, but it inevitably leads to drowsiness, periods of micro-sleep, and then falling asleep. At the first signs of drowsiness, the risk of an accident becomes immense. In Morocco, drowsiness at the wheel is the cause of 1/3 of fatal accidents on the freeways. Thus, in this paper, a new hybrid data analysis and an efficient machine learning algorithm are designed to detect the drowsiness of drivers who spend most of their time behind the wheel over long distances (older than 35 years). This analysis is based on a single channel of electroencephalogram (EEG) recordings using time, frequency fast Fourier transform (FFT), and power spectral density (PSD) analysis. To distinguish between the two states of alertness and drowsiness, several features were extracted from each domain (time, FFT, and PSD), and subjected to different classifier architectures to conduct a general comparison and achieve the highest detection accuracy (98.5%) and best time consumption (13 milliseconds)

Driver drowsiness detection in facial images

Driver fatigue is a significant factor in a large number of vehicle accidents. Thus, drowsy driver alert systems are meant to reduce the main cause of traffic accidents. Different approaches have been developed to tackle with the fatigue detection problem. Though most reliable techniques to asses fatigue involve the use of physical sensors to monitor drivers, they can be too intrusive and are less likely to be adopted by the car industry. A relatively new and effective trend consists on facial image analysis from video cameras that monitor drivers. How to extract effective features of fatigue from images is important for many image processing applications. This project proposes a face descriptor that can be used to detect driver fatigue in static frames. This descriptor represents each frame of a sequence as a pyramid of scaled images that are divided into non-overlapping blocks of equal size. The pyramid of images is combined with three different image descriptors. The final descriptors are filtered out using feature selection and a Support Vector Machine is used to predict the drowsiness state. The proposed method is tested on the public NTHUDDD dataset, which is the state-of-the-art dataset on driver drowsiness detection

Physiological-based Driver Monitoring Systems: A Scoping Review

A physiological-based driver monitoring system (DMS) has attracted research interest and has great potential for providing more accurate and reliable monitoring of the driver’s state during a driving experience. Many driving monitoring systems are driver behavior-based or vehicle-based. When these non-physiological based DMS are coupled with physiological-based data analysis from electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and electromyography (EMG), the physical and emotional state of the driver may also be assessed. Drivers’ wellness can also be monitored, and hence, traffic collisions can be avoided. This paper highlights work that has been published in the past five years related to physiological-based DMS. Specifically, we focused on the physiological indicators applied in DMS design and development. Work utilizing key physiological indicators related to driver identification, driver alertness, driver drowsiness, driver fatigue, and drunk driver is identified and described based on the PRISMA Extension for Scoping Reviews (PRISMA-Sc) Framework. The relationship between selected papers is visualized using keyword co-occurrence. Findings were presented using a narrative review approach based on classifications of DMS. Finally, the challenges of physiological-based DMS are highlighted in the conclusion. Doi: 10.28991/CEJ-2022-08-12-020 Full Text: PD

Driver Drowsiness Detection System

In recent years’ driver fatigue is one of the major causes of vehicle accidents in the world. A direct way of measuring driver fatigue is measuring the state of the driver i.e. drowsiness. So it is very important to detect the drowsiness of the driver to save life and property. This project is aimed towards developing a prototype of drowsiness detection system. This system is a real time system which captures image continuously and measures the state of the eye according to the specified algorithm and gives warning if required. Though there are several methods for measuring the drowsiness but this approach is completely non-intrusive which does not affect the driver in any way, hence giving the exact condition of the driver. For detection of drowsiness the per closure value of eye is considered. So when the closure of eye exceeds a certain amount then the driver is identified to be sleepy. For implementing this system several OpenCv libraries are used including Haar-cascade. The entire system is implemented using Raspberry-Pi

Fatigue Monitoring System

This work provides an innovative solution for monitoring fatigue for users behind workstations. A web camera was adjusted to work in near infrared range and a system of 880 nm IR diodes was implemented to create an IR vision system to localize and track the eye pupils. The software developed monitors and tracks eyes for signs of fatigue by measuring PERCLOS. The software developed runs on the workstation and is designed to draw limited computational power, so as to not interfere with the user task. To overcome low-frame rate imposed by the hardware limitations and to improve real time monitoring, two-phases detection and tacking algorithm is implemented. The proposed system successfully monitors fatigue at a rate of 8 fps. The system is well suited to monitor users in command centers, flight control centers, airport traffic dispatches, military operation and command centers, etc., but the work can be extended to wearable devices and other environments