Efficient and Robust Driver Fatigue Detection Framework Based on the Visual Analysis of Eye States

Fatigue detection based on vision is widely employed in vehicles due to its real-time and reliable detection results. With the coronavirus disease (COVID-19) outbreak, many proposed detection systems based on facial characteristics would be unreliable due to the face covering with the mask. In this paper, we propose a robust visual-based fatigue detection system for monitoring drivers, which is robust regarding the coverings of masks, changing illumination and head movement of drivers. Our system has three main modules: face key point alignment, fatigue feature extraction and fatigue measurement based on fused features. The innovative core techniques are described as follows: (1) a robust key point alignment algorithm by fusing global face information and regional eye information, (2) dynamic threshold methods to extract fatigue characteristics and (3) a stable fatigue measurement based on fusing percentage of eyelid closure (PERCLOS) and proportion of long closure duration blink (PLCDB). The excellent performance of our proposed algorithm and methods are verified in experiments. The experimental results show that our key point alignment algorithm is robust to different scenes, and the performance of our proposed fatigue measurement is more reliable due to the fusion of PERCLOS and PLCDB

On validating a generic camera-based blink detection system for cognitive load assessment

Detecting the human operator\u27s cognitive state is paramount in settings wherein maintaining optimal workload is necessary for task performance. Blink rate is an established metric of cognitive load, with a higher blink frequency being observed under conditions of greater workload. Measuring blink rate requires the use of eye-trackers which limits the adoption of this metric in the real-world. The authors aim to investigate the effectiveness of using a generic camera-based system as a way to assess the user\u27s cognitive load during a computer task. Participants completed a mental task while sitting in front of a computer. Blink rate was recorded via both the generic camera-based system and a scientific-grade eye-tracker for validation purposes. Cognitive load was also assessed through the performance in a single stimulus detection task. The blink rate recorded via the generic camera-based approach did not differ from the one obtained through the eye-tracker. No meaningful changes in blink rate were however observed with increasing cognitive load. Results show the generic-camera based system may represent a more affordable, ubiquitous means for assessing cognitive workload during computer task. Future work should further investigate ways to increase its accuracy during the completion of more realistic tasks

Estimating Level of Engagement from Ocular Landmarks

E-learning offers many advantages like being economical, flexible and customizable, but also has challenging aspects such as lack of – social-interaction, which results in contemplation and sense of remoteness. To overcome these and sustain learners’ motivation, various stimuli can be incorporated. Nevertheless, such adjustments initially require an assessment of engagement level. In this respect, we propose estimating engagement level from facial landmarks exploiting the facts that (i) perceptual decoupling is promoted by blinking during mentally demanding tasks; (ii) eye strain increases blinking rate, which also scales with task disengagement; (iii) eye aspect ratio is in close connection with attentional state and (iv) users’ head position is correlated with their level of involvement. Building empirical models of these actions, we devise a probabilistic estimation framework. Our results indicate that high and low levels of engagement are identified with considerable accuracy, whereas medium levels are inherently more challenging, which is also confirmed by inter-rater agreement of expert coders

A study on tiredness assessment by using eye blink detection

In this paper, the loss of attention of automotive drivers is studied by using eye blink detection. Facial landmark detection for detecting eye is explored. Afterward, eye blink is detected using Eye Aspect Ratio. By comparing the time of eye closure to a particular period, the driver’s tiredness is decided. The total number of eye blinks in a minute is counted to detect drowsiness. Calculation of total eye blinks in a minute for the driver is done, then compared it with a known standard value. If any of the above conditions fulfills, the system decides the driver is unconscious. A total of 120 samples were taken by placing the light source front, back, and side. There were 40 samples for each position of the light source. The maximum error rate occurred when the light source was placed back with a 15% error rate. The best scenario was 7.5% error rate where the light source was placed front side. The eye blinking process gave an average error of 11.67% depending on the various position of the light source. Another 120 samples were taken at a different time of the day for calculating total eye blink in a minute. The maximum number of blinks was in the morning with an average blink rate of 5.78 per minute, and the lowest number of blink rate was in midnight with 3.33% blink rate. The system performed satisfactorily and achieved the eye blink pattern with 92.7% accuracy

Estimating Level of Engagement from Ocular Landmarks

E-learning offers many advantages like being economical, flexible and customizable, but also has challenging aspects such as lack of – social-interaction, which results in contemplation and sense of remoteness. To overcome these and sustain learners’ motivation, various stimuli can be incorporated. Nevertheless, such adjustments initially require an assessment of engagement level. In this respect, we propose estimating engagement level from facial landmarks exploiting the facts that (i) perceptual decoupling is promoted by blinking during mentally demanding tasks; (ii) eye strain increases blinking rate, which also scales with task disengagement; (iii) eye aspect ratio is in close connection with attentional state and (iv) users’ head position is correlated with their level of involvement. Building empirical models of these actions, we devise a probabilistic estimation framework. Our results indicate that high and low levels of engagement are identified with considerable accuracy, whereas medium levels are inherently more challenging, which is also confirmed by inter-rater agreement of expert coders

Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences

The research on driver fatigue detection is of great significance to improve driving safety. This paper proposes a real-time comprehensive driver fatigue detection algorithm based on facial landmarks to improve the detection accuracy, which detects the driver’s fatigue status by using facial video sequences without equipping their bodies with other intelligent devices. A tasks-constrained deep convolutional network is constructed to detect the face region based on 68 key points, which can solve the optimization problem caused by the different convergence speeds of each task. According to the real-time facial video images, the eye feature of the eye aspect ratio (EAR), mouth aspect ratio (MAR) and percentage of eye closure time (PERCLOS) are calculated based on facial landmarks. A comprehensive driver fatigue assessment model is established to assess the fatigue status of drivers through eye/mouth feature selection. After a series of comparative experiments, the results show that this proposed algorithm achieves good performance in both accuracy and speed for driver fatigue detection.</jats:p