Driver monitoring system based on eye tracking

Dissertação de mestrado integrado em Engenharia Electrónica Industrial e ComputadoresRecent statistics indicate that driver drowsiness is one of the major causes of road accidents and deaths behind the wheel. This reveals the need of reliable systems capable of predict when drivers are in this state and warn them in order to avoid crashes with other vehicles or stationary objects. Therefore, the purpose of this dissertation is to develop a driver’s monitoring system based on eye tracking that will be able to detect driver’s drowsiness level and actuate accordingly. The alert to the driver may vary from a message on the cluster to a vibration on the seat. The proposed algorithm to estimate driver’s state only requires one variable: eyelid opening. Through this variable the algorithm computes several eye parameters used to decide if the driver is drowsy or not, namely: PERCLOS, blink frequency and blink duration. Eyelid opening is obtained over a software and hardware platform called SmartEye Pro. This eye tracking system uses infrared cameras and computer vision software to gather eye’s state information. Additionally, since this dissertation is part of the project "INNOVATIVE CAR HMI", from Bosch and University of Minho partnership, the driver monitoring system will be integrated in the Bosch DSM (Driver Simulator Mockup).Estatísticas recentes indicam que a sonolência do condutor é uma das principais causas de acidentes e mortes nas estradas. Isto revela a necessidade de sistemas fiáveis capazes de prever quando um condutor está sonolento e avisá-lo, de modo a evitar colisões com outros veículos ou objetos estacionários. Portanto, o propósito desta dissertação é desenvolver um sistema de monitorização do condutor baseado em eye tracking que será capaz de detetar o nível de sonolência do condutor e atuar em conformidade. O alerta para o condutor pode variar entre uma mensagem no painel de instrumentos ou uma vibração no assento. O algoritmo proposto para estimar o estado do condutor apenas requer a aquisição de uma variável: abertura da pálpebra. Através desta variável, o algoritmo computa alguns parâmetros utilizados para verificar se o condutor está sonolento ou não, nomeadamente: PERCLOS, frequência do pestanejar e duração do pestanejar. A abertura da pálpebra é obtida através de uma plataforma de hardware e software chamada SmartEye Pro. Esta plataforma de eye tracking utiliza câmaras infravermelho e software de visão por computador para obter informação sobre o estado dos olhos. Adicionalmente, uma vez que esta dissertação está inserida projeto: "INNOVATIVE CAR HMI", da parceria entre a Bosch e a Universidade do Minho, o sistema desenvolvido será futuramente integrado no Bosch DSM (Driver Simulator Mockup)

Prototype Drowsiness Detection System

Driver fatigue is one of the major causes of accidents in the world. Detecting the drowsiness of the driver is one of the surest ways of measuring driver fatigue. In this project we aim to develop a prototype drowsiness detection system. This system works by monitoring the eyes of the driver and sounding an alarm when he/she is drowsy. The system so designed is a non-intrusive real-time monitoring system. The priority is on improving the safety of the driver without being obtrusive. In this project the eye blink of the driver is detected. If the drivers eyes remain closed for more than a certain period of time, the driver is said to be drowsy and an alarm is sounded. The programming for this is done in OpenCV using the Haarcascade library for the detection of facial features

On validating a generic camera-based blink detection system for cognitive load assessment

Detecting the human operator\u27s cognitive state is paramount in settings wherein maintaining optimal workload is necessary for task performance. Blink rate is an established metric of cognitive load, with a higher blink frequency being observed under conditions of greater workload. Measuring blink rate requires the use of eye-trackers which limits the adoption of this metric in the real-world. The authors aim to investigate the effectiveness of using a generic camera-based system as a way to assess the user\u27s cognitive load during a computer task. Participants completed a mental task while sitting in front of a computer. Blink rate was recorded via both the generic camera-based system and a scientific-grade eye-tracker for validation purposes. Cognitive load was also assessed through the performance in a single stimulus detection task. The blink rate recorded via the generic camera-based approach did not differ from the one obtained through the eye-tracker. No meaningful changes in blink rate were however observed with increasing cognitive load. Results show the generic-camera based system may represent a more affordable, ubiquitous means for assessing cognitive workload during computer task. Future work should further investigate ways to increase its accuracy during the completion of more realistic tasks

Real-time drowsiness detection using wearable, lightweight EEG sensors

Driver drowsiness has always been a major concern for researchers and road use administrators. It has led to countless deaths accounting to significant percentile of deaths world over. Researchers have attempted to determine driver drowsiness using the following measures: (1) subjective measures (2) vehicle-based measures; (3) behavioral measures and (4) physiological measures.;Studies carried out to assess the efficacy of all the four measures, have brought out significant weaknesses in each of these measures. However detailed and comprehensive review has indicated that Physiological Measure namely EEG signal analysis provides most reliable and accurate information on driver drowsiness. In this paper a brief review of systems, and issues associated with them has been discussed with a view to evolve a novel system based on EEG signals especially for use in mine vehicles.;The feasibility of real-time drowsiness detection using commercially available, off-the-shelf, lightweight, wearable EEG sensors is explored. While EEG signals are known to be reliable indicators of fatigue and drowsiness, they have not been used widely due to their size and form factor. But the use of light-weight wearable EEGs alleviates this concern. Spectral analysis of EEG signals from these sensors using support vector machines is shown to classify drowsy states with high accuracy.;The system is validated using data collected on 23 subjects in fresh and drowsy states. The EEG signals are also used to characterize the blink duration and frequency of subjects. However, classification of drowsy states using blink analysis is shown to have lower accuracy than that using spectral analysis

Drowsiness Detection System in Real Time Based on Behavioral Characteristics of Driver using Machine Learning Approach

The process of determining if a person, generally a driver, is becoming sleepy or drowsy while performing a task such as driving is known as drowsiness detection. It is a necessary system for detecting and alerting drivers to their tiredness, which might impair their driving ability and lead to accidents. The project aims to create a reliable and efficient system capable of real-time detection of drowsiness using OpenCV, Dlib, and facial landmark detection technologies. The project's results show that the sleepiness detection method can accurately and precisely identify tiredness in real time. The technology is less intrusive and more economical than conventional sleepiness detection techniques. The system is based on a 68 facial landmark detector, which is a highly trained and effective detector capable of recognizing human face points. The detector aids in assessing whether the driver's eyes are closed or open.  The system analyses the data collected by the detector using machine learning methods to discover patterns associated with drowsiness. When drowsiness is detected, the system incorporates a warning mechanism, such as an alarm or a vibration in the steering wheel, to notify the driver. A variety of studies with different drivers and driving conditions were used to evaluate the performance of the real-time driver drowsiness detection system. The results show that the technology can detect tiredness properly and deliver timely warnings to the driver. This method can assist in preventing drowsy driving incidents, enhancing road safety, and saving lives. The results indicated that the algorithm had an average accuracy rate of 94% for identifying tiredness in drivers

Efficient and Robust Driver Fatigue Detection Framework Based on the Visual Analysis of Eye States

Fatigue detection based on vision is widely employed in vehicles due to its real-time and reliable detection results. With the coronavirus disease (COVID-19) outbreak, many proposed detection systems based on facial characteristics would be unreliable due to the face covering with the mask. In this paper, we propose a robust visual-based fatigue detection system for monitoring drivers, which is robust regarding the coverings of masks, changing illumination and head movement of drivers. Our system has three main modules: face key point alignment, fatigue feature extraction and fatigue measurement based on fused features. The innovative core techniques are described as follows: (1) a robust key point alignment algorithm by fusing global face information and regional eye information, (2) dynamic threshold methods to extract fatigue characteristics and (3) a stable fatigue measurement based on fusing percentage of eyelid closure (PERCLOS) and proportion of long closure duration blink (PLCDB). The excellent performance of our proposed algorithm and methods are verified in experiments. The experimental results show that our key point alignment algorithm is robust to different scenes, and the performance of our proposed fatigue measurement is more reliable due to the fusion of PERCLOS and PLCDB