Drowsiness Detection Based on Yawning Using Modified Pre-trained Model MobileNetV2 and ResNet50

Traffic accidents are fatal events that need special attention. According to research by the National Transportation Safety Committee, 80% of traffic accidents are caused by human error, one of which is tired and drowsy drivers. The brain can interpret the vital fatigue of a drowsy driver sign as yawning. Therefore, yawning detection for preventing drowsy drivers’ imprudent can be developed using computer vision. This method is easy to implement and does not affect the driver when handling a vehicle. The research aimed to detect drowsy drivers based on facial expression changes of yawning by combining the Haar Cascade classifier and a modified pre-trained model, MobileNetV2 and ResNet50. Both proposed models accurately detected real-time images using a camera. The analysis showed that the yawning detection model based on the ResNet50 algorithm is more reliable, with the model obtaining 99% of accuracy. Furthermore, ResNet50 demonstrated reproducible outcomes for yawning detection, considering having good training capabilities and overall evaluation results

Fatigue detection system to aid in remote work

During the Covid-19 pandemic there was a noticeable surge in the amount of remote workers. In the aftermath of the pandemic working from home still remains a reality for many workers with noticeable impacts on the mental health of people. With the increased stress caused by current situation and the harder time establishing boundaries there was an increase in the overall stress and fatigue in workers, leading to burnouts. Fatigue detection systems are used in several areas, mainly in the automotive industry as a mean to decrease the number of accidents. This research started by approaching the Artificial Intelligence (AI) area and its domains, followed by a study of the current techniques used in order to predict fatigue. With the main ones utilising eye state, facial landmarks, electrocardiogram or heart rate. After a research into existing Fatigue detection systems was done in order to identify the strengths of solutions currently in the market, whether in the automotive industry or other applications. This thesis proposes the creation of a system able to detect fatigue in a user as well as warn him when fatigue levels increase. This system incorporates a webcam analysing the users face and performing eye state detection in order to calculate the percentage of the time the eyes are closed (PERCLOS). Heart rate data was also analysed and a model was developed in order to incorporate this data, the percentage of time the eyes are closed, the program the user has open and time of day in order to predict the level of fatigue. By combining these two different techniques this system can be more effective and more accurate in giving predictions of the level of fatigue. The review of literature showed that the conjunction of these two techniques in predicting fatigue is novelty. The developed system also contains integration with smartwatch technology in order to both harness heart rate data as well as communicate with the user via pop up notifications to inform him when fatigue levels get too high. The conclusion of this work is that eye state detection using Artificial Intelligence can achieve a high accuracy and be a reliable tool in identifying fatigue in an user. The combination of Heart Rate and PERCLOS allows the system to have a higher accuracy as well as not being completely reliant on one sensor. The creation of a fatigue prediction model was hindered by the lack of existent data in order to train a model, a problem that could be fixed with the adoption of the system in a broader scope.Durante a pandemia de Covid-19, houve um aumento notável na quantidade de trabalhadores remotos. No rescaldo da pandemia, trabalhar a partir de casa continua a ser uma realidade para muitos trabalhadores, com impactos visíveis na saúde mental das pessoas. Com o aumento do stresse causado pela situação atual e a dificuldade de estabelecer limites, houve um aumento do stresse geral e da fadiga dos trabalhadores, levando ao esgotamento. Os sistemas de detecção de fadiga são utilizados em diversas áreas, principalmente na indústria automobilística como forma de diminuir o número de acidentes. Este estudo começou por abordar a área de Inteligência Artificial (IA) e os seus domínios, seguida de um estudo das técnicas atuais utilizadas para prever a fadiga. Com os principais utilizando o estado dos olhos, pontos de referência faciais, eletrocardiograma ou frequência cardíaca. Depois foi feita uma pesquisa sobre os sistemas de detecção de fadiga existentes de forma a identificar os pontos fortes das soluções actualmente no mercado, quer seja na indústria automóvel ou outras aplicações. Esta dissertação propõe a criação de um sistema capaz de detectar fadiga num utilizador, bem como alertar quando os níveis de fadiga aumentam. Este sistema incorpora uma webcam que analisa a face do utilizador e realiza a detecção do estado dos olhos para calcular a percentagem de tempo em que os olhos estão fechados (PERCLOS). Os dados de frequência cardíaca também foram analisados e um modelo foi desenvolvido para incorporar estes dados, a percentagem de tempo que os olhos ficam fechados, o programa que o utilizador tem aberto e a hora do dia para prever o nível de fadiga. Ao combinar essas duas técnicas diferentes, este sistema pode ser mais eficaz e mais preciso em fornecer previsões do nível de fadiga. A revisão da literatura mostrou que a conjunção dessas duas técnicas na previsão da fadiga é novidade. O sistema desenvolvido também contém integração com a tecnologia smartwatch para aproveitar os dados da frequência cardíaca e comunicar com o utilizador por meio de notificações pop-up para informá-lo quando os níveis de fadiga se encontrarem altos. A conclusão deste trabalho é que a detecção do estado ocular usando Inteligência Artificial pode alcançar uma alta precisão e ser uma ferramenta confiável na identificação de fadiga num utilizador. A combinação da frequência cardíaca e PERCLOS permite que o sistema tenha maior precisão, além de não depender completamente de um unico sensor. A criação de um modelo de previsão de fadiga foi dificultada pela falta de dados existentes para treinar um modelo, problema que poderia ser colmatado com a adoção do sistema numa população maior

Driver drowsiness detection in facial images

Driver fatigue is a significant factor in a large number of vehicle accidents. Thus, drowsy driver alert systems are meant to reduce the main cause of traffic accidents. Different approaches have been developed to tackle with the fatigue detection problem. Though most reliable techniques to asses fatigue involve the use of physical sensors to monitor drivers, they can be too intrusive and are less likely to be adopted by the car industry. A relatively new and effective trend consists on facial image analysis from video cameras that monitor drivers. How to extract effective features of fatigue from images is important for many image processing applications. This project proposes a face descriptor that can be used to detect driver fatigue in static frames. This descriptor represents each frame of a sequence as a pyramid of scaled images that are divided into non-overlapping blocks of equal size. The pyramid of images is combined with three different image descriptors. The final descriptors are filtered out using feature selection and a Support Vector Machine is used to predict the drowsiness state. The proposed method is tested on the public NTHUDDD dataset, which is the state-of-the-art dataset on driver drowsiness detection

SigSegment: A Signal-Based Segmentation Algorithm for Identifying Anomalous Driving Behaviours in Naturalistic Driving Videos

In recent years, distracted driving has garnered considerable attention as it continues to pose a significant threat to public safety on the roads. This has increased the need for innovative solutions that can identify and eliminate distracted driving behavior before it results in fatal accidents. In this paper, we propose a Signal-Based anomaly detection algorithm that segments videos into anomalies and non-anomalies using a deep CNN-LSTM classifier to precisely estimate the start and end times of an anomalous driving event. In the phase of anomaly detection and analysis, driver pose background estimation, mask extraction, and signal activity spikes are utilized. A Deep CNN-LSTM classifier was applied to candidate anomalies to detect and classify final anomalies. The proposed method achieved an overlap score of 0.5424 and ranked 9th on the public leader board in the AI City Challenge 2023, according to experimental validation results

Fatigue Driving Detection Method Based on IPPG Technology

Physiological signal index can accurately reflect the degree of fatigue, but the contact detection method will greatly affect the driver\u27s driving. This paper presents a non-contact method for detecting tired driving. It uses cameras and other devices to collect information about the driver\u27s face. By recording facial changes over a period and processing the captured video, pulse waves are extracted. Then the frequency domain index and nonlinear index of heart rate variability were extracted by pulse wave characteristics. Finally, the experiment proves that the method can clearly judge whether the driver is tired. In this study, the Imaging Photoplethysmography (IPPG) technology was used to realise non-contact driver fatigue detection. Compared with the non-contact detection method through identifying drivers\u27 blinking and yawning, the physiological signal adopted in this paper is more convincing. Compared with other methods that detect physiological signals to judge driver fatigue, the method in this paper has the advantages of being non-contact, fast, convenient and available for the cockpit environment