Real-Time Driver’s Monitoring Mobile Application through Head Pose, Drowsiness and Angry Detection

The current driver's monitoring system requires a set-up that includes the usage of a variety of camera equipment behind the steering wheel. It is highly impractical in a real-world environment as the set-up might cause annoyance or inconvenience to the driver. This project proposes a framework of using mobile devices and cloud services to monitor the driver's head pose, detect angry expression and drowsiness, and alerting them with audio feedback. With the help of a phone camera functionality, the driver’s facial expression data can be collected then further analyzed via image processing under the Microsoft Azure platform. A working mobile app is developed, and it can detect the head pose, angry emotion, and drowsy drivers by monitoring their facial expressions. Whenever an angry or drowsy face is detected, pop-up alert messages and audio feedback will be given to the driver. The benefit of this mobile app is it can remind drivers to drive calmly and safely until drivers manage to handle their emotions where anger or drowsy is no longer detected. The performance of the mobile app in classifying anger emotion is achieved at 96.66% while the performance to detect driver’s drowsiness is 82.2%. On average, the head pose detection success rate across the six scenarios presented is 85.67%

Drowsiness Detection System Through Eye and Mouth Analysis

Traffic jams are one of the serious issues in many developed countries. After the pandemic, many employees were allowed to travel interstate to work. This contributes to more severe jams, especially in the capital and nearby states. Long-distance driving and congestion can easily make the drivers sleepy and thus lead to traffic accidents. This paper aims to study and analyze facial cues to detect early symptoms of drowsy driving. The proposed method employs a deep learning approach, utilizing ensemble CNNs and Dlib's 68 landmark face detectors to analyze the facial cues. The analyzed symptoms include the frequency of eyes opened or closed and yawning or no yawning. Three individual CNN models and an ensemble CNN structure are built for the classification of the eyes and mouth yawn. The model training and validation accuracy graph and training loss and validation loss graph are plotted to verify that the models have not been overfitted. The ensemble CNN models achieved an approximate accuracy of 97.4% from the eyes and 96.5% from the mouth. It outperforms the other pre-trained models. The proposed system can immediately alert the driver and send text drowsy messages and emails to the third party, ensuring timely intervention to prevent accidents. The proposed method can be integrated into vehicles and transportation systems to ensure driver's safety. It can also be applied to monitor the driving behavior of those who drive long distance

Drowsiness Detection System in Real Time Based on Behavioral Characteristics of Driver using Machine Learning Approach

The process of determining if a person, generally a driver, is becoming sleepy or drowsy while performing a task such as driving is known as drowsiness detection. It is a necessary system for detecting and alerting drivers to their tiredness, which might impair their driving ability and lead to accidents. The project aims to create a reliable and efficient system capable of real-time detection of drowsiness using OpenCV, Dlib, and facial landmark detection technologies. The project's results show that the sleepiness detection method can accurately and precisely identify tiredness in real time. The technology is less intrusive and more economical than conventional sleepiness detection techniques. The system is based on a 68 facial landmark detector, which is a highly trained and effective detector capable of recognizing human face points. The detector aids in assessing whether the driver's eyes are closed or open.  The system analyses the data collected by the detector using machine learning methods to discover patterns associated with drowsiness. When drowsiness is detected, the system incorporates a warning mechanism, such as an alarm or a vibration in the steering wheel, to notify the driver. A variety of studies with different drivers and driving conditions were used to evaluate the performance of the real-time driver drowsiness detection system. The results show that the technology can detect tiredness properly and deliver timely warnings to the driver. This method can assist in preventing drowsy driving incidents, enhancing road safety, and saving lives. The results indicated that the algorithm had an average accuracy rate of 94% for identifying tiredness in drivers

Driver Drowsiness Detection using Hybrid Algorithm

In this work we focus on the discernment of sleepiness in drivers’ drowsiness proposing a hybrid algorithm which aims to confirm whether the driver's level of attention has decreased owing to a nap or any other medical issue, such as brain problems. Therefore, the proposed hybrid algorithm uses both Haarcascade classifier and Convolutional Neural Network (CNN) algorithm to detect drivers’ drowsiness. The driver's eyes will be monitored and an alert sound will be generated by Raspberry Pi module, but the face must be moving in real time, and the aspect ratio must be between 16:9 and 1.85:1. People often feel sleepy since activities like driving call for a proper mental state, and bad work-life balance has additional negative repercussions. When we give input through normal camera it analyses drivers state of eyes and mouth, actually it checks aspect ratio of eye. We proved in comparative trials that our hybrid algorithm beats current driving fatigue detection algorithms in speed as well as accuracy

Design of Video Detection for Drowsy Prevention Based on Car Driving

In this paper, a drowsiness prevention system was developed to prevent large-scaledisasters in traffic accidents. And drowsiness was predicted using face recognition facerecognition technique and eye blink recognition technique, and prediction was improved byapplying machine learning to improve drowsiness prediction. Additionally, the CO2 sensorchip was used to detect additional drowsiness prevention. In addition, STT (Speach To Text)was applied using voice recognition technology so that the driver can apply for a desiredmusic or broadcast or make a phone call in order to break drowsiness while driving

Drowsiness Classification for Internal Driving Situation Awareness on Mobile Platform

the sleeping driver is potentially more likely to cause an accident than the person who speeds up since the driver is the victim of sleepiness. Automobile industry researchers, including manufacturers, seek to solve this issue with various technical solutions that can avoid such a situation. This paper proposes an implementation of a lightweight method to detect driver's sleepiness using facial landmarks and head pose estimation based on neural network methodologies on a mobile device. We try to improve the accurateness by using face images that the camera detects and passes to CNN to identify sleepiness. Firstly, applied a behavioral landmark's sleepiness detection process. Then, an integrated Head Pose Estimation technique will strengthen the system's reliability. The preliminary findings of the tests demonstrate that with real-time capability, more than 86% identification accuracy can be reached in several real-world scenarios for all classes, including with glasses, without glasses, and light-dark background. This work aims to classify drowsiness, warn, and inform drivers, helping them to stop falling asleep at the wheel. The integrated CNN-based method is used to create a high accuracy and simple-to-use real-time driver drowsiness monitoring framework for embedded devices and Android phone

Intelligent and secure real-time auto-stop car system using deep-learning models

In this study, we introduce an innovative auto-stop car system empowered by deep learning technology, specifically employing two Convolutional Neural Networks (CNNs) for face recognition and travel drowsiness detection. Implemented on a Raspberry Pi 4, our system is designed to cater exclusively to certified drivers, ensuring enhanced safety through intelligent features. The face recognition CNN model accurately identifies authorized drivers, employing deep learning techniques to verify their identity before granting access to vehicle functions. This first model demonstrates a remarkable accuracy rate of 99.1%, surpassing existing solutions in secure driver authentication. Simultaneously, our second CNN focuses on real-time detecting+ of driver drowsiness, monitoring eye movements, and utilizing a touch sensor on the steering wheel. Upon detecting signs of drowsiness, the system issues an immediate alert through a speaker, initiating an emergency park and sending a distress message via Global Positioning System (GPS). The successful implementation of our proposed system on the Raspberry Pi 4, integrated with a real-time monitoring camera, attains an impressive accuracy of 99.1% for both deep learning models. This performance surpasses current industry benchmarks, showcasing the efficacy and reliability of our solution. Our auto-stop car system advances user convenience and establishes unparalleled safety standards, marking a significant stride in autonomous vehicle technology

Detection of Drivers Drowsiness on Four-Wheeled Vehicles using the Haar Cascade Algorithm and Eye Aspect Ratio

One of the most common types of threats to four-wheeled vehicle drivers is microsleep. Microsleep is a condition in which a person's loss of attention or consciousness due to a state of fatigue or drowsiness. In general, microsleep lasts for a short duration, about a fraction of a second to a full 10 seconds. One way to modify the driver's sleepy condition is to form a drowsiness detection system through the extraction of facial feature points. The extraction of facial feature points refers to 68 predictor landmarks with detection in the eyes and facial movements of the driver in the form of poses with the determination of the angle threshold of changes in the position of the face while driving which indicates a state of drowsiness. This study implements the use of the Haar Cascade Classifier algorithm in detecting the drowsiness of four-wheeled vehicle drivers and the Eye Aspect Ratio of the points that form the eyes using Euclidean Distance. In detecting the eye index on the face predictor landmarks uses the dlib python library to detect objects, face detection, and face landmark detection. This study also uses the Face Detector library to create a face detector object and a Landmark Predictor. The test results showed that the detection system was 98.33% accurate with the condition of facial features that could still be identified by the system even though the difference in face distance with the webcam acquisition tool was far away. This detection system is also able to detect driver drowsiness with an average time duration of less than 5 seconds with a distance of up to 50 meters.  The system detects drowsiness quickly with a notification in the form of a warning in the form of an alarm sound, which is very important in order to reduce the number of accidents due to drowsiness.One of the most common types of threats to four-wheeled vehicle drivers is microsleep. Microsleep is a condition in which a person's loss of attention or consciousness due to a state of fatigue or drowsiness. In general, microsleep lasts for a short duration, about a fraction of a second to a full 10 seconds. One way to modify the driver's sleepy condition is to form a drowsiness detection system through the extraction of facial feature points. The extraction of facial feature points refers to 68 predictor landmarks with detection in the eyes and facial movements of the driver in the form of poses with the determination of the angle threshold of changes in the position of the face while driving which indicates a state of drowsiness. This study implements the use of the Haar Cascade Classifier algorithm in detecting the drowsiness of four-wheeled vehicle drivers and the Eye Aspect Ratio of the points that form the eyes using Euclidean Distance. In detecting the eye index on the face predictor landmarks uses the dlib python library to detect objects, face detection, and face landmark detection. This study also uses the Face Detector library to create a face detector object and a Landmark Predictor. The test results showed that the detection system was 98.33% accurate with the condition of facial features that could still be identified by the system even though the difference in face distance with the webcam acquisition tool was far away. This detection system is also able to detect driver drowsiness with an average time duration of less than 5 seconds with a distance of up to 50 meters.  The system detects drowsiness quickly with a notification in the form of a warning in the form of an alarm sound, which is very important in order to reduce the number of accidents due to drowsiness

Driver-Drowsiness Detection System

Due to the drowsiness of drivers, car accidents kill thousands of people worldwide every year.This fact clearly illustrates the need for a sleep sensor application to help prevent such accidents and ultimately save lives. In this work, we propose a novel intensive learning method based on neutral neural networks (CNN) to deal with this problem. In this project we aim to develop a prototype drowsiness detection system. The system works by monitoring the driver's eyes and ringing the alarm while it is drying. The system is a real-time control system that is not intrusive. The priority is to improve driver safety without intrusion. In this project, the driver's eyelid is detected. When a driver's eyes are closed for an extended period of time, the driver is considered indifferent, and an alarm rings. The Haar Cascade library is used to detect facial features, and programming is performed in OpenCV

Smart vehicle management by using sensors and an IoT based black box

As the number of transports on the road increases every day, so does the number of accidents. Reckless driving and consuming alcohol are two of the leading causes of accidents. Apart from these issues, the safety of humans and vehicles is also critical. A thorough investigation is required to minimize the accident rate and improve human safety, particularly if an incident occurs. The purpose of this study is to develop a few sensor-based black box system that will help us reduce traffic accidents by continuously providing precise guidance to the driver. At the same time, the evidence will be uploaded to its server for further evaluation. This system also includes a way of detecting drowsiness in the driver. Finally, using global positioning system (GPS) and global system for mobile communications (GSM), the relevant authorities will get information on the vehicle's condition and whereabouts. For security purposes, a panic button is introduced here to get emergency help from the security personnel by detecting the victim’s area