Facial attributes recognition using computer vision to detect drowsiness and distraction in drivers

Driving is an activity that requires a high degree of concentration on the part of the person who performs it, since the slightest negligence is sufficient to provoke an accident with the consequent material and/or human losses. According to the most recent study published by the World Health Organization (WHO) in 2013, it was estimated that 1.25 million people died as a result of traffic accidents, whereas between 20 and 50 million did not die but consequences resulted in chronic conditions. Many of these accidents are caused by what is known as inattention. This term encloses different conditions such as distraction and drowsiness, which are, precisely, the ones that cause more fatalities. Many publications and research have tried to set figures indicating the consequences of inattention (and its subtypes), but there is no exact number of the accidents caused by inattention since all these studies have been carried out in different places, different time frames and, therefore, under different conditions. Overall, it has been estimated that inattention causes between 25% and 75% of accidents and near-accidents. A study on drowsiness while driving in ten European countries found that fatigue risks increasing reaction time by 86% and it is the fourth leading cause of death on Spanish roads. Distraction is also a major contributor to fatal accidents in Spain. According to the Directorate General of Traffic (DGT), distraction is the first violation found in fatal accidents, 13.15% of the cases. Overall, considering both distraction and drowsiness, the latest statistics on inattentive driving in Spanish drivers are alarming, appearing as the leading cause of fatalities (36%), well above excessive speed (21%) or alcohol consumption (11%).The reason for this PhD thesis is the direct consequences of the abovementioned figures and its purpose is to provide mechanisms to help reduce driver inattention effects using computer vision techniques. The extraction of facial attributes can be used to detect inattention robustly.Specifically, research establishes a frame of reference to characterize distraction in drivers in order to provide solid foundations for future research [1]. Based on this research [1], an architecture based on the analysis of visual characteristics has been proposed, constructed and validated by using techniques of computer vision and automatic learning for the detection of both distraction and drowsiness [2], integrating several innovative elements in order to operate in a completely autonomous way for the robust detection of the main visual indicators characterizing the driver's both distraction and drowsiness: (1) a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction [3]; (2) a face processing algorithm based on Local Binary Patterns (LBP) and Support Vector Machine (SVM) to detect facial attributes [4]; (3) detection unit for the presence/absence of the driver using both a marker and a machine learning algorithm [2]; (4) robust face tracking algorithm based on both the position of the camera and the face detection algorithm [2]; (5) a face alignment and normalization algorithm to improve the eyes state detection [3]; (6) driver drowsiness detection based on the eyes state detection over time [2]; (7) driver distraction detection based on the position of the head over time [2]. This architecture has been validated, firstly, with reference databases testing the different modules that compose it, and, secondly, with users in real environments, obtaining in both cases, excellent results with a suitable computational load for the embedded devices in vehicle environments [2]. In connection with the tests performed in real-world settings, 16 drivers were involved performing several activities imitating different signs of sleepiness and distraction. Overall, an accuracy of 93.11% is obtained considering all activities and all drivers [2].Additionally, other contributions of this thesis have been experimentally validated in controlled settings, but are expected to be included in the abovementioned architecture: (1) glasses detection algorithm prior to the detection of the eyes state [3] (the eyes state can not be accurately obtained if the driver is wearing glasses or sunglasses [1]); (2) face recognition and spoofing detection algorithm to identify the driver [5]; (3) physiological information (Heart Rate, Respiration Rate and Heart Rate Variability) are extracted from the users face [6] (using this information, cognitive load and stress can be obtained [1]); (4) a real-time big data architecture to process a large number of relatively small-sized images [7]. Therefore, future work will include these points to complete the architecture

Facial attributes recognition using computer vision to detect drowsiness and distraction in drivers

Driving is an activity that requires a high degree of concentration on the part of the person who performs it, since the slightest negligence is sufficient to provoke an accident with the consequent material and/or human losses. According to the most recent study published by the World Health Organization (WHO) in 2013, it was estimated that 1.25 million people died as a result of traffic accidents, whereas between 20 and 50 million did not die but consequences resulted in chronic conditions. Many of these accidents are caused by what is known as inattention. This term encloses different conditions such as distraction and drowsiness, which are, precisely, the ones that cause more fatalities. Many publications and research have tried to set figures indicating the consequences of inattention (and its subtypes), but there is no exact number of the accidents caused by inattention since all these studies have been carried out in different places, different time frames and, therefore, under different conditions. Overall, it has been estimated that inattention causes between 25% and 75% of accidents and near-accidents. A study on drowsiness while driving in ten European countries found that fatigue risks increasing reaction time by 86% and it is the fourth leading cause of death on Spanish roads. Distraction is also a major contributor to fatal accidents in Spain. According to the Directorate General of Traffic (DGT), distraction is the first violation found in fatal accidents, 13.15% of the cases. Overall, considering both distraction and drowsiness, the latest statistics on inattentive driving in Spanish drivers are alarming, appearing as the leading cause of fatalities (36%), well above excessive speed (21%) or alcohol consumption (11%).The reason for this PhD thesis is the direct consequences of the abovementioned figures and its purpose is to provide mechanisms to help reduce driver inattention effects using computer vision techniques. The extraction of facial attributes can be used to detect inattention robustly.Specifically, research establishes a frame of reference to characterize distraction in drivers in order to provide solid foundations for future research [1]. Based on this research [1], an architecture based on the analysis of visual characteristics has been proposed, constructed and validated by using techniques of computer vision and automatic learning for the detection of both distraction and drowsiness [2], integrating several innovative elements in order to operate in a completely autonomous way for the robust detection of the main visual indicators characterizing the driver’s both distraction and drowsiness: (1) a review of the role of computer vision technology applied to the development of monitoring systems to detect distraction [3]; (2) a face processing algorithm based on Local Binary Patterns (LBP) and Support Vector Machine (SVM) to detect facial attributes [4]; (3) detection unit for the presence/absence of the driver using both a marker and a machine learning algorithm [2]; (4) robust face tracking algorithm based on both the position of the camera and the face detection algorithm [2]; (5) a face alignment and normalization algorithm to improve the eyes state detection [3]; (6) driver drowsiness detection based on the eyes state detection over time [2]; (7) driver distraction detection based on the position of the head over time [2]. This architecture has been validated, firstly, with reference databases testing the different modules that compose it, and, secondly, with users in real environments, obtaining in both cases, excellent results with a suitable computational load for the embedded devices in vehicle environments [2]. In connection with the tests performed in real-world settings, 16 drivers were involved performing several activities imitating different signs of sleepiness and distraction. Overall, an accuracy of 93.11% is obtained considering all activities and all drivers [2].Additionally, other contributions of this thesis have been experimentally validated in controlled settings, but are expected to be included in the abovementioned architecture: (1) glasses detection algorithm prior to the detection of the eyes state [3] (the eyes state can not be accurately obtained if the driver is wearing glasses or sunglasses [1]); (2) face recognition and spoofing detection algorithm to identify the driver [5]; (3) physiological information (Heart Rate, Respiration Rate and Heart Rate Variability) are extracted from the users face [6] (using this information, cognitive load and stress can be obtained [1]); (4) a real-time big data architecture to process a large number of relatively small-sized images [7]. Therefore, future work will include these points to complete the architecture

Ultra-Efficient On-Device Object Detection on AI-Integrated Smart Glasses with TinyissimoYOLO

Smart glasses are rapidly gaining advanced functionality thanks to cutting-edge computing technologies, accelerated hardware architectures, and tiny AI algorithms. Integrating AI into smart glasses featuring a small form factor and limited battery capacity is still challenging when targeting full-day usage for a satisfactory user experience. This paper illustrates the design and implementation of tiny machine-learning algorithms exploiting novel low-power processors to enable prolonged continuous operation in smart glasses. We explore the energy- and latency-efficient of smart glasses in the case of real-time object detection. To this goal, we designed a smart glasses prototype as a research platform featuring two microcontrollers, including a novel milliwatt-power RISC-V parallel processor with a hardware accelerator for visual AI, and a Bluetooth low-power module for communication. The smart glasses integrate power cycling mechanisms, including image and audio sensing interfaces. Furthermore, we developed a family of novel tiny deep-learning models based on YOLO with sub-million parameters customized for microcontroller-based inference dubbed TinyissimoYOLO v1.3, v5, and v8, aiming at benchmarking object detection with smart glasses for energy and latency. Evaluations on the prototype of the smart glasses demonstrate TinyissimoYOLO's 17ms inference latency and 1.59mJ energy consumption per inference while ensuring acceptable detection accuracy. Further evaluation reveals an end-to-end latency from image capturing to the algorithm's prediction of 56ms or equivalently 18 fps, with a total power consumption of 62.9mW, equivalent to a 9.3 hours of continuous run time on a 154mAh battery. These results outperform MCUNet (TinyNAS+TinyEngine), which runs a simpler task (image classification) at just 7.3 fps per second