Driver Drowsiness Detection System

In recent years’ driver fatigue is one of the major causes of vehicle accidents in the world. A direct way of measuring driver fatigue is measuring the state of the driver i.e. drowsiness. So it is very important to detect the drowsiness of the driver to save life and property. This project is aimed towards developing a prototype of drowsiness detection system. This system is a real time system which captures image continuously and measures the state of the eye according to the specified algorithm and gives warning if required. Though there are several methods for measuring the drowsiness but this approach is completely non-intrusive which does not affect the driver in any way, hence giving the exact condition of the driver. For detection of drowsiness the per closure value of eye is considered. So when the closure of eye exceeds a certain amount then the driver is identified to be sleepy. For implementing this system several OpenCv libraries are used including Haar-cascade. The entire system is implemented using Raspberry-Pi

Analysis and detection of driver fatigue caused by sleep deprivation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2007.Includes bibliographical references (leaves 167-181).Human errors in attention and vigilance are among the most common causes of transportation accidents. Thus, effective countermeasures are crucial for enhancing road safety. By pursuing a practical and reliable design of an Active Safety system which aims to predict and avoid road accidents, we identify the characteristics of drowsy driving and devise a systematic way to infer the state of driver alertness based on driver-vehicle data. Although sleep and fatigue are major causes of impaired driving, neither effective regulations nor acceptable countermeasures are available yet. The first part of this thesis analyzes driver-vehicle systems with discrete sleep-deprivation levels, and reveals differences in the performance characteristics of drivers. Inspired by the human sleep-wake cycle mechanism and attributes of driver-vehicle systems, we design and perform human-in-the-loop experiments in a test bed built with STISIM Drive, an interactive fixed-based driving simulator. In the simulated driving, participants were given various driving tasks and secondary tasks for both non and partially sleep-deprived conditions. This experiment demonstrates that sleep deprivation has a greater effect on rule-based tasks than on skill-based tasks; when drivers are sleep-deprived, their performance of responding to unexpected disturbances degrades while they are robust enough to continue such routine driving tasks as straight lane tracking, following a lead vehicle, lane changes, etc. In the second part of the thesis we present both qualitative and quantitative guidelines for designing drowsy driver detection systems in a probabilistic framework based on the Bayesian network paradigm and experimental data.(cont.) We consider two major causes of sleep, i.e., sleep debt and circadian rhythm, in the framework with various driver-vehicle parameters, and also address temporal aspects of drowsiness and individual differences of subjects. The thesis concludes that detection of drowsy driving based on driver-vehicle data is a feasible but difficult problem which has diverse issues to be addressed; the ultimate challenge lies in the human operator.by Ji Hyun Yang.Ph.D

Psychophysiological models of hypovigilance detection: A scoping review

Hypovigilance represents a major contributor to accidents. In operational contexts, the burden of monitoring/managing vigilance often rests on operators. Recent advances in sensing technologies allow for the development of psychophysiology‐based (hypo)vigilance prediction models. Still, these models remain scarcely applied to operational situations and need better understanding. The current scoping review provides a state of knowledge regarding psychophysiological models of hypovigilance detection. Records evaluating vigilance measuring tools with gold standard comparisons and hypovigilance prediction performances were extracted from MEDLINE, PsychInfo, and Inspec. Exclusion criteria comprised aspects related to language, non‐empirical papers, and sleep studies. The Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS) and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were used for bias evaluation. Twenty‐one records were reviewed. They were mainly characterized by participant selection and analysis biases. Papers predominantly focused on driving and employed several common psychophysiological techniques. Yet, prediction methods and gold standards varied widely. Overall, we outline the main strategies used to assess hypovigilance, their principal limitations, and we discuss applications of these models

Sitting behaviour-based pattern recognition for predicting driver fatigue

The proposed approach based on physiological characteristics of sitting behaviours and sophisticated machine learning techniques would enable an effective and practical solution to driver fatigue prognosis since it is insensitive to the illumination of driving environment, non-obtrusive to driver, without violating driver’s privacy, more acceptable by drivers

Detecting fatigue in car drivers and aircraft pilots by using eye-motion metrics

Fatigue is widely recognised for risking the safety of aviation and ground transportation. To enhance transport safety, fatigue detection systems based on psychophysiological measures have been under development for many years. However, a reliable and robust fatigue detection system is still missing. This thesis starts with a literature review of fatigue concepts in the transportation field and the current psychophysiological measures to fatigue, and narrows down the focus to improving fatigue detection systems using eye-motion measures. A research gap was identified between current fatigue systems only focusing on part of sleepy symptoms and a comprehensive fatigue detection system including mental fatigue needed. To address this gap, four studies were conducted to reshape the understanding of fatigue in transportation and explore effective eye-motion metrics for indicating fatigue considering different causal factors. Studies 1 and 2 investigated the influence of two types of task-related fatigue on eye movement. Twenty participants completed a vigilance task before and after a 1-h simulator-based drive with a secondary task. Forty participants, divided equally into two groups, finished the same task before and after a 1-h and 1.5-h monotonous driving task. The results demonstrated that two types of task-related fatigue caused by cognitive overload and prolonged underload induced different physiological responses to eye-motion metrics. The results also proved that the increased mental fatigue decreased driver’s vigilance. Studies 3 and 4 simulated two hazardous fatigue scenarios for pilots. Study 3 explored the relationship between eye-motion metrics and pilot fatigue in an underload flight condition with sleep deprivation (low workload and sleep pressure). Study 4 explored the effective eye-motion metrics to estimate pilot’s cognitive fatigue imposed by time on task and high workload. The results suggested different eye-motion metrics to indicate sleepiness and mental fatigue. In addition, based on the sleepiness and mental fatigue indicators in Studies 3 and 4, several classifiers were built and evaluated to accurately detect sleepiness and mental fatigue. These findings show that considering casual factors such as sleep pressure, time on task and workload when using eye-motion metrics to detect fatigue can improve the accuracy and face validity of the current fatigue detection systems

Sleep homeostasis in the European jackdaw (<i>Coloeus monedula</i>):Sleep deprivation increases NREM sleep time and EEG power while reducing hemispheric asymmetry

Introduction: Sleep is a wide-spread phenomenon that is thought to occur in all animals. Yet, the function of it remains an enigma. Conducting sleep experiments in different species may shed light on the evolution and functions of sleep. Therefore, we studied sleep architecture and sleep homeostatic responses to sleep deprivation in the European jackdaw (Coloeus monedula).Methods: A total of nine young adult birds were implanted with epidural electrodes and equipped with miniature data loggers for recording movement activity (accelerometery) and electroencephalogram (EEG). Individually-housed jackdaws were recorded under controlled conditions with a 12:12-h light-dark cycle.Results: During baseline, the birds spent on average 48.5% of the time asleep (39.8% non-rapid eye movement (NREM) sleep and 8.7% rapid eye movement (REM) sleep). Most of the sleep occurred during the dark phase (dark phase: 75.3% NREM sleep and 17.2% REM sleep; light phase 4.3% NREM sleep and 0.1% REM sleep). After sleep deprivation of 4 and 8 h starting at lights off, the birds showed a dose-dependent increase in NREM sleep time. Also, NREM sleep EEG power in the 1.5–3 Hz frequency range, which is considered to be a marker of sleep homeostasis in mammals, was significantly increased for 1-2 h after both 4SD and 8SD. While there was little true unihemispheric sleep in the Jackdaws, there was a certain degree of hemispheric asymmetry in NREM sleep EEG power during baseline, which reduced after sleep deprivation in a dose-dependent manner.Conclusion: In conclusion, jackdaws display homeostatic regulation of NREM sleep and sleep pressure promotes coherence in EEG power

Regression Based Continuous Driving Fatigue Estimation: Towards Practical Implementation

Mental fatigue in drivers is one of the leading causes that give rise to traffic accidents. Electroencephalography (EEG) based driving fatigue studies showed promising performance in fatigue monitoring. However, complex methodologies are not suitable for practical implementation. In our simulation based setup that retained the constraints of real driving, we took a step closer to fatigue estimation in a practical scenario. We adopted a pre-processing pipeline with low computational complexity, which can be easily and practically implemented in real-time. Moreover, regression-based continuous fatigue estimation was achieved using power spectral features in conjunction with time as the fatigue label. We sought to compare three regression models and three time windows to demonstrate their effects on the performance of fatigue estimation. Dynamic time warping was proposed as a new measure for evaluating the performance of fatigue estimation. The results derived from the validation of the proposed framework on 19 subjects showed that our proposed framework was promising towards practical implementation. Fatigue estimation by the support vector regression with radial basis function kernel and 5-second window length achieved the best performance. We also provided a comprehensive analysis on the spatial distribution of channels and frequency bands mostly contributing to fatigue estimation, which can inform the feature and channel reduction for real-time fatigue monitoring in practical driving. After reducing the number of electrodes by 75%, the proposed framework retained comparable performance in fatigue estimation. This study demonstrates the feasibility and adaptability of our proposed framework in practical implementation of mental fatigue estimation

Modeling driver distraction mechanism and its safety impact in automated vehicle environment.

Automated Vehicle (AV) technology expects to enhance driving safety by eliminating human errors. However, driver distraction still exists under automated driving. The Society of Automotive Engineers (SAE) has defined six levels of driving automation from Level 0~5. Until achieving Level 5, human drivers are still needed. Therefore, the Human-Vehicle Interaction (HVI) necessarily diverts a driver’s attention away from driving. Existing research mainly focused on quantifying distraction in human-operated vehicles rather than in the AV environment. It causes a lack of knowledge on how AV distraction can be detected, quantified, and understood. Moreover, existing research in exploring AV distraction has mainly pre-defined distraction as a binary outcome and investigated the patterns that contribute to distraction from multiple perspectives. However, the magnitude of AV distraction is not accurately quantified. Moreover, past studies in quantifying distraction have mainly used wearable sensors’ data. In reality, it is not realistic for drivers to wear these sensors whenever they drive. Hence, a research motivation is to develop a surrogate model that can replace the wearable device-based data to predict AV distraction. From the safety perspective, there lacks a comprehensive understanding of how AV distraction impacts safety. Furthermore, a solution is needed for safely offsetting the impact of distracted driving. In this context, this research aims to (1) improve the existing methods in quantifying Human-Vehicle Interaction-induced (HVI-induced) driver distraction under automated driving; (2) develop a surrogate driver distraction prediction model without using wearable sensor data; (3) quantitatively reveal the dynamic nature of safety benefits and collision hazards of HVI-induced visual and cognitive distractions under automated driving by mathematically formulating the interrelationships among contributing factors; and (4) propose a conceptual prototype of an AI-driven, Ultra-advanced Collision Avoidance System (AUCAS-L3) targeting HVI-induced driver distraction under automated driving without eye-tracking and video-recording. Fixation and pupil dilation data from the eye tracking device are used to model driver distraction, focusing on visual and cognitive distraction, respectively. In order to validate the proposed methods for measuring and modeling driver distraction, a data collection was conducted by inviting drivers to try out automated driving under Level 3 automation on a simulator. Each driver went through a jaywalker scenario twice, receiving a takeover request under two types of HVI, namely “visual only” and “visual and audible”. Each driver was required to wear an eye-tracker so that the fixation and pupil dilation data could be collected when driving, along with driving performance data being recorded by the simulator. In addition, drivers’ demographical information was collected by a pre-experiment survey. As a result, the magnitude of visual and cognitive distraction was quantified, exploring the dynamic changes over time. Drivers are more concentrated and maintain a higher level of takeover readiness under the “visual and audible” warning, compared to “visual only” warning. The change of visual distraction was mathematically formulated as a function of time. In addition, the change of visual distraction magnitude over time is explained from the driving psychology perspective. Moreover, the visual distraction was also measured by direction in this research, and hotspots of visual distraction were identified with regard to driving safety. When discussing the cognitive distraction magnitude, the driver’s age was identified as a contributing factor. HVI warning type contributes to the significant difference in cognitive distraction acceleration rate. After drivers reach the maximum visual distraction, cognitive distraction tends to increase continuously. Also, this research contributes to quantitatively revealing how visual and cognitive distraction impacts the collision hazards, respectively. Moreover, this research contributes to the literature by developing deep learning-based models in predicting a driver’s visual and cognitive distraction intensity, focusing on demographics, HVI warning types, and driving performance. As a solution to safety issues caused by driver distraction, the AUCAS-L3 has been proposed. The AUCAS-L3 is validated with high accuracies in predicting (a) whether a driver is distracted and does not perform takeover actions and (b) whether crashes happen or not if taken over. After predicting the presence of driver distraction or a crash, AUCAS-L3 automatically applies the brake pedal for drivers as effective and efficient protection to driver distraction under automated driving. And finally, a conceptual prototype in predicting AV distraction and traffic conflict was proposed, which can predict the collision hazards in advance of 0.82 seconds on average

Connectivity Analysis of Brain States and Applications in Brain-Computer Interfaces

Human brain is organized by a large number of functionally correlated but spatially distributed cortical neurons. Cognitive processes are usually associated with dynamic interactions among multiple brain regions. Therefore, the understanding of brain functions requires the inves- tigation of the brain interaction patterns. This thesis contains two main aspects. The first aspect focuses on the neural basis for cognitive processes through the use of brain connectivity analysis. The second part targets on assessing brain connectivity patterns in realistic scenarios, e.g., in-car BCI and stroke patients. In the first part, we explored the neural correlates of error-related brain activity. We recorded scalp electroencephalogram (EEG) from 15 healthy subjects while monitoring the movement of a cursor on a computer screen, yielding particular brain connectivity patterns after monitoring external errors. This supports the presence of common role of medial frontal cortex in coordinating cross-regional activity during brain error processes, independent of their causes, either self-generated or external events. This part also included the investigation of the connectivity during left/right hand motor imagery, including 9 healthy subjects, which demonstrated particular intrahemispheric and interhemispheric information flows in two motor imagery tasks, i.e., the ÎŒ rhythm is highly modulated in intrahemispheric, whereas β and γ are modulated in interhemispheric interactions. This part also explored the neural correlates of reaction time during driving. An experiment with 15 healthy subjects in car simulator was designed, in which they needed to perform lane change to avoid collision with obstacles. Significant neural modulations were found in ERP (event-related potential), PSD (power spectral density), and frontoparietal network, which seems to reflect the underlying information transfer from sensory representation in the parietal cortex to behavioral adjusting in the frontal cortex. In the second part, we first explored the feasibility of using BCI as driving assistant system, in which visual stimuli were presented to evoke error/correct related potentials, and were classified to infer driverâs preferred turning direction. The system was validated in a car simulator with 22 subjects, and 7 joined online tests. The system was also tested in real car, yielding similar brain patterns and comparable classification accuracy. The second part also carried out the brain connectivity analysis in stroke patients.We performed exploratory study to correlate the recovery effects of BCI therapy, through the quantification of connectivity between healthy and lesioned hemispheres. The results indicate the benefits of BCI therapy for stroke patients, i.e., brain connectivity are more similar as healthy patterns, increased (decreased) flow from the damaged (undamaged) to the undamaged (damaged) cortex. Briefly, this thesis presents exploratory studies of brain connectivity analysis, investigating the neural basis of cognitive processes, and its contributions in the decoding phase. In particular, such analysis is not limited to laboratory researches, but also extended to clinical trials and driving scenarios, further supporting the findings observed in the ideal condition