Offline EEG-based driver drowsiness estimation using enhanced batch-mode active learning (EBMAL) for regression

© 2016 IEEE. There are many important regression problems in real-world brain-computer interface (BCI) applications, e.g., driver drowsiness estimation from EEG signals. This paper considers offline analysis: given a pool of unlabeled EEG epochs recorded during driving, how do we optimally select a small number of them to label so that an accurate regression model can be built from them to label the rest? Active learning is a promising solution to this problem, but interestingly, to our best knowledge, it has not been used for regression problems in BCI so far. This paper proposes a novel enhanced batch-mode active learning (EBMAL) approach for regression, which improves upon a baseline active learning algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better regression performance. We validate its effectiveness using driver drowsiness estimation from EEG signals. However, EBMAL is a general approach that can also be applied to many other offline regression problems beyond BCI

Automatic Driver Drowsiness Detection System

The proposed system aims to lessen the number of accidents that occur due to drivers’ drowsiness and fatigue, which will in turn increase transportation safety. This has become a common reason for accidents in recent times. Several facial and body gestures are considered signs of drowsiness and fatigue in drivers, including tiredness in the eyes and yawning. These features are an indication that the driver’s condition is improper. EAR (Eye Aspect Ratio) computes the ratio of distances between the horizontal and vertical eye landmarks, which is required for the detection of drowsiness. For the purpose of yawn detection, a YAWN value is calculated using the distance between the lower lip and the upper lip, and the distance will be compared against a threshold value. We have deployed an eSpeak module (text-to-speech synthesiser), which is used for giving appropriate voice alerts when the driver is feeling drowsy or is yawning. The proposed system is designed to decrease the rate of accidents and contribute to technology with the goal of preventing fatalities caused by road accidents. Over the past ten years, advances in artificial intelligence and computing technologies have improved driver monitoring systems. Several experimental studies have gathered data on actual driver fatigue using different artificial intelligence systems. In order to dramatically improve these systems' real-time performance, feature combinations are used. An updated evaluation of the driver sleepiness detection technologies put in place during the previous ten years is presented in this research. The paper discusses and displays current systems that track and identify drowsiness using various metrics. Based on the information used, each system can be categorised into one of four groups. Each system in this paper comes with a thorough discussion of the features, classification rules, and datasets it employs.&nbsp

Correcting inter-sectional accuracy differences in drowsiness detection systems using generative adversarial networks (GANs)

Doctoral Degrees. University of KwaZulu-Natal, Durban.oad accidents contribute to many injuries and deaths among the human population. There is substantial evidence that proves drowsiness is one of the most prominent causes of road accidents all over the world. This results in fatalities and severe injuries for drivers, passengers, and pedestrians. These alarming facts are raising the interest in equipping vehicles with robust driver drowsiness detection systems to minimise accident rates. One of the primary concerns of motor industries is the safety of passengers and as a consequence they have invested significantly in research and development to equip vehicles with systems that can help minimise to road accidents. A number research endeavours have attempted to use Artificial intelligence, and particularly Deep Neural Networks (DNN), to build intelligent systems that can detect drowsiness automatically. However, datasets are crucial when training a DNN. When datasets are unrepresentative, trained models are prone to bias because they are unable to generalise. This is particularly problematic for models trained in specific cultural contexts, which may not represent a wide range of races, and thus fail to generalise. This is a specific challenge for driver drowsiness detection task, where most publicly available datasets are unrepresentative as they cover only certain ethnicity groups. This thesis investigates the problem of an unrepresentative dataset in the training phase of Convolutional Neural Networks (CNNs) models. Firstly, CNNs are compared with several machine learning techniques to establish their superior suitability for the driver drowsiness detection task. An investigation into the implementation of CNNs was performed and highlighted that publicly available datasets such as NTHU, DROZY and CEW do not represent a wide spectrum of ethnicity groups and lead to biased systems. A population bias visualisation technique was proposed to help identify the regions, or individuals where a model is failing to generalise on a picture grid. Furthermore, the use of Generative Adversarial Networks (GANs) with lightweight convolutions called Depthwise Separable Convolutions (DSC) for image translation to multi-domain outputs was investigated in an attempt to generate synthetic datasets. This thesis further showed that GANs can be used to generate more realistic images with varied facial attributes for predicting drowsiness across multiple ethnicity groups. Lastly, a novel framework was developed to detect bias and correct it using synthetic generated images which are produced by GANs. Training models using this framework results in a substantial performance boost

Driving Fatigue Recognition with Functional Connectivity Based on Phase Synchronization

Accumulating evidences showed that the optimal brain network topology was altered with the progression of fatigue during car driving. However, the extent of discriminative power of functional connectivity that contribute to the driving fatigue detection is still unclear. In this study, we extracted two types of features (network properties and critical connections) to explore their usefulness in driving fatigue detection. EEG data were recorded twice from twenty healthy subjects during a simulated driving experiment. Multi-band functional connectivity matrices were established using phase lag index, which serve as input for the following graph theoretical analysis and critical connections determination between the most vigilant and fatigued states. We found a reorganisation of brain network towards less efficient architecture in fatigue state across all frequency bands. Further interrogations showed that the discriminative connections were mainly connected to frontal areas, i.e., most of the increased connections are from frontal pole to parietal or occipital regions. Moreover, we achieved a satisfactory classification accuracy (96.76%) using the discriminative connection features in β band. Our study demonstrated that graph theoretical properties and critical connections are of discriminative power for manifesting fatigue alterations and the critical connection is an efficient feature for driving fatigue detection

Bias Remediation in Driver Drowsiness Detection Systems Using Generative Adversarial Networks

Datasets are crucial when training a deep neural network. When datasets are unrepresentative, trained models are prone to bias because they are unable to generalise to real world settings. This is particularly problematic for models trained in specific cultural contexts, which may not represent a wide range of races, and thus fail to generalise. This is a particular challenge for Driver drowsiness detection, where many publicly available datasets are unrepresentative as they cover only certain ethnicity groups. Traditional augmentation methods are unable to improve a model's performance when tested on other groups with different facial attributes, and it is often challenging to build new, more representative datasets. In this paper, we introduce a novel framework that boosts the performance of detection of drowsiness for different ethnicity groups. Our framework improves Convolutional Neural Network (CNN) trained for prediction by using Generative Adversarial networks (GAN) for targeted data augmentation based on a population bias visualisation strategy that groups faces with similar facial attributes and highlights where the model is failing. A sampling method selects faces where the model is not performing well, which are used to fine-tune the CNN. Experiments show the efficacy of our approach in improving driver drowsiness detection for under represented ethnicity groups. Here, models trained on publicly available datasets are compared with a model trained using the proposed data augmentation strategy. Although developed in the context of driver drowsiness detection, the proposed framework is not limited to the driver drowsiness detection task, but can be applied to other applications.Comment: 9 pages, 7 figure

Between-Frequency Topographical and Dynamic High-Order Functional Connectivity for Driving Drowsiness Assessment

Previous studies exploring driving drowsiness utilized spectral power and functional connectivity without considering between-frequency and more complex synchronizations. To complement such lacks, we explored inter-regional synchronizations based on the topographical and dynamic properties between frequency bands using high-order functional connectivity (HOFC) and envelope correlation. We proposed the dynamic interactions of HOFC, associated-HOFC, and a global metric measuring the aggregated effect of the functional connectivity. The EEG dataset was collected from 30 healthy subjects, undergoing two driving sessions. The two-session setting was employed for evaluating the metric reliability across sessions. Based on the results, we observed reliably significant metric changes, mainly involving the alpha band. In HOFC θα , HOFC αβ , associated-HOFC θα , and associatedHOFC αβ , the connection-level metrics in frontal-central, central-central,and central-parietal/occipitalareas were significantly increased, indicating a dominance in the central region. Similar results were also obtained in the HOFC θαβ and aHOFC θαβ . For dynamic-low-order-FC and dynamicHOFC, the global metrics revealed a reliably significant increment in the alpha, theta-alpha, and alpha-beta bands. Modularity indexes of associated-HOFC α and associatedHOFC θα also exhibited reliably significant differences. This paper demonstrated that within-band and betweenfrequency topographical and dynamic FC can provide complementary information to the traditional individual-band LOFC for assessing driving drowsiness

Correlation between Situational Awareness and EEG signals

An important aspect in safety–critical domains is Situational Awareness (SA) where operators consolidate data into an understanding of the situation that needs to be updated dynamically as the situation changes over time. Among existing measures of SA, only physiological measures can assess the cognitive processes associated with SA in real-time. Some studies showed promise in detecting cognitive states associated with SA in complex tasks using brain signals (e.g. electroencephalogram/EEG). In this paper, an analytical methodology is proposed to identify EEG signatures associated with SA on various regions of the brain. A new data set from 32 participants completing the SA test in the PEBL is collected using a 32-channel dry-EEG headset. The proposed method is tested on the new data set and a correlation is identified between the frequency bands of b (12 - 30 Hz) and c (30 - 45 Hz) and SA. Also, activation of neurons in the left and right hemisphere of the parietal and temporal lobe is observed. These regions are responsible for the visuo-spatial ability and memory and reasoning tasks. Among the presented results, the highest achieved accuracy on test data is 67%