Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of certain types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. Different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or the brain activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Further, potential topics in relation to driving behaviour are identified that could benefit from the adoption of neuroimaging methods in future studies

Physiological-based Driver Monitoring Systems: A Scoping Review

A physiological-based driver monitoring system (DMS) has attracted research interest and has great potential for providing more accurate and reliable monitoring of the driver’s state during a driving experience. Many driving monitoring systems are driver behavior-based or vehicle-based. When these non-physiological based DMS are coupled with physiological-based data analysis from electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and electromyography (EMG), the physical and emotional state of the driver may also be assessed. Drivers’ wellness can also be monitored, and hence, traffic collisions can be avoided. This paper highlights work that has been published in the past five years related to physiological-based DMS. Specifically, we focused on the physiological indicators applied in DMS design and development. Work utilizing key physiological indicators related to driver identification, driver alertness, driver drowsiness, driver fatigue, and drunk driver is identified and described based on the PRISMA Extension for Scoping Reviews (PRISMA-Sc) Framework. The relationship between selected papers is visualized using keyword co-occurrence. Findings were presented using a narrative review approach based on classifications of DMS. Finally, the challenges of physiological-based DMS are highlighted in the conclusion. Doi: 10.28991/CEJ-2022-08-12-020 Full Text: PD

A hybrid deep neural network approach to recognize driving fatigue based on EEG signals

Electroencephalography (EEG) data serve as a reliable method for fatigue detection due to their intuitive representation of drivers' mental processes. However, existing research on feature generation has overlooked the effective and automated aspects of this process. The challenge of extracting features from unpredictable and complex EEG signals has led to the frequent use of deep learning models for signal classification. Unfortunately, these models often neglect generalizability to novel subjects. To address these concerns, this study proposes the utilization of a modified deep convolutional neural network, specifically the Inception-dilated ResNet architecture. Trained on spectrograms derived from segmented EEG data, the network undergoes analysis in both temporal and spatial-frequency dimensions. The primary focus is on accurately detecting and classifying fatigue. The inherent variability of EEG signals between individuals, coupled with limited samples during fatigue states, presents challenges in fatigue detection through brain signals. Therefore, a detailed structural analysis of fatigue episodes is crucial. Experimental results demonstrate the proposed methodology's ability to distinguish between alertness and sleepiness, achieving average accuracy rates of 98.87% and 82.73% on Figshare and SEED-VIG datasets, respectively, surpassing contemporary methodologies. Additionally, the study examines frequency bands' relative significance to further explore participants' inclinations in states of alertness and fatigue. This research paves the way for deeper exploration into the underlying factors contributing to mental fatigue.</p

EEG-Based Driver Fatigue Detection Using FAWT and Multiboosting Approaches

Globally, 14%-20% of road accidents are mainly due to driver fatigue, the causes of which are instance sickness, travelling for long distance, boredom as a result of driving along the same route consistently, lack of enough sleep, etc. This article presents a flexible analytic wavelet transform (FAWT)-based advanced machine learning method using single modality neurophysiological brain electroencephalogram signals to detect the driver fatigues (i.e., FATIGUE and REST) and to alarm the driver at the earliest to prevent the risks during driving. First, signals of undertaking study groups are subjected to the FAWT that separates the signals into LP and HP channels. Subsequently, relevant subband frequency components with proper setting of tuning parameters are extracted. Then, comprehensive low order features which are statistically significant for p < 0.05, are evaluated from the input subband searched space and embedded them to various ensemble methods under multiboost strategy. Results are evaluated in terms of various parameters including accuracy, F-score, AUC, and kappa. Results show that the proposed approach is promising in classification and it achieves optimum individual accuracies of 97.10% and 97.90% in categorizing FATIGUE and REST states with F-score of 97.50%, AUC of 0.975, and kappa of 0.950. Comparison of the proposed method with the prior methods in the context of feature, accuracy, and modality profiles undertaken, indicates the effectiveness and reliability of the proposed method for real-world applications

Wearable in-ear pulse oximetry: theory and applications

Wearable health technology, most commonly in the form of the smart watch, is employed by millions of users worldwide. These devices generally exploit photoplethysmography (PPG), the non-invasive use of light to measure blood volume, in order to track physiological metrics such as pulse and respiration. Moreover, PPG is commonly used in hospitals in the form of pulse oximetry, which measures light absorbance by the blood at different wavelengths of light to estimate blood oxygen levels (SpO2). This thesis aims to demonstrate that despite its widespread usage over many decades, this sensor still possesses a wealth of untapped value. Through a combination of advanced signal processing and harnessing the ear as a location for wearable sensing, this thesis introduces several novel high impact applications of in-ear pulse oximetry and photoplethysmography. The aims of this thesis are accomplished through a three pronged approach: rapid detection of hypoxia, tracking of cognitive workload and fatigue, and detection of respiratory disease. By means of the simultaneous recording of in-ear and finger pulse oximetry at rest and during breath hold tests, it was found that in-ear SpO2 responds on average 12.4 seconds faster than the finger SpO2. This is likely due in part to the ear being in close proximity to the brain, making it a priority for oxygenation and thus making wearable in-ear SpO2 a good proxy for core blood oxygen. Next, the low latency of in-ear SpO2 was further exploited in the novel application of classifying cognitive workload. It was found that in-ear pulse oximetry was able to robustly detect tiny decreases in blood oxygen during increased cognitive workload, likely caused by increased brain metabolism. This thesis demonstrates that in-ear SpO2 can be used to accurately distinguish between different levels of an N-back memory task, representing different levels of mental effort. This concept was further validated through its application to gaming and then extended to the detection of driver related fatigue. It was found that features derived from SpO2 and PPG were predictive of absolute steering wheel angle, which acts as a proxy for fatigue. The strength of in-ear PPG for the monitoring of respiration was investigated with respect to the finger, with the conclusion that in-ear PPG exhibits far stronger respiration induced intensity variations and pulse amplitude variations than the finger. All three respiratory modes were harnessed through multivariate empirical mode decomposition (MEMD) to produce spirometry-like respiratory waveforms from PPG. It was discovered that these PPG derived respiratory waveforms can be used to detect obstruction to breathing, both through a novel apparatus for the simulation of breathing disorders and through the classification of chronic obstructive pulmonary disease (COPD) in the real world. This thesis establishes in-ear pulse oximetry as a wearable technology with the potential for immense societal impact, with applications from the classification of cognitive workload and the prediction of driver fatigue, through to the detection of chronic obstructive pulmonary disease. The experiments and analysis in this thesis conclusively demonstrate that widely used pulse oximetry and photoplethysmography possess a wealth of untapped value, in essence teaching the old PPG sensor new tricks.Open Acces

Deep Learning for Detecting Multi-Level Driver Fatigue Using Physiological Signals: A Comprehensive Approach

Data Availability Statement: Tabriz University’s ethics committee in Tabriz, Iran. Data access is private and not publicly available.Copyright © 2023 by the authors. A large share of traffic accidents is related to driver fatigue. In recent years, many studies have been organized in order to diagnose and warn drivers. In this research, a new approach was presented in order to detect multi-level driver fatigue. A multi-level driver tiredness diagnostic database based on physiological signals including ECG, EEG, EMG, and respiratory effort was developed for this aim. The EEG signal was used for processing and other recorded signals were used to confirm the driver’s fatigue so that fatigue was not confirmed based on self-report questionnaires. A customized architecture based on adversarial generative networks and convolutional neural networks (end-to-end) was utilized to select/extract features and classify different levels of fatigue. In the customized architecture, with the objective of eliminating uncertainty, type 2 fuzzy sets were used instead of activation functions such as Relu and Leaky Relu, and the performance of each was investigated. The final accuracy obtained in the three scenarios considered, two-level, three-level, and five-level, were 96.8%, 95.1%, and 89.1%, respectively. Given the suggested model’s optimal performance, which can identify five various levels of driver fatigue with high accuracy, it can be employed in practical applications of driver fatigue to warn drivers.This research received no external funding

A regression method for EEG-based cross-dataset fatigue detection

Introduction: Fatigue is dangerous for certain jobs requiring continuous concentration. When faced with new datasets, the existing fatigue detection model needs a large amount of electroencephalogram (EEG) data for training, which is resource-consuming and impractical. Although the cross-dataset fatigue detection model does not need to be retrained, no one has studied this problem previously. Therefore, this study will focus on the design of the cross-dataset fatigue detection model.Methods: This study proposes a regression method for EEG-based cross-dataset fatigue detection. This method is similar to self-supervised learning and can be divided into two steps: pre-training and the domain-specific adaptive step. To extract specific features for different datasets, a pretext task is proposed to distinguish data on different datasets in the pre-training step. Then, in the domain-specific adaptation stage, these specific features are projected into a shared subspace. Moreover, the maximum mean discrepancy (MMD) is exploited to continuously narrow the differences in the subspace so that an inherent connection can be built between datasets. In addition, the attention mechanism is introduced to extract continuous information on spatial features, and the gated recurrent unit (GRU) is used to capture time series information.Results: The accuracy and root mean square error (RMSE) achieved by the proposed method are 59.10% and 0.27, respectively, which significantly outperforms state-of-the-art domain adaptation methods.Discussion: In addition, this study discusses the effect of labeled samples. When the number of labeled samples is 10% of the total number, the accuracy of the proposed model can reach 66.21%. This study fills a vacancy in the field of fatigue detection. In addition, the EEG-based cross-dataset fatigue detection method can be used for reference by other EEG-based deep learning research practices

Electroencephalogram (EEG)-based systems to monitor driver fatigue: a review

An efficient system that is capable to detect driver fatigue is urgently needed to help avoid road crashes. Recently, there has been an increase of interest in the application of electroencephalogram (EEG) to detect driver fatigue. Feature extraction and signal classification are the most critical steps in the EEG signal analysis. A reliable method for feature extraction is important to obtain robust signal classification. Meanwhile, a robust algorithm for signal classification will accurately classify the feature to a particular class. This paper concisely reviews the pros and cons of the existing techniques for feature extraction and signal classification and its fatigue detection accuracy performance. The integration of combined entropy (feature extraction) with support vector machine (SVM) and random forest (classifier) gives the best fatigue detection accuracy of 98.7% and 97.5% respectively. The outcomes from this study will guide future researchers in choosing a suitable technique for feature extraction and signal classification for EEG data processing and shed light on directions for future research and development of driver fatigue countermeasures

EEG-based driving fatigue detection using multilevel feature extraction and iterative hybrid feature selection

Brain activities can be evaluated by using Electroencephalogram (EEG) signals. One of the primary reasons for traffic accidents is driver fatigue, which can be identified by using EEG signals. This work aims to achieve a highly accurate and straightforward process to detect driving fatigue by using EEG signals. Two main problems, which are feature generation and feature selection, are defined to achieve this aim. This work solves these problems by using two different approaches. Deep networks are efficient feature generators and extract features in low, medium, and high levels. These features can be generated by using multileveled or multilayered feature extraction. Therefore, we proposed a multileveled feature generator that uses a one-dimensional binary pattern (BP) and statistical features together, and levels are created using a one-dimensional discrete wavelet transform (1D-DWT). A five-level fused feature extractor is presented by using BP, statistical features of 1D-DWT together. Moreover, a 2-layered feature selection method is proposed using ReliefF and iterative neighborhood component analysis (RFINCA) to solve the feature selection problem. The goals of the RFINCA are to choose the optimal number of features automatically and use the effectiveness of ReliefF and neighborhood component analysis (NCA) together. A driving fatigue EEG dataset was used as a testbed to denote the effectiveness of eighteen conventional classifiers. According to the experimental results, a highly accurate EEG classification approach is presented. The proposed method also reached 100.0% classification accuracy by using a k-nearest neighborhood classifier.</p

Random subspace K-NN based ensemble classifier for driver fatigue detection utilizing selected EEG channels

Nowadays, many studies have been conducted to assess driver fatigue, as it has become one of the leading causes of traffic crashes. However, with the use of advanced features and machine learning approaches, EEG signals may be processed in an effective way, allowing fatigue to be detected promptly and efficiently. An optimal channel selection approach and a competent classification algorithm might be viewed as a critical aspect of efficient fatigue detection by the driver. In the present framework, a new channel selection algorithm based on correlation coefficients and an ensemble classifier based on random subspace k-nearest neighbour (k-NN) has been presented to enhance the classification performance of EEG data for driver fatigue detection. Moreover, power spectral density (PSD) was used to extract the feature, confirming the presented method's robustness. Additionally, to make the fatigue detection system faster, we conducted the experiment in three different time windows, including 0.5s, 0.75s, and 1s. It was found that the proposed method attained classification accuracy of 99.99% in a 0.5 second time window to identify driver fatigue by means of EEG. The outstanding performance of the presented framework can be used effectively in EEG-based driver fatigue detection