EEG-Analysis for Cognitive Failure Detection in Driving Using Type-2 Fuzzy Classifiers

The paper aims at detecting on-line cognitive failures in driving by decoding the EEG signals acquired during visual alertness, motor-planning and motor-execution phases of the driver. Visual alertness of the driver is detected by classifying the pre-processed EEG signals obtained from his pre-frontal and frontal lobes into two classes: alert and non-alert. Motor-planning performed by the driver using the pre-processed parietal signals is classified into four classes: braking, acceleration, steering control and no operation. Cognitive failures in motor-planning are determined by comparing the classified motor-planning class of the driver with the ground truth class obtained from the co-pilot through a hand-held rotary switch. Lastly, failure in motor execution is detected, when the time-delay between the onset of motor imagination and the EMG response exceeds a predefined duration. The most important aspect of the present research lies in cognitive failure classification during the planning phase. The complexity in subjective plan classification arises due to possible overlap of signal features involved in braking, acceleration and steering control. A specialized interval/general type-2 fuzzy set induced neural classifier is employed to eliminate the uncertainty in classification of motor-planning. Experiments undertaken reveal that the proposed neuro-fuzzy classifier outperforms traditional techniques in presence of external disturbances to the driver. Decoding of visual alertness and motor-execution are performed with kernelized support vector machine classifiers. An analysis reveals that at a driving speed of 64 km/hr, the lead-time is over 600 milliseconds, which offer a safe distance of 10.66 meters

Exploration of Subjective Color Perceptual-Ability by EEG-Induced Type-2 Fuzzy Classifiers

Perceptual-ability informally refers to the ability of a person to recognize a stimulus. This paper deals with color perceptual-ability measurement of subjects using brain response to basic color (red, green and blue) stimuli. It also attempts to determine subjective ability to recognize the base colors in presence of noise tolerance of the base colors, referred to as recognition tolerance. Because of intra- and inter-session variations in subjective brain signal features for a given color stimulus, there exists uncertainty in perceptual-ability. In addition, small variations in the color stimulus result in wide variations in brain signal features, introducing uncertainty in perceptual-ability of the subject. Type-2 fuzzy logic has been employed to handle the uncertainty in color perceptual-ability measurements due to a) variations in brain signal features for a given color, and b) the presence of colored noise on the base colors. Because of limited power of uncertainty management of interval type-2 fuzzy sets and high computational overhead of its general type-2 counterpart, we developed a semi-general type-2 fuzzy classifier to recognize the base color. It is important to note that the proposed technique transforms a vertical slice based general type-2 fuzzy set into an equivalent interval type-2 counterpart to reduce the computational overhead, without losing the contributions of the secondary memberships. The proposed semi-general type-2 fuzzy sets induced classifier yields superior performance in classification accuracy with respect to existing type-1, type-2 and other well-known classifiers. The brain-understanding of a perceived base or noisy base colors is also obtained by exact low resolution electromagnetic topographic analysis (e-LORETA) software. This is used as the reference for our experimental results of the semi-general type-2 classifier in color perceptual-ability detection. Statistical tests undertaken confirm the superiority of the proposed classifier over its competitors. The proposed technique is expected to have interesting applications in identifying people with excellent color perceptual-ability for chemical, pharmaceutical and textile industries

Hemodynamic Analysis for Cognitive Load Assessment and Classification in Motor Learning Tasks Using Type-2 Fuzzy Sets

The paper addresses a novel approach to assess and classify the cognitive load of subjects from their hemodynamic response while engaged in motor learning tasks, such as vehicle-driving. A set of complex motor-activity-learning stimuli for braking, steering-control and acceleration is prepared to experimentally measure and classify the cognitive load of the car-drivers in three distinct classes: High, Medium and Low. New models of General and Interval Type-2 Fuzzy classifiers are proposed to reduce the scope of uncertainty in cognitive load classification due to the fluctuation of the hemodynamic features within and across sessions. The proposed classifiers offer high classification accuracy over 96%, leaving behind the traditional type-1/type-2 fuzzy and other standard classifiers. Experiments undertaken also offer a deep biological insight concerning the shift of brain-activations from the orbito-frontal to the ventro-lateral prefrontal cortex during high-to-low transition in cognitive load. Further, the activation of the dorsolateral prefrontal cortex is also reduced during low cognitive load of subjects. The proposed research outcome may directly be utilized to identify driving learners with low cognitive load for difficult motor learning tasks, such as taking a U-turn in a narrow space and motion control on the top of a bridge to avoid possible collision with the car ahead

Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of certain types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. Different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or the brain activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Further, potential topics in relation to driving behaviour are identified that could benefit from the adoption of neuroimaging methods in future studies

Multimodal Brain-Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines

Through this paper, we introduce a novel driver cognitive load assessment dataset, CL-Drive, which contains Electroencephalogram (EEG) signals along with other physiological signals such as Electrocardiography (ECG) and Electrodermal Activity (EDA) as well as eye tracking data. The data was collected from 21 subjects while driving in an immersive vehicle simulator, in various driving conditions, to induce different levels of cognitive load in the subjects. The tasks consisted of 9 complexity levels for 3 minutes each. Each driver reported their subjective cognitive load every 10 seconds throughout the experiment. The dataset contains the subjective cognitive load recorded as ground truth. In this paper, we also provide benchmark classification results for different machine learning and deep learning models for both binary and ternary label distributions. We followed 2 evaluation criteria namely 10-fold and leave-one-subject-out (LOSO). We have trained our models on both hand-crafted features as well as on raw data.Comment: 13 pages, 8 figures, 11 tables. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notic

Improving Engagement Assessment by Model Individualization and Deep Learning

This dissertation studies methods that improve engagement assessment for pilots. The major work addresses two challenging problems involved in the assessment: individual variation among pilots and the lack of labeled data for training assessment models. Task engagement is usually assessed by analyzing physiological measurements collected from subjects who are performing a task. However, physiological measurements such as Electroencephalography (EEG) vary from subject to subject. An assessment model trained for one subject may not be applicable to other subjects. We proposed a dynamic classifier selection algorithm for model individualization and compared it to other two methods: base line normalization and similarity-based model replacement. Experimental results showed that baseline normalization and dynamic classifier selection can significantly improve cross-subject engagement assessment. For complex tasks such as piloting an air plane, labeling engagement levels for pilots is challenging. Without enough labeled data, it is very difficult for traditional methods to train valid models for effective engagement assessment. This dissertation proposed to utilize deep learning models to address this challenge. Deep learning models are capable of learning valuable feature hierarchies by taking advantage of both labeled and unlabeled data. Our results showed that deep models are better tools for engagement assessment when label information is scarce. To further verify the power of deep learning techniques for scarce labeled data, we applied the deep learning algorithm to another small size data set, the ADNI data set. The ADNI data set is a public data set containing MRI and PET scans of Alzheimer\u27s Disease (AD) patients for AD diagnosis. We developed a robust deep learning system incorporating dropout and stability selection techniques to identify the different progression stages of AD patients. The experimental results showed that deep learning is very effective in AD diagnosis. In addition, we studied several imbalance learning techniques that are useful when data is highly unbalanced, i.e., when majority classes have many more training samples than minority classes. Conventional machine learning techniques usually tend to classify all data samples into majority classes and to perform poorly for minority classes. Unbalanced learning techniques can balance data sets before training and can improve learning performance