AFFECTIVE COMPUTING AND AUGMENTED REALITY FOR CAR DRIVING SIMULATORS

Car simulators are essential for training and for analyzing the behavior, the responses and the performance of the driver. Augmented Reality (AR) is the technology that enables virtual images to be overlaid on views of the real world. Affective Computing (AC) is the technology that helps reading emotions by means of computer systems, by analyzing body gestures, facial expressions, speech and physiological signals. The key aspect of the research relies on investigating novel interfaces that help building situational awareness and emotional awareness, to enable affect-driven remote collaboration in AR for car driving simulators. The problem addressed relates to the question about how to build situational awareness (using AR technology) and emotional awareness (by AC technology), and how to integrate these two distinct technologies [4], into a unique affective framework for training, in a car driving simulator

Systematic Review of Experimental Paradigms and Deep Neural Networks for Electroencephalography-Based Cognitive Workload Detection

This article summarizes a systematic review of the electroencephalography (EEG)-based cognitive workload (CWL) estimation. The focus of the article is twofold: identify the disparate experimental paradigms used for reliably eliciting discreet and quantifiable levels of cognitive load and the specific nature and representational structure of the commonly used input formulations in deep neural networks (DNNs) used for signal classification. The analysis revealed a number of studies using EEG signals in its native representation of a two-dimensional matrix for offline classification of CWL. However, only a few studies adopted an online or pseudo-online classification strategy for real-time CWL estimation. Further, only a couple of interpretable DNNs and a single generative model were employed for cognitive load detection till date during this review. More often than not, researchers were using DNNs as black-box type models. In conclusion, DNNs prove to be valuable tools for classifying EEG signals, primarily due to the substantial modeling power provided by the depth of their network architecture. It is further suggested that interpretable and explainable DNN models must be employed for cognitive workload estimation since existing methods are limited in the face of the non-stationary nature of the signal.Comment: 10 Pages, 4 figure

InfoFlowNet: A Multi-head Attention-based Self-supervised Learning Model with Surrogate Approach for Uncovering Brain Effective Connectivity

Deciphering brain network topology can enhance the depth of neuroscientific knowledge and facilitate the development of neural engineering methods. Effective connectivity, a measure of brain network dynamics, is particularly useful for investigating the directional influences among different brain regions. In this study, we introduce a novel brain causal inference model named InfoFlowNet, which leverages the self-attention mechanism to capture associations among electroencephalogram (EEG) time series. The proposed method estimates the magnitude of directional information flow (dIF) among EEG processes by measuring the loss of model inference resulting from the shuffling of the time order of the original time series. To evaluate the feasibility of InfoFlowNet, we conducted experiments using a synthetic time series and two EEG datasets. The results demonstrate that InfoFlowNet can extract time-varying causal relationships among processes, reflected in the fluctuation of dIF values. Compared with the Granger causality model and temporal causal discovery framework, InfoFlowNet can identify more significant causal edges underlying EEG processes while maintaining an acceptable computation time. Our work demonstrates the potential of InfoFlowNet for analyzing effective connectivity in EEG data. The findings highlight the importance of effective connectivity in understanding the complex dynamics of the brain network

RFNet: Riemannian Fusion Network for EEG-based Brain-Computer Interfaces

This paper presents the novel Riemannian Fusion Network (RFNet), a deep neural architecture for learning spatial and temporal information from Electroencephalogram (EEG) for a number of different EEG-based Brain Computer Interface (BCI) tasks and applications. The spatial information relies on Spatial Covariance Matrices (SCM) of multi-channel EEG, whose space form a Riemannian Manifold due to the Symmetric and Positive Definite structure. We exploit a Riemannian approach to map spatial information onto feature vectors in Euclidean space. The temporal information characterized by features based on differential entropy and logarithm power spectrum density is extracted from different windows through time. Our network then learns the temporal information by employing a deep long short-term memory network with a soft attention mechanism. The output of the attention mechanism is used as the temporal feature vector. To effectively fuse spatial and temporal information, we use an effective fusion strategy, which learns attention weights applied to embedding-specific features for decision making. We evaluate our proposed framework on four public datasets from three popular fields of BCI, notably emotion recognition, vigilance estimation, and motor imagery classification, containing various types of tasks such as binary classification, multi-class classification, and regression. RFNet approaches the state-of-the-art on one dataset (SEED) and outperforms other methods on the other three datasets (SEED-VIG, BCI-IV 2A, and BCI-IV 2B), setting new state-of-the-art values and showing the robustness of our framework in EEG representation learning

Using EEG measures to quantify reduced daytime vigilance in patients diagnosed with obstructive sleep apnoea using a novel electroencephalogram analysis method

Introduction Vigilance in obstructive sleep apnoea (OSA) does not correlate well with disease severity/ symptoms: Hence the need for a simple objective test. One such method could be quantitative analysis of the awake electroencephalogram (qEEG). qEEG is conventionally analysed using Power Spectral Analysis (PSA) looking at different EEG frequencies of delta, theta, alpha and beta. A novel method of analyzing the qEEG: De-trended fluctuation analysis (DFA) provides a single value: the scaling exponent (SE), which measures the fluctuations in the EEG signal. Artefact removal from qEEG is mandatory with the gold standard being manual scoring. Another method of automated artefact removal is independent component analysis (ICA). Objective Investigate the role of PSA and DFA (SE) as an objective measure of testing vigilance and validate the use of ICA in patients diagnosed with OSA. Methodology Retrospective cross-sectional study of untreated OSA patients. Results ICA and manual artefact removal gave well-correlated results in the DFA (SE), but not PSA. EEG slowing measured by PSA and DFA did not correlate to impaired performance during a battery of 14 separate performance tests. Conclusion ICA and manual artefact removal can be interchangeably used in extracting DFA measurements with confidence. In PSA metrics the use of ICA may not be reliable

Using EEG measures to quantify reduced daytime vigilance in patients diagnosed with obstructive sleep apnoea using a novel electroencephalogram analysis method

Introduction Vigilance in obstructive sleep apnoea (OSA) does not correlate well with disease severity/ symptoms: Hence the need for a simple objective test. One such method could be quantitative analysis of the awake electroencephalogram (qEEG). qEEG is conventionally analysed using Power Spectral Analysis (PSA) looking at different EEG frequencies of delta, theta, alpha and beta. A novel method of analyzing the qEEG: De-trended fluctuation analysis (DFA) provides a single value: the scaling exponent (SE), which measures the fluctuations in the EEG signal. Artefact removal from qEEG is mandatory with the gold standard being manual scoring. Another method of automated artefact removal is independent component analysis (ICA). Objective Investigate the role of PSA and DFA (SE) as an objective measure of testing vigilance and validate the use of ICA in patients diagnosed with OSA. Methodology Retrospective cross-sectional study of untreated OSA patients. Results ICA and manual artefact removal gave well-correlated results in the DFA (SE), but not PSA. EEG slowing measured by PSA and DFA did not correlate to impaired performance during a battery of 14 separate performance tests. Conclusion ICA and manual artefact removal can be interchangeably used in extracting DFA measurements with confidence. In PSA metrics the use of ICA may not be reliable

Methods and Apparatus for Autonomous Robotic Control

Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on "stovepiped," or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements

Improving EEG-based driver fatigue classification using sparse-deep belief networks

© 2017 Chai, Ling, San, Naik, Nguyen, Tran, Craig and Nguyen. This paper presents an improvement of classification performance for electroencephalography (EEG)-based driver fatigue classification between fatigue and alert states with the data collected from 43 participants. The system employs autoregressive (AR) modeling as the features extraction algorithm, and sparse-deep belief networks (sparse-DBN) as the classification algorithm. Compared to other classifiers, sparse-DBN is a semi supervised learning method which combines unsupervised learning for modeling features in the pre-training layer and supervised learning for classification in the following layer. The sparsity in sparse-DBN is achieved with a regularization term that penalizes a deviation of the expected activation of hidden units from a fixed low-level prevents the network from overfitting and is able to learn low-level structures as well as high-level structures. For comparison, the artificial neural networks (ANN), Bayesian neural networks (BNN), and original deep belief networks (DBN) classifiers are used. The classification results show that using AR feature extractor and DBN classifiers, the classification performance achieves an improved classification performance with a of sensitivity of 90.8%, a specificity of 90.4%, an accuracy of 90.6%, and an area under the receiver operating curve (AUROC) of 0.94 compared to ANN (sensitivity at 80.8%, specificity at 77.8%, accuracy at 79.3% with AUC-ROC of 0.83) and BNN classifiers (sensitivity at 84.3%, specificity at 83%, accuracy at 83.6% with AUROC of 0.87). Using the sparse-DBN classifier, the classification performance improved further with sensitivity of 93.9%, a specificity of 92.3%, and an accuracy of 93.1% with AUROC of 0.96. Overall, the sparse-DBN classifier improved accuracy by 13.8, 9.5, and 2.5% over ANN, BNN, and DBN classifiers, respectively

Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm

Driver drowsiness and loss of vigilance are a major cause of road accidents. Monitoring physiological signals while driving provides the possibility of detecting and warning of drowsiness and fatigue. The aim of this paper is to maximize the amount of drowsiness-related information extracted from a set of electroencephalogram (EEG), electrooculogram (EOG), and electrocardiogram (ECG) signals during a simulation driving test. Specifically, we develop an efficient fuzzy mutual-information (MI)- based wavelet packet transform (FMIWPT) feature-extraction method for classifying the driver drowsiness state into one of predefined drowsiness levels. The proposed method estimates the required MI using a novel approach based on fuzzy memberships providing an accurate-information content-estimation measure. The quality of the extracted features was assessed on datasets collected from 31 drivers on a simulation test. The experimental results proved the significance of FMIWPT in extracting features that highly correlate with the different drowsiness levels achieving a classification accuracy of 95%-97% on an average across all subjects. © 2011 IEEE

Vision-Based distraction analysis tested on a realistic driving simulator

Abstract-This paper presents a non intrusive approach to obtain driver's face pose estimation based on stereo graylevel image processing. Face pose estimation is based on an automatic and incremental 3D model creation and its correct tracking. From this information, gaze focalization in the scene is calculated in order to detect driver distraction. Different distraction activities are inferred in a realistic simulator and a study of the incidence of these distracting activities in the driver's behaviour is carried out. Some experimental results and conclusions are presented