Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases

Deep learning and wearable sensors for the diagnosis and monitoring of Parkinson’s disease: A systematic review

Parkinson’s disease (PD) is a neurodegenerative disorder that produces both motor and non-motor complications, degrading the quality of life of PD patients. Over the past two decades, the use of wearable devices in combination with machine learning algorithms has provided promising methods for more objective and continuous monitoring of PD. Recent advances in artificial intelligence have provided new methods and algorithms for data analysis, such as deep learning (DL). The aim of this article is to provide a comprehensive review of current applications where DL algorithms are employed for the assessment of motor and nonmotor manifestations (NMM) using data collected via wearable sensors. This paper provides the reader with a summary of the current applications of DL and wearable devices for the diagnosis, prognosis, and monitoring of PD, in the hope of improving the adoption, applicability, and impact of both technologies as support tools. Following PRISMA (Systematic Reviews and Meta-Analyses) guidelines, sixty-nine studies were selected and analyzed. For each study, information on sample size, sensor configuration, DL approaches, validation methods, and results according to the specific symptom under study were extracted and summarized. Furthermore, quality assessment was conducted according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) method. The majority of studies (74%) were published within the last three years, demonstrating the increasing focus on wearable technology and DL approaches for PD assessment. However, most papers focused on monitoring (59%) and computer-assisted diagnosis (37%), while few papers attempted to predict treatment response. Motor symptoms (86%) were treated much more frequently than NMM (14%). Inertial sensors were the most commonly used technology, followed by force sensors and microphones. Finally, convolutional neural networks (52%) were preferred to other DL approaches, while extracted features (38%) and raw data (37%) were similarly used as input for DL models. The results of this review highlight several challenges related to the use of wearable technology and DL methods in the assessment of PD, despite the advantages this technology could bring in the development and implementation of automated systems for PD assessment

Confidence Inference in Defensive Cyber Operator Decision Making

Cyber defense analysts face the challenge of validating machine generated alerts regarding network-based security threats. Operations tempo and systematic manpower issues have increased the importance of these individual analyst decisions, since they typically are not reviewed or changed. Analysts may not always be confident in their decisions. If confidence can be accurately assessed, then analyst decisions made under low confidence can be independently reviewed and analysts can be offered decision assistance or additional training. This work investigates the utility of using neurophysiological and behavioral correlates of decision confidence to train machine learning models to infer confidence in analyst decisions. Electroencephalography (EEG) and behavioral data was collected from eight participants in a two-task human-subject experiment and used to fit several popular classifiers. Results suggest that for simple decisions, it is possible to classify analyst decision confidence using EEG signals. However, more work is required to evaluate the utility of EEG signals for classification of decision confidence in complex decisions

Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey

Time Series Classification and Extrinsic Regression are important and challenging machine learning tasks. Deep learning has revolutionized natural language processing and computer vision and holds great promise in other fields such as time series analysis where the relevant features must often be abstracted from the raw data but are not known a priori. This paper surveys the current state of the art in the fast-moving field of deep learning for time series classification and extrinsic regression. We review different network architectures and training methods used for these tasks and discuss the challenges and opportunities when applying deep learning to time series data. We also summarize two critical applications of time series classification and extrinsic regression, human activity recognition and satellite earth observation

Advanced Biometrics with Deep Learning

Biometrics, such as fingerprint, iris, face, hand print, hand vein, speech and gait recognition, etc., as a means of identity management have become commonplace nowadays for various applications. Biometric systems follow a typical pipeline, that is composed of separate preprocessing, feature extraction and classification. Deep learning as a data-driven representation learning approach has been shown to be a promising alternative to conventional data-agnostic and handcrafted pre-processing and feature extraction for biometric systems. Furthermore, deep learning offers an end-to-end learning paradigm to unify preprocessing, feature extraction, and recognition, based solely on biometric data. This Special Issue has collected 12 high-quality, state-of-the-art research papers that deal with challenging issues in advanced biometric systems based on deep learning. The 12 papers can be divided into 4 categories according to biometric modality; namely, face biometrics, medical electronic signals (EEG and ECG), voice print, and others

Deep Learning Techniques for Electroencephalography Analysis

In this thesis we design deep learning techniques for training deep neural networks on electroencephalography (EEG) data and in particular on two problems, namely EEG-based motor imagery decoding and EEG-based affect recognition, addressing challenges associated with them. Regarding the problem of motor imagery (MI) decoding, we first consider the various kinds of domain shifts in the EEG signals, caused by inter-individual differences (e.g. brain anatomy, personality and cognitive profile). These domain shifts render multi-subject training a challenging task and impede robust cross-subject generalization. We build a two-stage model ensemble architecture and propose two objectives to train it, combining the strengths of curriculum learning and collaborative training. Our subject-independent experiments on the large datasets of Physionet and OpenBMI, verify the effectiveness of our approach. Next, we explore the utilization of the spatial covariance of EEG signals through alignment techniques, with the goal of learning domain-invariant representations. We introduce a Riemannian framework that concurrently performs covariance-based signal alignment and data augmentation, while training a convolutional neural network (CNN) on EEG time-series. Experiments on the BCI IV-2a dataset show that our method performs superiorly over traditional alignment, by inducing regularization to the weights of the CNN. We also study the problem of EEG-based affect recognition, inspired by works suggesting that emotions can be expressed in relative terms, i.e. through ordinal comparisons between different affective state levels. We propose treating data samples in a pairwise manner to infer the ordinal relation between their corresponding affective state labels, as an auxiliary training objective. We incorporate our objective in a deep network architecture which we jointly train on the tasks of sample-wise classification and pairwise ordinal ranking. We evaluate our method on the affective datasets of DEAP and SEED and obtain performance improvements over deep networks trained without the additional ranking objective

Deep learning classification model of mental workload levels using EEG signals

Understanding and improving humance performance, especially in situations that require safety, productivity, and well-being, relies on categorising mental workload (MWL). Traditional methods for measuring MWL, such as in driving and piloting, have given us some understanding, but these methods must accurately distinguish between low and high workload levels. Excessive work can tyre participants, while insufficient work can make them bored and inefficient. Traditional MWL assessment tools, such as questionnaires, sometimes make it harder for people to manage their MWL, especially when they struggle to express or understand their thoughts and feelings. The recent work shift to neurophysiological signals, specifically electroencephalogram (EEG), provides a promising way to measure brain activity related to MWL non-invasively. Advanced techniques such as deep learning have made it easier to study EEG signals in more detail. Our goal was to develop a clear and consistent approach for using EEG signals to classify MWL effectively. Our approach focused on each process stage, from preparing the data to evaluating the model and addressing common mistakes and misunderstandings in current techniques. The first study addresses the challenges of using EEG data contaminated by artefacts for assessing MWL. EEG signal artefacts, such as eye movement or muscle activity, can skew MWL assessment. Recently, there has been significant progress in using deep learning models to interpret EEG signals, but the challenge remains. The preprocessing pipeline for EEG artefact removal is broad and inconsistently adopted; some pipelines are time-consuming and contain human intervention steps, so they are unsuitable for automation systems. Therefore, this study focused on automatic EEG artefact removal for deep learning analysis. Furthermore, we examined the impact of various preprocessing techniques on the effectiveness of deep learning models in classifying MWL levels. We used state-of-the-art models such as Stacked LSTM, BLSTM, and BLSTM-LSTM, and found that certain techniques—specifically, the ADJUST algorithm—significantly enhanced model performance. However, the sophisticated models could extract relevant information from raw data, indicating a reduced need for preprocessing. The second study shifted the focus to channel selection to refine the automation of MWL classification and reduce unnecessary computational expenses by using unnecessary electrodes, aligning more closely to real-world applications. We prioritised the best electrode setup focusing on brain activity related to MWL. We removed unnecessary data using Riemannian geometry, an effective method for EEG channel selection. We aimed to balance information sufficiency with computational efficiency and to reduce the number of electrodes. The study also evaluated covariance estimators for Riemannian geometry for their effectiveness in channel selection and impact on deep learning models for MWL classification, as the traditional Empirical Covariance (EC) has limitations for the EEG signal. Finally, the third study tackled a critical but frequently overlooked aspect of MWL level classification using machine learning or deep learning techniques: the temporal nature of EEG signals. We underscored that the traditional cross-validation technique violates the sequential nature of time series data, leading to data leakage, model overfitting, and inaccurate MWL assessment. Specifically, to predict the subject’s MWL level, we could not randomly split data and use future data to train the model and predict the previous MWL level. To address this problem, this study focused on the model training phase, specifically on the importance of time series cross-validation methods. We adopted the expanding window and rolling window strategies, finding that using the expanding window strategy outperformed those using the rolling window strategy. This research carefully developed a comprehensive and consistent method for classifying MWL using EEG signals. We aimed to correct misunderstandings and set a standard in brain-computer interface (BCI) systems. This will help guide future research and development efforts.Understanding and improving humance performance, especially in situations that require safety, productivity, and well-being, relies on categorising mental workload (MWL). Traditional methods for measuring MWL, such as in driving and piloting, have given us some understanding, but these methods must accurately distinguish between low and high workload levels. Excessive work can tyre participants, while insufficient work can make them bored and inefficient. Traditional MWL assessment tools, such as questionnaires, sometimes make it harder for people to manage their MWL, especially when they struggle to express or understand their thoughts and feelings. The recent work shift to neurophysiological signals, specifically electroencephalogram (EEG), provides a promising way to measure brain activity related to MWL non-invasively. Advanced techniques such as deep learning have made it easier to study EEG signals in more detail. Our goal was to develop a clear and consistent approach for using EEG signals to classify MWL effectively. Our approach focused on each process stage, from preparing the data to evaluating the model and addressing common mistakes and misunderstandings in current techniques. The first study addresses the challenges of using EEG data contaminated by artefacts for assessing MWL. EEG signal artefacts, such as eye movement or muscle activity, can skew MWL assessment. Recently, there has been significant progress in using deep learning models to interpret EEG signals, but the challenge remains. The preprocessing pipeline for EEG artefact removal is broad and inconsistently adopted; some pipelines are time-consuming and contain human intervention steps, so they are unsuitable for automation systems. Therefore, this study focused on automatic EEG artefact removal for deep learning analysis. Furthermore, we examined the impact of various preprocessing techniques on the effectiveness of deep learning models in classifying MWL levels. We used state-of-the-art models such as Stacked LSTM, BLSTM, and BLSTM-LSTM, and found that certain techniques—specifically, the ADJUST algorithm—significantly enhanced model performance. However, the sophisticated models could extract relevant information from raw data, indicating a reduced need for preprocessing. The second study shifted the focus to channel selection to refine the automation of MWL classification and reduce unnecessary computational expenses by using unnecessary electrodes, aligning more closely to real-world applications. We prioritised the best electrode setup focusing on brain activity related to MWL. We removed unnecessary data using Riemannian geometry, an effective method for EEG channel selection. We aimed to balance information sufficiency with computational efficiency and to reduce the number of electrodes. The study also evaluated covariance estimators for Riemannian geometry for their effectiveness in channel selection and impact on deep learning models for MWL classification, as the traditional Empirical Covariance (EC) has limitations for the EEG signal. Finally, the third study tackled a critical but frequently overlooked aspect of MWL level classification using machine learning or deep learning techniques: the temporal nature of EEG signals. We underscored that the traditional cross-validation technique violates the sequential nature of time series data, leading to data leakage, model overfitting, and inaccurate MWL assessment. Specifically, to predict the subject’s MWL level, we could not randomly split data and use future data to train the model and predict the previous MWL level. To address this problem, this study focused on the model training phase, specifically on the importance of time series cross-validation methods. We adopted the expanding window and rolling window strategies, finding that using the expanding window strategy outperformed those using the rolling window strategy. This research carefully developed a comprehensive and consistent method for classifying MWL using EEG signals. We aimed to correct misunderstandings and set a standard in brain-computer interface (BCI) systems. This will help guide future research and development efforts