708 research outputs found
Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring
How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks
Electroencephalography (EEG) measures the neuronal activities in different
brain regions via electrodes. Many existing studies on EEG-based emotion
recognition do not fully exploit the topology of EEG channels. In this paper,
we propose a regularized graph neural network (RGNN) for EEG-based emotion
recognition. RGNN considers the biological topology among different brain
regions to capture both local and global relations among different EEG
channels. Specifically, we model the inter-channel relations in EEG signals via
an adjacency matrix in a graph neural network where the connection and
sparseness of the adjacency matrix are inspired by neuroscience theories of
human brain organization. In addition, we propose two regularizers, namely
node-wise domain adversarial training (NodeDAT) and emotion-aware distribution
learning (EmotionDL), to better handle cross-subject EEG variations and noisy
labels, respectively. Extensive experiments on two public datasets, SEED and
SEED-IV, demonstrate the superior performance of our model than
state-of-the-art models in most experimental settings. Moreover, ablation
studies show that the proposed adjacency matrix and two regularizers contribute
consistent and significant gain to the performance of our RGNN model. Finally,
investigations on the neuronal activities reveal important brain regions and
inter-channel relations for EEG-based emotion recognition
Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals
An electroencephalography (EEG) based Brain Computer Interface (BCI) enables
people to communicate with the outside world by interpreting the EEG signals of
their brains to interact with devices such as wheelchairs and intelligent
robots. More specifically, motor imagery EEG (MI-EEG), which reflects a
subjects active intent, is attracting increasing attention for a variety of BCI
applications. Accurate classification of MI-EEG signals while essential for
effective operation of BCI systems, is challenging due to the significant noise
inherent in the signals and the lack of informative correlation between the
signals and brain activities. In this paper, we propose a novel deep neural
network based learning framework that affords perceptive insights into the
relationship between the MI-EEG data and brain activities. We design a joint
convolutional recurrent neural network that simultaneously learns robust
high-level feature presentations through low-dimensional dense embeddings from
raw MI-EEG signals. We also employ an Autoencoder layer to eliminate various
artifacts such as background activities. The proposed approach has been
evaluated extensively on a large- scale public MI-EEG dataset and a limited but
easy-to-deploy dataset collected in our lab. The results show that our approach
outperforms a series of baselines and the competitive state-of-the- art
methods, yielding a classification accuracy of 95.53%. The applicability of our
proposed approach is further demonstrated with a practical BCI system for
typing.Comment: 10 page
STILN: A Novel Spatial-Temporal Information Learning Network for EEG-based Emotion Recognition
The spatial correlations and the temporal contexts are indispensable in
Electroencephalogram (EEG)-based emotion recognition. However, the learning of
complex spatial correlations among several channels is a challenging problem.
Besides, the temporal contexts learning is beneficial to emphasize the critical
EEG frames because the subjects only reach the prospective emotion during part
of stimuli. Hence, we propose a novel Spatial-Temporal Information Learning
Network (STILN) to extract the discriminative features by capturing the spatial
correlations and temporal contexts. Specifically, the generated 2D power
topographic maps capture the dependencies among electrodes, and they are fed to
the CNN-based spatial feature extraction network. Furthermore, Convolutional
Block Attention Module (CBAM) recalibrates the weights of power topographic
maps to emphasize the crucial brain regions and frequency bands. Meanwhile,
Batch Normalizations (BNs) and Instance Normalizations (INs) are appropriately
combined to relieve the individual differences. In the temporal contexts
learning, we adopt the Bidirectional Long Short-Term Memory Network (Bi-LSTM)
network to capture the dependencies among the EEG frames. To validate the
effectiveness of the proposed method, subject-independent experiments are
conducted on the public DEAP dataset. The proposed method has achieved the
outstanding performance, and the accuracies of arousal and valence
classification have reached 0.6831 and 0.6752 respectively
Deep Learning Model With Adaptive Regularization for EEG-Based Emotion Recognition Using Temporal and Frequency Features
Since EEG signal acquisition is non-invasive and portable, it is convenient to be used for different applications. Recognizing emotions based on Brain-Computer Interface (BCI) is an important active BCI paradigm for recognizing the inner state of persons. There are extensive studies about emotion recognition, most of which heavily rely on staged complex handcrafted EEG feature extraction and classifier design. In this paper, we propose a hybrid multi-input deep model with convolution neural networks (CNNs) and bidirectional Long Short-term Memory (Bi-LSTM). CNNs extract time-invariant features from raw EEG data, and Bi-LSTM allows long-range lateral interactions between features. First, we propose a novel hybrid multi-input deep learning approach for emotion recognition from raw EEG signals. Second, in the first layers, we use two CNNs with small and large filter sizes to extract temporal and frequency features from each raw EEG epoch of 62-channel 2-s and merge with differential entropy of EEG band. Third, we apply the adaptive regularization method over each parallel CNN’s layer to consider the spatial information of EEG acquisition electrodes. The proposed method is evaluated on two public datasets, SEED and DEAP. Our results show that our technique can significantly improve the accuracy in comparison with the baseline where no adaptive regularization techniques are used
Noise Reduction of EEG Signals Using Autoencoders Built Upon GRU based RNN Layers
Understanding the cognitive and functional behaviour of the brain by its electrical activity is an important area of research. Electroencephalography (EEG) is a method that measures and record electrical activities of the brain from the scalp. It has been used for pathology analysis, emotion recognition, clinical and cognitive research, diagnosing various neurological and psychiatric disorders and for other applications. Since the EEG signals are sensitive to activities other than the brain ones, such as eye blinking, eye movement, head movement, etc., it is not possible to record EEG signals without any noise. Thus, it is very important to use an efficient noise reduction technique to get more accurate recordings. Numerous traditional techniques such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), wavelet transformations and machine learning techniques were proposed for reducing the noise in EEG signals. The aim of this paper is to investigate the effectiveness of stacked autoencoders built upon Gated Recurrent Unit (GRU) based Recurrent Neural Network (RNN) layers (GRU-AE) against PCA. To achieve this, Harrell-Davis decile values for the reconstructed signals’ signal-to- noise ratio distributions were compared and it was found that the GRU-AE outperformed PCA for noise reduction of EEG signals
A Hybrid End-to-End Spatio-Temporal Attention Neural Network with Graph-Smooth Signals for EEG Emotion Recognition
Recently, physiological data such as electroencephalography (EEG) signals
have attracted significant attention in affective computing. In this context,
the main goal is to design an automated model that can assess emotional states.
Lately, deep neural networks have shown promising performance in emotion
recognition tasks. However, designing a deep architecture that can extract
practical information from raw data is still a challenge. Here, we introduce a
deep neural network that acquires interpretable physiological representations
by a hybrid structure of spatio-temporal encoding and recurrent attention
network blocks. Furthermore, a preprocessing step is applied to the raw data
using graph signal processing tools to perform graph smoothing in the spatial
domain. We demonstrate that our proposed architecture exceeds state-of-the-art
results for emotion classification on the publicly available DEAP dataset. To
explore the generality of the learned model, we also evaluate the performance
of our architecture towards transfer learning (TL) by transferring the model
parameters from a specific source to other target domains. Using DEAP as the
source dataset, we demonstrate the effectiveness of our model in performing
cross-modality TL and improving emotion classification accuracy on DREAMER and
the Emotional English Word (EEWD) datasets, which involve EEG-based emotion
classification tasks with different stimuli
Spatial-temporal Transformers for EEG Emotion Recognition
Electroencephalography (EEG) is a popular and effective tool for emotion
recognition. However, the propagation mechanisms of EEG in the human brain and
its intrinsic correlation with emotions are still obscure to researchers. This
work proposes four variant transformer frameworks~(spatial attention, temporal
attention, sequential spatial-temporal attention and simultaneous
spatial-temporal attention) for EEG emotion recognition to explore the
relationship between emotion and spatial-temporal EEG features. Specifically,
spatial attention and temporal attention are to learn the topological structure
information and time-varying EEG characteristics for emotion recognition
respectively. Sequential spatial-temporal attention does the spatial attention
within a one-second segment and temporal attention within one sample
sequentially to explore the influence degree of emotional stimulation on EEG
signals of diverse EEG electrodes in the same temporal segment. The
simultaneous spatial-temporal attention, whose spatial and temporal attention
are performed simultaneously, is used to model the relationship between
different spatial features in different time segments. The experimental results
demonstrate that simultaneous spatial-temporal attention leads to the best
emotion recognition accuracy among the design choices, indicating modeling the
correlation of spatial and temporal features of EEG signals is significant to
emotion recognition
CNN and LSTM-Based Emotion Charting Using Physiological Signals
Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of 98.73% is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral-and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.Peer reviewe
- …