Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring

How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal

Mutual Information in the Frequency Domain for Application in Biological Systems

Biological systems are comprised of multiple components that typically interact nonlinearly and produce multiple outputs (time series/signals) with specific frequency characteristics. Although the exact knowledge of the underlying mechanism remains unknown, the outputs observed from these systems can provide the dependency relations through quantitative methods and increase our understanding of the original systems. The nonlinear relations at specific frequencies require advanced dependency measures to capture the generalized interactions beyond typical correlation in the time domain or coherence in the frequency domain. Mutual information from Information Theory is such a quantity that can measure statistical dependency between random variables. Herein, we develop a model–free methodology for detection of nonlinear relations between time series with respect to frequency, that can quantify dependency under a general probabilistic framework. Classic nonlinear dynamical system and their coupled forms (Lorenz, bidirectionally coupled Lorenz, and unidirectionally coupled Macky–Glass systems) are employed to generate artificial data and to test the proposed methodology. Comparisons between the performances of this measure and a conventional linear measure are presented from applications to the artificial data. This set of results indicates that the proposed methodology is better in capturing the dependency between the variables of the systems. This measure of dependency is also applied to a real–world electrophysiological dataset for emotion analysis to study brain stimuli–response functional connectivity. The results reveal distinct brain regions and specific frequencies that are involved in emotional processing

Automated Classification for Electrophysiological Data: Machine Learning Approaches for Disease Detection and Emotion Recognition

Smart healthcare is a health service system that utilizes technologies, e.g., artificial intelligence and big data, to alleviate the pressures on healthcare systems. Much recent research has focused on the automatic disease diagnosis and recognition and, typically, our research pays attention on automatic classifications for electrophysiological signals, which are measurements of the electrical activity. Specifically, for electrocardiogram (ECG) and electroencephalogram (EEG) data, we develop a series of algorithms for automatic cardiovascular disease (CVD) classification, emotion recognition and seizure detection. With the ECG signals obtained from wearable devices, the candidate developed novel signal processing and machine learning method for continuous monitoring of heart conditions. Compared to the traditional methods based on the devices at clinical settings, the developed method in this thesis is much more convenient to use. To identify arrhythmia patterns from the noisy ECG signals obtained through the wearable devices, CNN and LSTM are used, and a wavelet-based CNN is proposed to enhance the performance. An emotion recognition method with a single channel ECG is developed, where a novel exploitative and explorative GWO-SVM algorithm is proposed to achieve high performance emotion classification. The attractive part is that the proposed algorithm has the capability to learn the SVM hyperparameters automatically, and it can prevent the algorithm from falling into local solutions, thereby achieving better performance than existing algorithms. A novel EEG-signal based seizure detector is developed, where the EEG signals are transformed to the spectral-temporal domain, so that the dimension of the input features to the CNN can be significantly reduced, while the detector can still achieve superior detection performance

Finding Frustration: a Dive into the EEG of Drivers

Emotion recognition technologies for driving are increasingly used to render automotive travel more pleasurable and, more importantly, safer. Since emotions such as frustration and anger can lead to an increase in traffic accidents, this thesis explored the utility of electroencephalogram (EEG) features to recognize the driver’s frustration level. It, therefore, sought to find a balance between the ecologically valid emotion induction of a driving simulator and the noise-sensitive but highly informative measure of the EEG. Participants’ brain activity was captured with the CGX quick-30 mobile EEG system. 19 participants completed four different frustration-inducing and two baseline driving scenarios in a 360° driving simulator. Subsequently, the participants continuously rated their frustration level based on the replay of each scenario. The resulting subjective measures were used to classify EEG time periods into episodes with or without frustration. Results showed that the frequently used measure of the Alpha Asymmetry Index (AAI) had, as hypothesized, significantly more negative indices for high frustration (vs. no frustration). However, a commingling effect of anger on this result could not be dismissed. The results could not provide evidence for the yet to be replicated previous research of frustration correlates within narrow-band oscillations (delta, theta, alpha, and beta) at specified electrode positions (frontal, central, and posterior). This thesis concludes with suggestions for subsequent research endeavors and forthcoming practical implications in the form of insights acquired

Emotion Recognition with Asymmetry Features of EEG Signals

Currently the study of affective computing (AC) includes a focus on researching emotion regulation and recognition. Recent studies in this field have utilized deep learning architectures to enhance emotion recognition from EEG signals. An alternative approach to deep learning is to use feature engineering to extract relevant features to train supervised machine learning models. Current theories in the neuroscience field can guide this feature engineering process. Neuroscientists have suggested various models to clarify how emotions are processed. One of these models suggests that positive emotions are processed in the left hemisphere, while negative emotions are processed in the right hemisphere. This emotional processing model has inspired previous studies to propose asymmetrical features to predict emotions. However, none of these studies have statistically evaluated whether the inclusion of asymmetrical features could yield benefits such as increased accuracy or reduced training time. To address that direction, this research presents both statistical evaluations for emotion regulation and a comparable model for emotion recognition. The outcomes show that brain hemispheres and frequency bands participate differently in processing emotions and observed the presence of the two asymmetry emotion processing models but in different frequency ranges. Also, the results from this study imply that by using asymmetry EEG, emotion recognition approaches can use fewer features without significantly compromising performance.Master of Science in Applied Computer Scienc

The Neural Detection of Emotion In Naturalistic Settings.

PhDThe Field of Emotion research has experienced resurgence partially due to the interest in Affective Computing, which includes calls for natural emotion to be studied in natural type settings. A new generation of commercial mobile EEG headsets present the potential for new forms of experimental design that may move beyond laboratory settings. Across the Arts and Cultural sectors there are longstanding questions of how we may objectively evaluate creative output, and also subjective responses to such artefacts. This research adjoins these concerns to ask; How can low-cost, portable EEG devices impact on our understanding of cultural experiences in the wild? Using a commercial emotiv Epoch EEG headset, we investigated gauging Valence and Arousal levels across the two contrasting experimental settings of a live theatre performance, and a controlled laboratory setting. Our results found that only Valence could be reliably detected, and only with a good degree of confidence in laboratory settings. This determines that we may only be able to gather very general information regarding cultural experiences via the enlisted EEG technology and methods, and only in controlled conditionsEPSR