An approach to emotion recognition in single-channel EEG signals: a mother child interaction

In this work, we perform a first approach to emotion recognition from EEG single channel signals extracted in four (4) mother-child dyads experiment in developmental psychology -- Single channel EEG signals are analyzed and processed using several window sizes by performing a statistical analysis over features in the time and frequency domains -- Finally, a neural network obtained an average accuracy rate of 99% of classification in two emotional states such as happiness and sadness20th Argentinean Bioengineering Society Congress, SABI 2015 (XX Congreso Argentino de Bioingeniería y IX Jornadas de Ingeniería Clínica)28–30 October 2015, San Nicolás de los Arroyos, Argentin

An approach to emotion recognition in single-channel EEG signals: a mother child interaction

In this work, we perform a first approach to emotion recognition from EEG single channel signals extracted in four (4) mother-child dyads experiment in developmental psychology -- Single channel EEG signals are analyzed and processed using several window sizes by performing a statistical analysis over features in the time and frequency domains -- Finally, a neural network obtained an average accuracy rate of 99% of classification in two emotional states such as happiness and sadness20th Argentinean Bioengineering Society Congress, SABI 2015 (XX Congreso Argentino de Bioingeniería y IX Jornadas de Ingeniería Clínica)28–30 October 2015, San Nicolás de los Arroyos, Argentin

Optimization of least squares support vector machine technique using genetic algorithm for electroencephalogram multi-dimensional signals

Human-computer intelligent interaction (HCII) is a rising field of science that aims to refine and enhance the interaction between computer and human. Since emotion plays a vital role in human daily life, the ability of computer to interpret and response to human emotion is a crucial element for future intelligent system.Accordingly, several studies have been conducted to recognise human emotion using different technique such as facial expression, speech, galvanic skin response (GSR), or heart rate (HR).However, such techniques have problems mainly in terms of credibility and reliability as people can fake their feeling and response. Electroencephalogram (EEG) on the other has shown to be a very effective way in recognising human emotion as this technique records the brain activity of human and they can hardly be deceived by voluntary control. Regardless the popularity of EEG in recognizing human emotion, this study field is relatively challenging as EEG signal is nonlinear, involves myriad factors and chaotic in nature.These issues have led to high dimensional problem and poor classification results.To address such problems, this study has proposed a novel computational model, which consist of three main stages, namely a) feature extraction; b) feature selection and c) classifier. Discrete wavelet packet transform (DWPT) has been used to extract EEG signals feature and ultimately 204,800 features from 32 subject-independent have been obtained. Meanwhile, Genetic Algorithm (GA) and Least squares support vector machine (LS-SVM) have been used as a feature selection technique and classifier respectively.This computational model is tested on the common DEAP pre-processed EEG dataset in order to classify three levels of valence and arousal.The empirical results have shown that the proposed GA-LSSVM, has improved the classification results to 49.22% and 54.83% for valence and arousal respectively, whereas is it observed that 46.33% of valence and 48.30% of arousal classification were achieved when no feature selection technique is applied on the identical classifier

EEG Based Emotion Monitoring Using Wavelet and Learning Vector Quantization

Emotional identification is necessary for example in Brain Computer Interface (BCI) application and when emotional therapy and medical rehabilitation take place. Some emotional states can be characterized in the frequency of EEG signal, such excited, relax and sad. The signal extracted in certain frequency useful to distinguish the three emotional state. The classification of the EEG signal in real time depends on extraction methods to increase class distinction, and identification methods with fast computing. This paper proposed human emotion monitoring in real time using Wavelet and Learning Vector Quantization (LVQ). The process was done before the machine learning using training data from the 10 subjects, 10 trial, 3 classes and 16 segments (equal to 480 sets of data). Each data set processed in 10 seconds and extracted into Alpha, Beta, and Theta waves using Wavelet. Then they become input for the identification system using LVQ three emotional state that is excited, relax, and sad. The results showed that by using wavelet we can improve the accuracy of 72% to 87% and number of training data variation increased the accuracy. The system was integrated with wireless EEG to monitor emotion state in real time with change each 10 seconds. It takes 0.44 second, was not significant toward 10 seconds

Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition

The electroencephalogram (EEG) has great attraction in emotion recognition studies due to its resistance to deceptive actions of humans. This is one of the most significant advantages of brain signals in comparison to visual or speech signals in the emotion recognition context. A major challenge in EEG-based emotion recognition is that EEG recordings exhibit varying distributions for different people as well as for the same person at different time instances. This nonstationary nature of EEG limits the accuracy of it when subject independency is the priority. The aim of this study is to increase the subject-independent recognition accuracy by exploiting pretrained state-of-the-art Convolutional Neural Network (CNN) architectures. Unlike similar studies that extract spectral band power features from the EEG readings, raw EEG data is used in our study after applying windowing, pre-adjustments and normalization. Removing manual feature extraction from the training system overcomes the risk of eliminating hidden features in the raw data and helps leverage the deep neural network’s power in uncovering unknown features. To improve the classification accuracy further, a median filter is used to eliminate the false detections along a prediction interval of emotions. This method yields a mean cross-subject accuracy of 86.56% and 78.34% on the Shanghai Jiao Tong University Emotion EEG Dataset (SEED) for two and three emotion classes, respectively. It also yields a mean cross-subject accuracy of 72.81% on the Database for Emotion Analysis using Physiological Signals (DEAP) and 81.8% on the Loughborough University Multimodal Emotion Dataset (LUMED) for two emotion classes. Furthermore, the recognition model that has been trained using the SEED dataset was tested with the DEAP dataset, which yields a mean prediction accuracy of 58.1% across all subjects and emotion classes. Results show that in terms of classification accuracy, the proposed approach is superior to, or on par with, the reference subject-independent EEG emotion recognition studies identified in literature and has limited complexity due to the elimination of the need for feature extraction.<br

EmoEEG - recognising people's emotions using electroencephalography

Tese de mestrado integrado em Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas), Universidade de Lisboa, Faculdade de Ciências, 2020As emoções desempenham um papel fulcral na vida humana, estando envolvidas numa extensa variedade de processos cognitivos, tais como tomada de decisão, perceção, interações sociais e inteligência. As interfaces cérebro-máquina (ICM) são sistemas que convertem os padrões de atividade cerebral de um utilizador em mensagens ou comandos para uma determinada aplicação. Os usos mais comuns desta tecnologia permitem que pessoas com deficiência motora controlem braços mecânicos, cadeiras de rodas ou escrevam. Contudo, também é possível utilizar tecnologias ICM para gerar output sem qualquer controle voluntário. A identificação de estados emocionais é um exemplo desse tipo de feedback. Por sua vez, esta tecnologia pode ter aplicações clínicas tais como a identificação e monitorização de patologias psicológicas, ou aplicações multimédia que facilitem o acesso a músicas ou filmes de acordo com o seu conteúdo afetivo. O interesse crescente em estabelecer interações emocionais entre máquinas e pessoas, levou à necessidade de encontrar métodos fidedignos de reconhecimento emocional automático. Os autorrelatos podem não ser confiáveis devido à natureza subjetiva das próprias emoções, mas também porque os participantes podem responder de acordo com o que acreditam que os outros responderiam. A fala emocional é uma maneira eficaz de deduzir o estado emocional de uma pessoa, pois muitas características da fala são independentes da semântica ou da cultura. No entanto, a precisão ainda é insuficiente quando comparada com outros métodos, como a análise de expressões faciais ou sinais fisiológicos. Embora o primeiro já tenha sido usado para identificar emoções com sucesso, ele apresenta desvantagens, tais como o fato de muitas expressões faciais serem "forçadas" e o fato de que as leituras só são possíveis quando o rosto do sujeito está dentro de um ângulo muito específico em relação à câmara. Por estes motivos, a recolha de sinais fisiológicos tem sido o método preferencial para o reconhecimento de emoções. O uso do EEG (eletroencefalograma) permite-nos monitorizar as emoções sentidas sob a forma de impulsos elétricos provenientes do cérebro, permitindo assim obter uma ICM para o reconhecimento afetivo. O principal objetivo deste trabalho foi estudar a combinação de diferentes elementos para identificar estados afetivos, estimando valores de valência e ativação usando sinais de EEG. A análise realizada consistiu na criação de vários modelos de regressão para avaliar como diferentes elementos afetam a precisão na estimativa de valência e ativação. Os referidos elementos foram os métodos de aprendizagem automática, o género do indivíduo, o conceito de assimetria cerebral, os canais de elétrodos utilizados, os algoritmos de extração de características e as bandas de frequências analisadas. Com esta análise foi possível criarmos o melhor modelo possível, com a combinação de elementos que maximiza a sua precisão. Para alcançar os nossos objetivos, recorremos a duas bases de dados (AMIGOS e DEAP) contendo sinais de EEG obtidos durante experiências de desencadeamento emocional, juntamente com a autoavaliação realizada pelos respetivos participantes. Nestas experiências, os participantes visionaram excertos de vídeos de conteúdo afetivo, de modo a despoletar emoções sobre eles, e depois classificaram-nas atribuindo o nível de valência e ativação experienciado. Os sinais EEG obtidos foram divididos em epochs de 4s e de seguida procedeu-se à extração de características através de diferentes algoritmos: o primeiro, segundo e terceiro parâmetros de Hjorth; entropia espectral; energia e entropia de wavelets; energia e entropia de FMI (funções de modos empíricos) obtidas através da transformada de Hilbert-Huang. Estes métodos de processamento de sinal foram escolhidos por já terem gerado resultados bons noutros trabalhos relacionados. Todos estes métodos foram aplicados aos sinais EEG dentro das bandas de frequência alfa, beta e gama, que também produziram bons resultados de acordo com trabalhos já efetuados. Após a extração de características dos sinais EEG, procedeu-se à criação de diversos modelos de estimação da valência e ativação usando as autoavaliações dos participantes como “verdade fundamental”. O primeiro conjunto de modelos criados serviu para aferir quais os melhores métodos de aprendizagem automática a utilizar para os testes vindouros. Após escolher os dois melhores, tentámos verificar as diferenças no processamento emocional entre os sexos, realizando a estimativa em homens e mulheres separadamente. O conjunto de modelos criados a seguir visou testar o conceito da assimetria cerebral, que afirma que a valência emocional está relacionada com diferenças na atividade fisiológica entre os dois hemisférios cerebrais. Para este teste específico, foram consideradas a assimetria diferencial e racional segundo pares de elétrodos homólogos. Depois disso, foram criados modelos de estimação de valência e ativação considerando cada um dos elétrodos individualmente. Ou seja, os modelos seriam gerados com todos os métodos de extração de características, mas com os dados obtidos de um elétrodo apenas. Depois foram criados modelos que visassem comparar cada um dos algoritmos de extração de características utilizados. Os modelos gerados nesta fase incluíram os dados obtidos de todos os elétrodos, já que anteriormente se verificou que não haviam elétrodos significativamente melhores que outros. Por fim, procedeu-se à criação dos modelos com a melhor combinação de elementos possível, otimizaram-se os parâmetros dos mesmos, e procurámos também aferir a sua validação. Realizámos também um processo de classificação emocional associando cada par estimado de valores de valência e ativação ao quadrante correspondente no modelo circumplexo de afeto. Este último passo foi necessário para conseguirmos comparar o nosso trabalho com as soluções existentes, pois a grande maioria delas apenas identificam o quadrante emocional, não estimando valores para a valência e ativação. Em suma, os melhores métodos de aprendizagem automática foram RF (random forest) e KNN (k-nearest neighbours), embora a combinação dos melhores métodos de extração de características fosse diferente para os dois. KNN apresentava melhor precisão considerando todos os métodos de extração menos a entropia espectral, enquanto que RF foi mais preciso considerando apenas o primeiro parâmetro de Hjorth e a energia de wavelets. Os valores dos coeficientes de Pearson obtidos para os melhores modelos otimizados ficaram compreendidos entre 0,8 e 0,9 (sendo 1 o valor máximo). Não foram registados melhoramentos nos resultados considerando cada género individualmente, pelo que os modelos finais foram criados usando os dados de todos os participantes. É possível que a diminuição da precisão dos modelos criados para cada género seja resultado da menor quantidade de dados envolvidos no processo de treino. O conceito de assimetria cerebral só foi útil nos modelos criados usando a base de dados DEAP, especialmente para a estimação de valência usando as características extraídas segundo a banda alfa. Em geral, as nossas abordagens mostraram-se a par ou mesmo superiores a outros trabalhos, obtendo-se valores de acurácia de 86.5% para o melhor modelo de classificação gerado com a base de dados AMIGOS e 86.6% usando a base de dados DEAP.Emotion recognition is a field within affective computing that is gaining increasing relevance and strives to predict an emotional state using physiological signals. Understanding how these biological factors are expressed according to one’s emotions can enhance the humancomputer interaction (HCI). This knowledge, can then be used for clinical applications such as the identification and monitoring of psychiatric disorders. It can also be used to provide better access to multimedia content, by assigning affective tags to videos or music. The goal of this work was to create several models for estimating values of valence and arousal, using features extracted from EEG signals. The different models created were meant to compare how various elements affected the accuracy of the model created. These elements were the machine learning techniques, the gender of the individual, the brain asymmetry concept, the electrode channels, the feature extraction methods and the frequency of the brain waves analysed. The final models contained the best combination of these elements and achieved PCC values over 0.80. As a way to compare our work with previous approaches, we also implemented a classification procedure to find the correspondent quadrant in the valence and arousal space according to the circumplex model of affect. The best accuracies achieved were over 86%, which was on par or even superior to some of the works already done

EEG-induced Fear-type Emotion Classification Through Wavelet Packet Decomposition, Wavelet Entropy, and SVM

Among the most significant characteristics of human beings is their ability to feel emotions. In recent years, human-machine interface (HM) research has centered on ways to empower the classification of emotions. Mainly, human-computer interaction (HCI) research concentrates on methods that enable computers to reveal the emotional states of humans. In this research, an emotion detection system based on visual IAPPS pictures through EMOTIV EPOC EEG signals was proposed. We employed EEG signals acquired from channels (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) for individuals in a visual induced setting (IAPS fear and neutral aroused pictures). The wavelet packet transform (WPT) combined with the wavelet entropy algorithm was applied to the EEG signals. The entropy values were extracted for every two classes. Finally, these feature matrices were fed into the SVM (Support Vector Machine) type classifier to generate the classification model. Also, we evaluated the proposed algorithm as area under the ROC (Receiver Operating Characteristic) curve, or simply AUC (Area under the curve) was utilized as an alternative single-number measure. Overall classification accuracy was obtained at 91.0%. For classification, the AUC value given for SVM was 0.97. The calculations confirmed that the proposed approaches are successful for the detection of the emotion of fear stimuli via EMOTIV EPOC EEG signals and that the accuracy of the classification is acceptable

Hierarchical classification of event-related potentials for the recognition of gender differences in the attention task

Research on the functioning of human cognition has been a crucial problem studied for years. Electroencephalography (EEG) classification methods may serve as a precious tool for understanding the temporal dynamics of human brain activity, and the purpose of such an approach is to increase the statistical power of the differences between conditions that are too weak to be detected using standard EEG methods. Following that line of research, in this paper, we focus on recognizing gender differences in the functioning of the human brain in the attention task. For that purpose, we gathered, analyzed, and finally classified event-related potentials (ERPs). We propose a hierarchical approach, in which the electrophysiological signal preprocessing is combined with the classification method, enriched with a segmentation step, which creates a full line of electrophysiological signal classification during an attention task. This approach allowed us to detect differences between men and women in the P3 waveform, an ERP component related to attention, which were not observed using standard ERP analysis. The results provide evidence for the high effectiveness of the proposed method, which outperformed a traditional statistical analysis approach. This is a step towards understanding neuronal differences between men’s and women’s brains during cognition, aiming to reduce the misdiagnosis and adverse side effects in underrepresented women groups in health and biomedical research