2,813 research outputs found
Emotional Brain-Computer Interfaces
Research in Brain-computer interface (BCI) has significantly increased during the last few years. In addition to their initial role as assisting devices for the physically challenged, BCIs are now proposed for a wider range of applications. As in any HCI application, BCIs can also benefit from adapting their operation to the emotional state of the user. BCIs have the advantage of having access to brain activity which can provide signicant insight into the user's emotional state. This information can be utilized in two manners. 1) Knowledge of the inuence of the emotional state on brain activity patterns can allow the BCI to adapt its recognition algorithms, so that the intention of the user is still correctly interpreted in spite of signal deviations induced by the subject's emotional state. 2) The ability to recognize emotions can be used in BCIs to provide the user with more natural ways of controlling the BCI through affective modulation. Thus, controlling a BCI by recollecting a pleasant memory can be possible and can potentially lead to higher information transfer rates.\ud
These two approaches of emotion utilization in BCI are elaborated in detail in this paper in the framework of noninvasive EEG based BCIs
Self-supervised Learning for Electroencephalogram: A Systematic Survey
Electroencephalogram (EEG) is a non-invasive technique to record
bioelectrical signals. Integrating supervised deep learning techniques with EEG
signals has recently facilitated automatic analysis across diverse EEG-based
tasks. However, the label issues of EEG signals have constrained the
development of EEG-based deep models. Obtaining EEG annotations is difficult
that requires domain experts to guide collection and labeling, and the
variability of EEG signals among different subjects causes significant label
shifts. To solve the above challenges, self-supervised learning (SSL) has been
proposed to extract representations from unlabeled samples through
well-designed pretext tasks. This paper concentrates on integrating SSL
frameworks with temporal EEG signals to achieve efficient representation and
proposes a systematic review of the SSL for EEG signals. In this paper, 1) we
introduce the concept and theory of self-supervised learning and typical SSL
frameworks. 2) We provide a comprehensive review of SSL for EEG analysis,
including taxonomy, methodology, and technique details of the existing
EEG-based SSL frameworks, and discuss the difference between these methods. 3)
We investigate the adaptation of the SSL approach to various downstream tasks,
including the task description and related benchmark datasets. 4) Finally, we
discuss the potential directions for future SSL-EEG research.Comment: 35 pages, 12 figure
Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data
There are threefold challenges in emotion recognition. First, it is difficult
to recognize human's emotional states only considering a single modality.
Second, it is expensive to manually annotate the emotional data. Third,
emotional data often suffers from missing modalities due to unforeseeable
sensor malfunction or configuration issues. In this paper, we address all these
problems under a novel multi-view deep generative framework. Specifically, we
propose to model the statistical relationships of multi-modality emotional data
using multiple modality-specific generative networks with a shared latent
space. By imposing a Gaussian mixture assumption on the posterior approximation
of the shared latent variables, our framework can learn the joint deep
representation from multiple modalities and evaluate the importance of each
modality simultaneously. To solve the labeled-data-scarcity problem, we extend
our multi-view model to semi-supervised learning scenario by casting the
semi-supervised classification problem as a specialized missing data imputation
task. To address the missing-modality problem, we further extend our
semi-supervised multi-view model to deal with incomplete data, where a missing
view is treated as a latent variable and integrated out during inference. This
way, the proposed overall framework can utilize all available (both labeled and
unlabeled, as well as both complete and incomplete) data to improve its
generalization ability. The experiments conducted on two real multi-modal
emotion datasets demonstrated the superiority of our framework.Comment: arXiv admin note: text overlap with arXiv:1704.07548, 2018 ACM
Multimedia Conference (MM'18
Prerequisites for Affective Signal Processing (ASP)
Although emotions are embraced by science, their recognition has not reached a satisfying level. Through a concise overview of affect, its signals, features, and classification methods, we provide understanding for the problems encountered. Next, we identify the prerequisites for successful Affective Signal Processing: validation (e.g., mapping of constructs on signals), triangulation, a physiology-driven approach, and contributions of the signal processing community. Using these directives, a critical analysis of a real-world case is provided. This illustrates that the prerequisites can become a valuable guide for Affective Signal Processing (ASP)
Reconnaissance de l'émotion thermique
Pour améliorer les interactions homme-ordinateur dans les domaines de la santé, de l'e-learning et des jeux vidéos, de nombreux chercheurs ont étudié la reconnaissance des émotions à partir des signaux de texte, de parole, d'expression faciale, de détection d'émotion ou d'électroencéphalographie (EEG). Parmi eux, la reconnaissance d'émotion à l'aide d'EEG a permis une précision satisfaisante. Cependant, le fait d'utiliser des dispositifs d'électroencéphalographie limite la gamme des mouvements de l'utilisateur. Une méthode non envahissante est donc nécessaire pour faciliter la détection des émotions et ses applications. C'est pourquoi nous avons proposé d'utiliser une caméra thermique pour capturer les changements de température de la peau, puis appliquer des algorithmes d'apprentissage machine pour classer les changements d'émotion en conséquence. Cette thèse contient deux études sur la détection d'émotion thermique avec la comparaison de la détection d'émotion basée sur EEG. L'un était de découvrir les profils de détection émotionnelle thermique en comparaison avec la technologie de détection d'émotion basée sur EEG; L'autre était de construire une application avec des algorithmes d'apprentissage en machine profonds pour visualiser la précision et la performance de la détection d'émotion thermique et basée sur EEG. Dans la première recherche, nous avons appliqué HMM dans la reconnaissance de l'émotion thermique, et après avoir comparé à la détection de l'émotion basée sur EEG, nous avons identifié les caractéristiques liées à l'émotion de la température de la peau en termes d'intensité et de rapidité. Dans la deuxième recherche, nous avons mis en place une application de détection d'émotion qui supporte à la fois la détection d'émotion thermique et la détection d'émotion basée sur EEG en appliquant les méthodes d'apprentissage par machine profondes - Réseau Neuronal Convolutif (CNN) et Mémoire à long court-terme (LSTM). La précision de la détection d'émotion basée sur l'image thermique a atteint 52,59% et la précision de la détection basée sur l'EEG a atteint 67,05%. Dans une autre étude, nous allons faire plus de recherches sur l'ajustement des algorithmes d'apprentissage machine pour améliorer la précision de détection d'émotion thermique.To improve computer-human interactions in the areas of healthcare, e-learning and video
games, many researchers have studied on recognizing emotions from text, speech, facial
expressions, emotion detection, or electroencephalography (EEG) signals. Among them,
emotion recognition using EEG has achieved satisfying accuracy. However, wearing
electroencephalography devices limits the range of user movement, thus a noninvasive method
is required to facilitate the emotion detection and its applications. That’s why we proposed using
thermal camera to capture the skin temperature changes and then applying machine learning
algorithms to classify emotion changes accordingly. This thesis contains two studies on thermal
emotion detection with the comparison of EEG-base emotion detection. One was to find out the
thermal emotional detection profiles comparing with EEG-based emotion detection technology;
the other was to implement an application with deep machine learning algorithms to visually
display both thermal and EEG based emotion detection accuracy and performance. In the first
research, we applied HMM in thermal emotion recognition, and after comparing with EEG-base
emotion detection, we identified skin temperature emotion-related features in terms of intensity
and rapidity. In the second research, we implemented an emotion detection application
supporting both thermal emotion detection and EEG-based emotion detection with applying the
deep machine learning methods – Convolutional Neutral Network (CNN) and LSTM (Long-
Short Term Memory). The accuracy of thermal image based emotion detection achieved 52.59%
and the accuracy of EEG based detection achieved 67.05%. In further study, we will do more
research on adjusting machine learning algorithms to improve the thermal emotion detection
precision
- …