A dataset of continuous affect annotations and physiological signals for emotion analysis

From a computational viewpoint, emotions continue to be intriguingly hard to understand. In research, direct, real-time inspection in realistic settings is not possible. Discrete, indirect, post-hoc recordings are therefore the norm. As a result, proper emotion assessment remains a problematic issue. The Continuously Annotated Signals of Emotion (CASE) dataset provides a solution as it focusses on real-time continuous annotation of emotions, as experienced by the participants, while watching various videos. For this purpose, a novel, intuitive joystick-based annotation interface was developed, that allowed for simultaneous reporting of valence and arousal, that are instead often annotated independently. In parallel, eight high quality, synchronized physiological recordings (1000 Hz, 16-bit ADC) were made of ECG, BVP, EMG (3x), GSR (or EDA), respiration and skin temperature. The dataset consists of the physiological and annotation data from 30 participants, 15 male and 15 female, who watched several validated video-stimuli. The validity of the emotion induction, as exemplified by the annotation and physiological data, is also presented.Comment: Dataset available at: https://rmc.dlr.de/download/CASE_dataset/CASE_dataset.zi

Discrete classification technique applied to TV advertisements liking recognition system based on low‑cost EEG headsets

Background: In this paper a new approach is applied to the area of marketing research. The aim of this paper is to recognize how brain activity responds during the visualization of short video advertisements using discrete classification techniques. By means of low cost electroencephalography devices (EEG), the activation level of some brain regions have been studied while the ads are shown to users. We may wonder about how useful is the use of neuroscience knowledge in marketing, or what could provide neuroscience to marketing sector, or why this approach can improve the accuracy and the final user acceptance compared to other works. Methods: By using discrete techniques over EEG frequency bands of a generated dataset, C4.5, ANN and the new recognition system based on Ameva, a discretization algorithm, is applied to obtain the score given by subjects to each TV ad. Results: The proposed technique allows to reach more than 75 % of accuracy, which is an excellent result taking into account the typology of EEG sensors used in this work. Furthermore, the time consumption of the algorithm proposed is reduced up to 30 % compared to other techniques presented in this paper. Conclusions: This bring about a battery lifetime improvement on the devices where the algorithm is running, extending the experience in the ubiquitous context where the new approach has been tested.Ministerio de Economía y Competitividad HERMES TIN2013-46801-C4-1-rJunta de Andalucia Simon TIC-805

The use of EEG in assessing the emotional state of a person

Diplomová práce se zabývá možnostmi zpracování EEG a klasifikací emocí do tříd v rámci emočního dvoudimenzionálního prostoru. První část tvoří literární rešerše na téma využití záznamu EEG pro zachycení emoční odezvy člověka na stimulační podněty zvukového, vizuálního a audiovizuálního (multimediálního) charakteru. Je zde diskutováno hledisko emoce z pohledu fyziologie a psychologie. Následně jsou uvedený technické poznatky ze záznamu emoce různými modalitami a jejich zpracování, analýzy a klasifikace v rámci dvoudimenzionálního prostoru Valence - Arousal. Na základě teoretických poznatků byl navržený komplexní postup měření a vyhodnocení emocí za použití EEG. Za tímto účelem byly sestaveny dva měřící protokoly s audiovizuální stimulací. Dále je zde uveden postup předzpracování a analýzy dat, výběr příznaků a klasifikace za použití moderních i zavedených metod. Celý navržený postup byl následně realizován a otestován v rámci dvou měření za využití moderního EEG přístroje EGI GES 400MR v laboratorních podmínkách a komerčního, cenově dostupného přístroje Emotiv EPOC v podmínkách mimo laboratorní prostředí. Signály byly zpracovány a emoce klasifikovány na základě vybraných popisných elementů. Výsledkem práce je hodnocení úspěšnosti klasifikace emocí v různých konfiguracích pro výběr popisných elementů a klasifikačních metod a jejich parametrů.This thesis is focused on EEG processing and emotion classification within two-dimensional emotion space. First part consists of theoretical research about emotional responses of human subjects on sound, image and video stimuli. Emotions are examined from aspect of physiology and psychology. Furthermore technical overview of measurement, analysis and emotion classification within two-dimensional emotional space is discussed. Based on gathered knowledge measurement setup with audiovisual stimuli was designed and measured with two independent instruments – EGI GES400MR in laboratory conditions and Emotiv EPOC device in non-laboratory conditions. Signals were processed and emotions were classified based on chosen features. Performance of classifiers in multiple feature selection setups was evaluated.

Une forêt de réalité virtuelle pour la réduction du stress physiologique chez les personnes sans troubles neurodégénératifs

Ce travail de recherche pour la réduction du stress physiologique a été effectué initialement auprès de personnes n'ayant pas de troubles neurodégénératifs en vue de soutenir le projet visant à résorber l'anxiété des personnes atteintes de la maladie d'Alzheimer, qui se trouvent à un stade avancé, qui ont perdu leur motricité et qui ne peuvent donc plus se promener dans la nature, ou en forêt. L'exposition à la nature permet non seulement de se sentir mieux émotionnellement, mais elle contribue également au bien-être physique, en réduisant la tension artérielle, la fréquence cardiaque, la tension musculaire et la production d'hormones de stress. L'exposition à la nature à l'aide de technologies de réalité virtuelle pourrait donc apporter des avantages en termes de bien-être émotionnel aux personnes qui ne peuvent pas accéder à l'extérieur. L'article 1 présente une revue de la littérature, qui montre que bien qu'il existe un corpus comparant les effets entre la nature réelle et la nature virtuelle, il n'y a pas beaucoup d'études qui ont comparé l'effet relatif entre la nature reproduite avec différentes techniques de développement en réalité virtuelle. Plus précisément, si les différences en termes d'exploration active ou passive, la richesse de la scène ou son dynamisme, pourraient affecter le potentiel thérapeutique de ces environnements simulés, n'apparaît pas encore clairement. L'article 2 présente une étude auprès des adultes en bonne santé a été réalisée pour tester les effets de 10 min d'exposition à la nature virtuelle dans une forêt. L'objectif était d'évaluer si l'exploration d'une forêt virtuelle pouvait induire un état de connexion avec la nature, améliorer le confort des utilisateurs et réduire leur stress. Il a été émis l'hypothèse que l'exploration d'un environnement forestier virtuel à travers un visiocasque aurait un effet thérapeutique sur les manifestations de stress physiologique, dans un premier temps, chez l'adulte sans troubles neurodégénératifs.This research work for the reduction of physiological stress was initially carried out with people without neurodegenerative disorders in order to support the project aimed at reducing the anxiety of people with Alzheimer's disease, who are located at an advanced stage, who have lost their motor skills and who can no longer walk in nature, or in the forest. Exposure to nature not only makes you feel better emotionally, but it also contributes to physical well-being, by reducing blood pressure, heart rate, muscle tension and the production of stress hormones. Exposure to nature using virtual reality technologies could therefore bring emotional well-being benefits to people who cannot access the outdoors. Article 1 presents a review of the literature, which shows that although there is a body of work comparing the effects between real and virtual nature, there are not many studies that have compared the relative effect between nature reproduced with different development techniques in virtual reality. More precisely, whether the differences in terms of active or passive exploration, the richness of the scene or its dynamism, could affect the therapeutic potential of these simulated environments, is not yet clear. Article 2 presents a study in healthy adults was performed to test the effects of 10 min of exposure to virtual nature in a forest. The objective was to assess whether exploring a virtual forest could induce a state of connection with nature, improve user comfort and reduce stress. It was hypothesized that the exploration of a virtual forest environment through a head-mounted display would have a therapeutic effect on the manifestations of physiological stress, initially in adults without neurodegenerative disorders

Recent Developments in Smart Healthcare

Medicine is undergoing a sector-wide transformation thanks to the advances in computing and networking technologies. Healthcare is changing from reactive and hospital-centered to preventive and personalized, from disease focused to well-being centered. In essence, the healthcare systems, as well as fundamental medicine research, are becoming smarter. We anticipate significant improvements in areas ranging from molecular genomics and proteomics to decision support for healthcare professionals through big data analytics, to support behavior changes through technology-enabled self-management, and social and motivational support. Furthermore, with smart technologies, healthcare delivery could also be made more efficient, higher quality, and lower cost. In this special issue, we received a total 45 submissions and accepted 19 outstanding papers that roughly span across several interesting topics on smart healthcare, including public health, health information technology (Health IT), and smart medicine

Beyond the Valley of the Genitals: Using eye-tracking to analyze sexual arousal and desire in women and men

ABSTRACT Beyond the Valley of the Genitals: Using eye tracking to analyze sexual arousal and desire in women and men Lucia Farisello, PhD. Concordia University, 2017 Traditional models of sexual arousal and desire in humans have focused on either physiological measures (Kaplan, 1974; Masters & Johnson), or on self-report (ex.Derogatis & Melisaratos, 1979). However, some have also proposed that cognitive processes play a key role in connecting both arousal and desire. It is unknown if a stimulus is deemed sexually salient at a low processing level (i.e., at the level of sensation), or if more higher-level cognitive processing (i.e., perception, recognition) is required to generate a sexual response to the stimuli, or a combination of both. In addition, are there gender differences to this perception of sexual stimuli. Therefore, the aim of this thesis was to use cognitive measures that target low level and high level processing tasks to examine whether eye-tracking methodology could reveal patterns that constitute a more objective assessment of sexual arousal and desire. The results indicate that low level tasks, which used timed response tasks with visual sexual stimuli, created a delay effect predominantly in men, and to a lesser extent in women. When women were subjectively aroused (as assessed using the SADI; Toledano & Pfaus, 2006) the observed level of cognitive delay increased (i.e., latency to respond to stimuli). However, low level processing does not produce a sexually induced cognitive delay effect in women. This finding suggests a reflexive response in women that is not sufficient to impose a cognitive delay. In contrast, using high level processing tasks that exposed participants to viewing sexual stimuli for longer durations (specifically, viewing nude versus clothed images, viewing high versus low arousal images, and viewing an erotic movie) lead to gender distinct patterns of eye movements concordant with reported levels of subjective arousal. Interestingly, women shown specific eye movement patterns when viewing images that they rate as highly arousing (in comparison to low arousing images). Together, these data suggest that women may require longer exposure to sexual stimuli in order to engage and sustain desire, which can then produce concordant results with self-reported arousal. Keywords: Cognition, Eye tracking, Visual Sexual Stimuli, Sexually induced cognitive delay

Affective virtual environments: a psychophysiological HCI system concept

The recent “resurrection” of interest in Virtual Reality has stimulated interest in the quest for true “immersion” in computer-generated worlds. True immersion may only ever be achieved through advanced BCI systems, but, until that day arrives, it is important to understand how it may be possible to measure human engagement and emotions within virtual worlds using psychophysiological techniques. This study aims to design an affective computing system, capable of responding to human emotions, within virtual environments. Based on the development of a Valence/Arousal model, a controllable affective VR, capable of evoking multiple emotions, has been constructed. Multiple variations of the VR have been evaluated subjectively using over 68 participants. More objective, physiologically-based experiments have been executed, in which the EEG, GSR and heart rates of 45 participants have been recorded during exposure to the most powerful affective environments, identified in the earlier study. Multiple affective recognition systems have been trained and crossvalidated against 30 participants and evaluated using the other 15 individuals. The results suggested that the trained classifiers perform highly accurately in the training database, but achieve random classification accuracies in the new dataset. It was highlighted that the extreme performance attenuation is due to the high individual differences in participants’ physiological responses, in emotional experiences

Affective state recognition in Virtual Reality from electromyography and photoplethysmography using head-mounted wearable sensors.

The three core components of Affective Computing (AC) are emotion expression recognition, emotion processing, and emotional feedback. Affective states are typically characterized in a two-dimensional space consisting of arousal, i.e., the intensity of the emotion felt; and valence, i.e., the degree to which the current emotion is pleasant or unpleasant. These fundamental properties of emotion can not only be measured using subjective ratings from users, but also with the help of physiological and behavioural measures, which potentially provide an objective evaluation across users. Multiple combinations of measures are utilised in AC for a range of applications, including education, healthcare, marketing, and entertainment. As the uses of immersive Virtual Reality (VR) technologies are growing, there is a rapidly increasing need for robust affect recognition in VR settings. However, the integration of affect detection methodologies with VR remains an unmet challenge due to constraints posed by the current VR technologies, such as Head Mounted Displays. This EngD project is designed to overcome some of the challenges by effectively integrating valence and arousal recognition methods in VR technologies and by testing their reliability in seated and room-scale full immersive VR conditions. The aim of this EngD research project is to identify how affective states are elicited in VR and how they can be efficiently measured, without constraining the movement and decreasing the sense of presence in the virtual world. Through a three-years long collaboration with Emteq labs Ltd, a wearable technology company, we assisted in the development of a novel multimodal affect detection system, specifically tailored towards the requirements of VR. This thesis will describe the architecture of the system, the research studies that enabled this development, and the future challenges. The studies conducted, validated the reliability of our proposed system, including the VR stimuli design, data measures and processing pipeline. This work could inform future studies in the field of AC in VR and assist in the development of novel applications and healthcare interventions

Continuous Emotion Detection in Response to Music Videos

Viewers' preference for multimedia selection depends highly on their emotional experience. In this paper, we present an emotion detection method for music videos using central and peripheral nervous system physiological signals as well as multimedia content analysis. A set of 40 music clips eliciting a broad range of emotions were first selected. After extracting the one minute long emotional highlight of each video, they were shown to 32 participants while their physiological responses were recorded. Participants self-reported their felt emotions after watching each clip by means of arousal, valence, dominance, and liking ratings. The physiological signals included electroencephalogram, galvanic skin response, respiration pattern, skin temperature, electromyograms and blood volume pulse using plethysmograph. Emotional features were extracted from the signals and the multimedia content. The emotional features were used to train a linear ridge regressor to detect emotions for each participant using a leave-one-out cross-validation strategy. The performance of the personalized emotion detection is shown to be significantly superior to a random regressor