Decoding subjective emotional arousal from EEG during an immersive Virtual Reality experience

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation:37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a Long Short-Term Memory recurrent neural network.We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.Competing Interest StatementThe authors have declared no competing interest

Decoding subjective emotional arousal from eeg during an immersive virtual reality experience

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation: 37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a Long Short-Term Memory recurrent neural network. We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience

How immersive virtual reality methods may meet the criteria of the National Academy of Neuropsychology and American Academy of Clinical Neuropsychology:A software review of the Virtual Reality Everyday Assessment Lab (VR-EAL)

International audienceClinical tools involving immersive virtual reality (VR) may bring several advantages to cognitive neuroscience and neuropsychology. However, there are some technical and methodological pitfalls. The American Academy of Clinical Neuropsychology (AACN) and the National Academy of Neuropsychology (NAN) raised 8 key issues pertaining to Computerized Neuropsychological Assessment Devices. These issues pertain to: (1) the safety and effectivity; (2) the identity of the end-user; (3) the technical hardware and software features; (4) privacy and data security; (5) the psychometric properties; (6) examinee issues; (7) the use of reporting services; and (8) the reliability of the responses and results. The VR Everyday Assessment Lab (VR-EAL) is the first immersive VR neuropsychological battery with enhanced ecological validity for the assessment of everyday cognitive functions by offering a pleasant testing experience without inducing cybersickness. The VR-EAL meets the criteria of the NAN and AACN, addresses the methodological pitfalls, and brings advantages for neuropsychological testing. However, there are still shortcomings of the VR-EAL, which should be addressed. Future iterations should strive to improve the embodiment illusion in VR-EAL and the creation of an open access VR software library should be attempted. The discussed studies demonstrate the utility of VR methods in cognitive neuroscience and neuropsychology

Exploring Emotions and Engagement: A Multi-componential Analysis Using Films and Virtual Reality

In the digital age, where our lives are intertwined with intelligent systems and immersive experiences, understanding how emotions are shaped and influenced is more crucial than ever. Despite the attention to discrete and dimensional models, neuroscientific evidence supports that emotions are complex and multi-faceted. While the Component Process Model (CPM) acknowledges the complexity of emotions through its five interconnected components: appraisal, motivation, physiology, expression, and feeling, it has received limited attention in Affective Computing. Despite some recent advances in full CPM research, limitations exist. The relatively narrow emphasis on full CPM has resulted in a scarcity of available datasets for in-depth exploration. Most of these datasets are film-based, with only one in Virtual Reality (VR), and all have received limited computational analysis, especially in exploratory and Machine Learning aspects. Passive film-based emotion induction has merits and limitations, as it positions participants as observers. Introducing active VR stimuli can enhance emotion elicitation due to its immersive nature, but current CPM VR analyses rely on subjective reports. VR as an empathy machine is often identified in cutting-edge emotion research; however, limited attention has been given to understanding these attributes of VR, such as engagement. This thesis aims to comprehend emotions through full CPM with computational models. It starts with analysing a film-based dataset having subjective and objective measures and presents the role of physiology in emotion discrimination. Subsequently, we underscore the significance of micro-level annotations using another film-based dataset with larger continuous subjective annotations. The thesis also introduces a data-driven approach using interactive VR games and collected multimodal measures (self-reports, physiological, facial expressions, and movements) from 39 participants. The new dataset shows the role of different components in emotion differentiation when emotions are induced actively. Furthermore, the thesis presents an innovative approach to measuring engagement in VR games. We examine the simultaneous occurrence of player motivation and physiological responses to explore potential associations with body movements. Our explorations into emotions and engagement within a multi-componential framework, utilising both films and VR games, present numerous opportunities for advancing our understanding of human behaviour and interactions to foster a more empathetic world

Emotion and Stress Recognition Related Sensors and Machine Learning Technologies

This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective

KEER2022

Avanttítol: KEER2022. DiversitiesDescripció del recurs: 25 juliol 202

Accessibility of Health Data Representations for Older Adults: Challenges and Opportunities for Design

Health data of consumer off-the-shelf wearable devices is often conveyed to users through visual data representations and analyses. However, this is not always accessible to people with disabilities or older people due to low vision, cognitive impairments or literacy issues. Due to trade-offs between aesthetics predominance or information overload, real-time user feedback may not be conveyed easily from sensor devices through visual cues like graphs and texts. These difficulties may hinder critical data understanding. Additional auditory and tactile feedback can also provide immediate and accessible cues from these wearable devices, but it is necessary to understand existing data representation limitations initially. To avoid higher cognitive and visual overload, auditory and haptic cues can be designed to complement, replace or reinforce visual cues. In this paper, we outline the challenges in existing data representation and the necessary evidence to enhance the accessibility of health information from personal sensing devices used to monitor health parameters such as blood pressure, sleep, activity, heart rate and more. By creating innovative and inclusive user feedback, users will likely want to engage and interact with new devices and their own data

Recent Developments in Smart Healthcare

Medicine is undergoing a sector-wide transformation thanks to the advances in computing and networking technologies. Healthcare is changing from reactive and hospital-centered to preventive and personalized, from disease focused to well-being centered. In essence, the healthcare systems, as well as fundamental medicine research, are becoming smarter. We anticipate significant improvements in areas ranging from molecular genomics and proteomics to decision support for healthcare professionals through big data analytics, to support behavior changes through technology-enabled self-management, and social and motivational support. Furthermore, with smart technologies, healthcare delivery could also be made more efficient, higher quality, and lower cost. In this special issue, we received a total 45 submissions and accepted 19 outstanding papers that roughly span across several interesting topics on smart healthcare, including public health, health information technology (Health IT), and smart medicine

Decoding subjective emotional arousal during a naturalistic VR experience from EEG using LSTMs

Emotional arousal (EA) denotes a heightened state of activation that has both subjective and physiological aspects. The neurophysiology of subjective EA, among other mind-brain-body phenomena, can best be tested when subjects are stimulated in a natural fashion. Immersive virtual reality (VR) enables naturalistic experimental stimulation and thus promises to increase the ecological validity of research findings i.e., how well they generalize to real-life settings. In this study, 45 participants experienced virtual rollercoaster rides while their brain activity was recorded using electroencephalography (EEG). A Long Short-Term Memory (LSTM) recurrent neural network (RNN) was then trained on the alpha-frequency (8-12 Hz) component of the EEG signal (input) and the retrospectively acquired continuous reports of subjective EA (target). With the LSTM-based model, subjective EA could be predicted significantly above chance level. This demonstrates a novel EEG-based decoding approach for subjective states of experience in naturalistic research designs using VR