VIF: Virtual Interactive Fiction (with a twist)

Nowadays computer science can create digital worlds that deeply immerse users; it can also process in real time brain activity to infer their inner states. What marvels can we achieve with such technologies? Go back to displaying text. And unfold a story that follows and molds users as never before.Comment: Pervasive Play - CHI '16 Workshop, May 2016, San Jose, United State

TOBE: Tangible Out-of-Body Experience

We propose a toolkit for creating Tangible Out-of-Body Experiences: exposing the inner states of users using physiological signals such as heart rate or brain activity. Tobe can take the form of a tangible avatar displaying live physiological readings to reflect on ourselves and others. Such a toolkit could be used by researchers and designers to create a multitude of potential tangible applications, including (but not limited to) educational tools about Science Technologies Engineering and Mathematics (STEM) and cognitive science, medical applications or entertainment and social experiences with one or several users or Tobes involved. Through a co-design approach, we investigated how everyday people picture their physiology and we validated the acceptability of Tobe in a scientific museum. We also give a practical example where two users relax together, with insights on how Tobe helped them to synchronize their signals and share a moment

Recent advances in EEG-based neuroergonomics for Human-Computer Interaction

International audienceHuman-Computer Interfaces (HCI) are increasingly ubiquitous in multiple applications including industrial design, education, art or entertainment. As such, HCI could be used by very different users, with very different skills and needs. This thus requires user-centered design approaches and appropriate evaluation methods to maximize User eXperience (UX). Existing evaluation methods include behavioral studies, testbeds, questionnaires and inquiries, among others. While useful, such methods suffer from several limitations as they can be either ambiguous, lack real-time recordings, or disrupt the interaction. Neuroergonomics can be an adequate tool to complement traditional evaluation methods. Notably, Electroencephalography (EEG)-based evaluation of UX has the potential to address the limitations above, by providing objective, real-time and non-disruptive metrics of the ergonomics quality of a given HCI (Frey 2014). In this abstract, we present an overview of our recent works in that direction. In particular, we show how we can process EEG signals in order to derive metrics characterizing 1) how the user perceive the HCI display (HCI output) and 2) how the user interacts with the HCI (HCI input)

Framework for Electroencephalography-based Evaluation of User Experience

Measuring brain activity with electroencephalography (EEG) is mature enough to assess mental states. Combined with existing methods, such tool can be used to strengthen the understanding of user experience. We contribute a set of methods to estimate continuously the user's mental workload, attention and recognition of interaction errors during different interaction tasks. We validate these measures on a controlled virtual environment and show how they can be used to compare different interaction techniques or devices, by comparing here a keyboard and a touch-based interface. Thanks to such a framework, EEG becomes a promising method to improve the overall usability of complex computer systems.Comment: in ACM. CHI '16 - SIGCHI Conference on Human Factors in Computing System, May 2016, San Jose, United State

Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals

Designing 3D User Interfaces (UI) requires adequate evaluation tools to ensure good usability and user experience. While many evaluation tools are already available and widely used, existing approaches generally cannot provide continuous and objective measures of usa-bility qualities during interaction without interrupting the user. In this paper, we propose to use brain (with ElectroEncephaloGraphy) and physiological (ElectroCardioGraphy, Galvanic Skin Response) signals to continuously assess the mental effort made by the user to perform 3D object manipulation tasks. We first show how this mental effort (a.k.a., mental workload) can be estimated from such signals, and then measure it on 8 participants during an actual 3D object manipulation task with an input device known as the CubTile. Our results suggest that monitoring workload enables us to continuously assess the 3DUI and/or interaction technique ease-of-use. Overall, this suggests that this new measure could become a useful addition to the repertoire of available evaluation tools, enabling a finer grain assessment of the ergonomic qualities of a given 3D user interface.Comment: Published in INTERACT, Sep 2015, Bamberg, German

Estimating Visual Comfort in Stereoscopic Displays Using Electroencephalography: A Proof-of-Concept

International audienceWith stereoscopic displays, a depth sensation that is too strong could impede visual comfort and result in fatigue or pain. Electroencephalography (EEG) is a technology which records brain activity. We used it to develop a novel brain-computer interface that monitors users' states in order to reduce visual strain. We present the first proof-of-concept system that discriminates comfortable conditions from uncomfortable ones during stereoscopic vision using EEG. It reacts within 1s to depth variations, achieving 63% accuracy on average and 74% when 7 consecutive variations are measured. This study could lead to adaptive systems that automatically suit stereoscopic displays to users and viewing conditions

Measuring Cognitive Conflict in Virtual Reality with Feedback-Related Negativity

As virtual reality (VR) emerges as a mainstream platform, designers have started to experiment new interaction techniques to enhance the user experience. This is a challenging task because designers not only strive to provide designs with good performance but also carefully ensure not to disrupt users' immersive experience. There is a dire need for a new evaluation tool that extends beyond traditional quantitative measurements to assist designers in the design process. We propose an EEG-based experiment framework that evaluates interaction techniques in VR by measuring intentionally elicited cognitive conflict. Through the analysis of the feedback-related negativity (FRN) as well as other quantitative measurements, this framework allows designers to evaluate the effect of the variables of interest. We studied the framework by applying it to the fundamental task of 3D object selection using direct 3D input, i.e. tracked hand in VR. The cognitive conflict is intentionally elicited by manipulating the selection radius of the target object. Our first behavior experiment validated the framework in line with the findings of conflict-induced behavior adjustments like those reported in other classical psychology experiment paradigms. Our second EEG-based experiment examines the effect of the appearance of virtual hands. We found that the amplitude of FRN correlates with the level of realism of the virtual hands, which concurs with the Uncanny Valley theory

A mutual information based adaptive windowing of informative EEG for emotion recognition

Emotion recognition using brain wave signals involves using high dimensional electroencephalogram (EEG) data. In this paper, a window selection method based on mutual information is introduced to select an appropriate signal window to reduce the length of the signals. The motivation of the windowing method comes from EEG emotion recognition being computationally costly and the data having low signal-to-noise ratio. The aim of the windowing method is to find a reduced signal where the emotions are strongest. In this paper, it is suggested, that using only the signal section which best describes emotions improves the classification of emotions. This is achieved by iteratively comparing different-length EEG signals at different time locations using the mutual information between the reduced signal and emotion labels as criterion. The reduced signal with the highest mutual information is used for extracting the features for emotion classification. In addition, a viable framework for emotion recognition is introduced. Experimental results on publicly available datasets, DEAP and MAHNOB-HCI, show significant improvement in emotion recognition accuracy

Four Channel Multivariate Coherence Training: Development and Evidence in Support of a New Form of Neurofeedback

As the field of neurofeedback and neuromodulation grows, trends toward using neurofeedback to treat problems of brain dysfunction have emerged. While the use of connectivity based fMRI guided neurofeedback has shown itself to be efficacious, the expense related to the treatment calls for a more practical solution. The use of QEEG guided neurofeedback in the treatment has shown promise as an emerging treatment. To date, EEG based neurofeedback approaches have used technology with limited sophistication. We designed a new form of neurofeedback that uses four channels of EEG with a multivariate calculation of coherence metrics. Following a mathematical presentation of this model, we present findings of a multi-site study with clinical subjects with various diagnoses. We compared this form of multivariate coherence neurofeedback to the more standard two channel coherence training. Findings showed that there was a significant difference between the groups with four channel multivariate coherence neurofeedback leading to greater changes in EEG metrics. Compared to two channel coherence training, four channel multivariate coherence neurofeedback led to a greater than 50% change in power and 400% in coherence values per session. The significance of these findings is discussed in relation to complex calculations of effective connectivity and how this might lead to even greater enhancements in neurofeedback efficacy