Beyond Detection: Investing in Practical and Theoretical Applications of Emotion + Visualization

Emotion is a dynamic variable that modulates how we perceive, reason about, and interact with our environment. Recent studies have established that emotion’s influence carries to data analysis and visualization, impacting performance in ways both positive and negative. While we are still in the infancy of understanding the role emotion plays in analytical contexts, advances in physiological sensing and emotion research have raised the possibility of creating emotion-aware systems. In this position paper, we argue that it is critical to consider the potential advances that can be made even in the face of imperfect sensing, while we continue to address the practical challenges of monitoring emotion in the wild. To underscore the importance of this line of inquiry, we highlight several key challenges related to detection, adaptation, and impact of emotional states for users of data visualization systems, and motivate promising avenues for future research in these areas

Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty based on Brain State

We present Brain Automated Chorales (BACh), an adaptive brain-computer system that dynamically increases the levels of difficulty in a musical learning task based on pianists\u27 cognitive workload measured by functional near-infrared spectroscopy. As users\u27 cognitive workload fell below a certain threshold, suggesting that they had mastered the material and could handle more cognitive information, BACh automatically increased the difficulty of the learning task. We found that learners played with significantly increased accuracy and speed in the brain-based adaptive task compared to our control condition. Participant feedback indicated that they felt they learned better with BACh and they liked the timings of the level changes. The underlying premise of BACh can be applied to learning situations where a task can be broken down into increasing levels of difficulty

MYND: Unsupervised Evaluation of Novel BCI Control Strategies on Consumer Hardware

Neurophysiological studies are typically conducted in laboratories with limited ecological validity, scalability, and generalizability of findings. This is a significant challenge for the development of brain-computer interfaces (BCIs), which ultimately need to function in unsupervised settings on consumer-grade hardware. We introduce MYND: A framework that couples consumer-grade recording hardware with an easy-to-use application for the unsupervised evaluation of BCI control strategies. Subjects are guided through experiment selection, hardware fitting, recording, and data upload in order to self-administer multi-day studies that include neurophysiological recordings and questionnaires. As a use case, we evaluate two BCI control strategies ("Positive memories" and "Music imagery") in a realistic scenario by combining MYND with a four-channel electroencephalogram (EEG). Thirty subjects recorded 70.4 hours of EEG data with the system at home. The median headset fitting time was 25.9 seconds, and a median signal quality of 90.2% was retained during recordings.Neural activity in both control strategies could be decoded with an average offline accuracy of 68.5% and 64.0% across all days. The repeated unsupervised execution of the same strategy affected performance, which could be tackled by implementing feedback to let subjects switch between strategies or devise new strategies with the platform.Comment: 9 pages, 5 figures. Submitted to PNAS. Minor revisio

Speed-accuracy comparison of navigational interfaces

The goal of this research is to test the effect of different computer interfaces on the amount of time it takes a user to move a cursor from a start point to a target, using Fitts’ Law, a model that describes the performance of pointing of input devices. Participants in a study used a mouse, Xbox 360 controller, and Nintendo Wii remote to point at and select target regions. The goal is to see the effects of interface, distance to the target, and target width on movement time, information throughput, and hit rate. Additional path metrics and the speed-accuracy tradeoff will be covered

35 Designing Implicit Interfaces for Physiological Computing: Guidelines and Lessons Learned Using fNIRS

A growing body of recent work has shown the feasibility of brain and body sensors as input to interactive systems. However, the interaction techniques and design decisions for their effective use are not well defined. We present a conceptual framework for considering implicit input from the brain, along with design principles and patterns we have developed from our work. We also describe a series of controlled, offline studies that lay the foundation for our work with functional near-infrared spectroscopy (fNIRS) neuroimaging, as well as our real-time platform that serves as a testbed for exploring brain-based adaptive interaction techniques. Finally, we present case studies illustrating the principles and patterns for effective use of brain data in human-computer interaction. We focus on signals coming from the brain, but these principles apply broadly to other sensor data and in domains such as aviation, education, medicine, driving, and anything involving multitasking or varying cognitive workload

Improving Bayesian Reasoning: The Effects of Phrasing, Visualization, and Spatial Ability

Decades of research have repeatedly shown that people perform poorly at estimating and understanding conditional probabilities that are inherent in Bayesian reasoning problems. Yet in the medical domain, both physicians and patients make daily, life-critical judgments based on conditional probability. Although there have been a number of attempts to develop more effective ways to facilitate Bayesian reasoning, reports of these findings tend to be inconsistent and sometimes even contradictory. For instance, the reported accuracies for individuals being able to correctly estimate conditional probability range from 6% to 62%. In this work, we show that problem representation can significantly affect accuracies. By controlling the amount of information presented to the user, we demonstrate how text and visualization designs can increase overall accuracies to as high as 77%. Additionally, we found that for users with high spatial ability, our designs can further improve their accuracies to as high as 100%. By and large, our findings provide explanations for the inconsistent reports on accuracy in Bayesian reasoning tasks and show a significant improvement over existing methods. We believe that these findings can have immediate impact on risk communication in health-related fields