Examining the reliability of using fNIRS in realistic HCI settings for spatial and verbal tasks

Recent efforts have shown that functional near-infrared spectroscopy (fNIRS) has potential value for brain sensing in HCI user studies. Research has shown that, although large head movement significantly affects fNIRS data, typical keyboard use, mouse movement, and non-task-related verbalisations do not affect measurements during Verbal tasks. This work aims to examine the Reliability of fNIRS, by 1) confirming these prior findings, and 2) significantly extending our understanding of how artefacts affect recordings during Spatial tasks, since much of user interfaces and interaction is inherently spatial. Our results show that artefacts have a significantly different impact during Verbal and Spatial tasks. We contribute clearer insights into using fNIRS as a tool within HCI user studies

Examining the reliability of using fNIRS in realistic HCI settings for spatial and verbal tasks

Recent efforts have shown that functional near-infrared spectroscopy (fNIRS) has potential value for brain sensing in HCI user studies. Research has shown that, although large head movement significantly affects fNIRS data, typical keyboard use, mouse movement, and non-task-related verbalisations do not affect measurements during Verbal tasks. This work aims to examine the Reliability of fNIRS, by 1) confirming these prior findings, and 2) significantly extending our understanding of how artefacts affect recordings during Spatial tasks, since much of user interfaces and interaction is inherently spatial. Our results show that artefacts have a significantly different impact during Verbal and Spatial tasks. We contribute clearer insights into using fNIRS as a tool within HCI user studies

Brain activity and mental workload associated with artistic practice

We present the first stage of our on-going artist-driven BCI collaboration, where we equipped an artist with the brain scanning technique functional Near Infrared Spectroscopy (fNIRS) in order to record mental workload levels during her creative practice. The artists are interested in exposing the hidden cognitive processes involved in their creative practice, in order to reuse or integrate the data into their performances. The researchers are interested in collecting unstructured ‘in the wild’ fNIRS data, and to see how the artists interpret the data retrospectively. We highlight some interesting early examples from the data and describe our on-going plans. We will have completed a second data collection before the workshop

Real-time physiological measure and feedback of workload

Understanding and identifying individuals’ capabilities and limitations has always been a challenge within work contexts, but its importance cannot be underestimated. Humans have a limited mental capacity [142], which means that they can only perform a nite set of tasks at any given period of time. Identifying these limitations is a key factor in the reduction and prevention of what is referred to as Mental Workload Overload. These measures are used in research and industry to evaluate the interaction of users with new systems and tasks. Current techniques involve asking users to subjectively assess and self report their levels of workloads using techniques and questionnaires such as NASA-TLX and Instantaneous Self-Assessment (ISA). The subjective measures become highly important when it comes to evaluating more complex systems and tasks, where performance based measures become highly difficult to measure. Even though they are critical for evaluation of these systems, there are certain limitations that cannot be overlooked when using them. Firstly, subjective measures rely on the participants’ ability to judge and report the state throughout the task. This requires not only extra effort from the operator, but also skill and potential training. Secondly, subjective measures, if used in real-time have the potential to interrupt and negatively affect performance; if used post-task, they rely on the operators’ ability to recall what happened during certain moments in the past. Direct physiological measures offer an opportunity to capture workload whilst overcoming these limitations. However, new research is needed to understand how physiological data can be interpreted within the context of theories of mental workload. The research presented in this thesis explores the use of one particular physiological approach, functional Near Infrared Spectroscopy (fNIRS), to assess workload in controlled laboratory settings, to overcome the limitations and complement the use of subjective measures; a measure based on participants’ brain and physiological responses to task demand, that is independent of the task and/or the operator (without interrupting the task or relying on the operator skill to self report). We have examined the reliability of the technique, and significantly extended our understanding of how artefacts affect recordings during both - a Verbal memory task of remembering a seven digit number and a Spacial memory task of remembering a 6x6 shaped grid. Our results showed that artefacts have a significantly different impact during the two types of tasks, further contributing insights into the existing guidelines of using fNIRS to assess workload during typical human computer interaction evaluation settings. We have further evaluated the sensitivity of the tool and understand the potential implications of using fNIRS as a measure in real-time. Our findings validated fNIRS as a sensitive workload measure, having consistent results in line with subjective measures, confirming a correlation between fNIRS and subjective workload questionnaires NASA-TLX and ISA. Having shown the relationship between fNIRS and workload, the last part of this thesis explores the use of fNIRS as a novel approach to providing users with concurrent feedback of their Mental Workload based on the measurements obtained objectively from fNIRS. We compare this feedback to traditional methods of asking users to self-assess and report their own mental workload during an Air Traffic Controller simulation game. In line with previous work, we con rm that self-reporting methods affect both perceived and actual performance. Furthermore, we found that our objective concurrent feedback technique allowed participants to reflect metacognitively on their Mental Workload during tasks, without reducing either actual or perceived performance. fNIRS showed potential to be a useful and reliable additional channel of information about the user during interaction, without further restricting the user during a typical evaluation settings. We found it sensitive to workload, being able to distinguish between various levels of workload, and with great potential for real time, continuous use during tasks. Finally, we explored a new direction of using fNIRS’s assessment of workload in real time, and we investigated how users can use feedback of their current workload state during tasks. This proved to allow users to think metacognitively about their workload during tasks, without negatively affecting their performance or workload

How stress affects functional near-infrared spectroscopy (fNIRS) measurements of mental workload

Recent work has demonstrated that functional Near-Infrared Spectroscopy has the potential to measure changes in Mental Workload with increasing ecological validity. It is not clear, however, whether these measurements are affected by anxiety and stress of the workload, where our informal observations see some participants enjoying the workload and succeeding in tasks, while others worry and struggle with the tasks. This research evaluated the effects of stress on fNIRS measurements and performance, using the Montreal Imaging Stress Task to manipulate the experience of stress. While our results largely support this hypothesis, our conclusions were undermined by data from the Rest condition, which indicated that Mental Workload and Stress were often higher than during tasks. We hypothesize that participants were experiencing anxiety in anticipation of subsequent stress tasks. We discuss this hypothesis and present a revised study designed to better control for this result

Using fNIRS in usability testing: understanding the effect of web form layout on mental workload

Amongst the many tasks in our lives, we encounter web forms on a regular basis, whether they are mundane like registering for a website, or complex and important like tax returns. There are many aspects of Usability, but one concern for user interfaces is to reduce mental workload and error rates. Whilst most assessment of mental workload is subjective and retrospective reporting by users, we examine the potential of functional Near Infrared Spectroscopy (fNIRS) as a tool for objectively and concurrently measuring mental workload during usability testing. We use this technology to evaluate the design of three different form layouts for a car insurance claim process, and show that a form divided into subforms increases mental workload, contrary to our expectations. We conclude that fNIRS is highly suitable for objectively examining mental workload during usability testing, and will therefore be able to provide more detailed insight than summative retrospective assessments. Further, for the fNIRS community, we show that the technology can easily move beyond typical psychology tasks, and be used for more natural study tasks

Using fNIRS in usability testing: understanding the effect of web form layout on mental workload

Amongst the many tasks in our lives, we encounter web forms on a regular basis, whether they are mundane like registering for a website, or complex and important like tax returns. There are many aspects of Usability, but one concern for user interfaces is to reduce mental workload and error rates. Whilst most assessment of mental workload is subjective and retrospective reporting by users, we examine the potential of functional Near Infrared Spectroscopy (fNIRS) as a tool for objectively and concurrently measuring mental workload during usability testing. We use this technology to evaluate the design of three different form layouts for a car insurance claim process, and show that a form divided into subforms increases mental workload, contrary to our expectations. We conclude that fNIRS is highly suitable for objectively examining mental workload during usability testing, and will therefore be able to provide more detailed insight than summative retrospective assessments. Further, for the fNIRS community, we show that the technology can easily move beyond typical psychology tasks, and be used for more natural study tasks

Measuring Mental Workload Variations in Office Work Tasks using fNIRS

The motivation behind using physiological measures to estimate cognitive activity is typically to build technology that can help people to understand themselves and their work, or indeed for systems to do so and adapt. While functional Near Infrared Spectroscopy (fNIRS) has been shown to reliably reflect manipulations of mental workload in different work tasks, we still need to establish whether fNIRS can differentiate variety within common office-like tasks in order to broaden our understanding of the factors involved in tracking them in real working conditions. 20 healthy participants (8 females, 12 males), whose work included office-like tasks, took part in a user study that investigated a) the sensitivity of fNIRS for measuring mental workload variations in representations of everyday reading and writing tasks, and b) how representations of natural interruptions are reflected in the data. Results supported fNIRS measuring PFC activation in differentiating between workload levels for reading tasks but not writing tasks in terms of increased oxygenated haemoglobin (O2Hb) and decreased deoxygenated haemoglobin (HHb), for harder conditions compared to easier conditions. There was considerable support for fNIRS in detecting changes in workload levels due to interruptions. Variations in workload levels during the interruptions could be understood in relation to spare capacity models. These findings may guide future work into sustained monitoring of cognitive activity in real-world settings

Trust Me, I’m an Influencer! - A Comparison of Perceived Trust in Human and Virtual Influencers

Influencers in social media are often perceived as a trusted source for many people which is why companies increasingly promote their products through them. However, influencers can also cause reputational damage for a brand. Virtual (computer-generated) influencers can be used to minimize these risks and to better tailor content to a target group of a company. As trust is one success factor of online marketing, we examine differences in the perception of trust in human and virtual influencers. In a first online survey study, we presented N = 112 participants the content of human and virtual influencers, published on Instagram. Preliminary findings reveal that although participants were often unsure whether the presented influencer was human or computer-generated, perceived trust, social presence, and humanness was consistently rated higher for human influencers. To gain deeper insights into potential, unconscious decision conflicts which can determine trust evaluations, a follow-up neuroimaging study is discussed

Mental workload as personal data: designing a cognitive activity tracker

Research continues to correlate physical signals with mental activity, as opposed to physical activity, with physiological sensors. Further, with the proliferation of wearable technology, it seems imminent that our smart watches can soon keep track of our mental activity as well as our physical activity. Our research is working towards accurately measuring Mental Workload ‘in the wild’ using physiological sensors. While we work towards that goal, however, we have begun to explore the design aspects of representing personal cognitive data to users; analogous to a step counter for physical activity. We present the results of diary studies, focus groups, and prototyping exercises to identify design considerations for future cognitive activity trackers