Collecting neurophysiological data to investigate users’ cognitive states during game play

This paper explores the potential of collecting neurophysiological data in order to further understand user’s learning experience. The experimental setup involves collecting electroencephalographic signal (EEG) from the brain cortex to infer users’ cognitive state while they played an educational video game designed to support the learning of Newtonian mechanics. Preliminary results suggest that this neuroscience perspective is quite promising in the idea of quantitatively characterizing users’ learning experience. This could be an innovative and promising avenue in general game development or in educational videogame research field

DETERMINING THE DEGREE OF PLAYER ENGAGEMENT IN A COMPUTER GAME WITH ELEMENTS OF A SOCIAL CAMPAIGN USING COGNITIVE NEUROSCIENCE TECHNIQUES

Due to the popularity of video games in various applications, including both commercial and social marketing, there is a need to assess their content in terms of player satisfaction, already at the production stage. For this purpose, the indices used in EEG tests can be used. In this publication, a formula has been created based on the player's commitment to determining which elements in the game should be improved and for which graphic emblems connected with social campaigns were more memorable and whether this was related to commitment. The survey was conducted using a 2D platform game created in Unity based on observations of 28 recipients. To evaluate the elements occurring in the game at which we obtain a higher memory for graphic characters, a corresponding pattern was created based on player involvement. The optimal Index for moving and static objects and the Index for destruction were then selected based on the feedback. Referring to the issue of graphic emblems depicting social campaigns should be placed in a place where other activities such as fighting will not be distracted, everyone will be able to reach the level where the recently placed advertisement is. This study present the developed method to determine the degree of player's engagement in particular elements in the game using the EEG and to explore the relationship between the visibility of social advertising and engagement in a 2D platform game where the player has to collect three keys and defeat the ultimate opponent.&nbsp

Using EEG data to predict engagement in face-to-face conversations

To date engagement in face-to-face conversation has been studied almost exclusively through the post event measurement of self-reporting surveys or questionnaires. Electroencephalography (EEG) has been used for decades to examine brain activity for both research and diagnostic purposes. Medical grade EEG equipment is both costly and confined to being used within laboratory settings. With the recent advent of off-the-shelf consumer grade portable EEG-devices, novel psychological research on cognitive computations that have traditionally been confined to self-report, is now a reality. Although it is well documented that people use their cognitive abilities during conversations, an extensive literature search found no studies on the use of EEG data to obtain a neurological engagement score during conversation. Consequently, the present study sought to remedy a gap in the literature, and capitalised on the readily available consumer-grade portable EEG equipment. A within-participants quantitative study with 42 participants examined whether EEG predicted engagement during face-to-face getting acquainted conversations. Participants’ alpha and beta brain activity were examined from EEG data collected during two separate conversations, and participants also completed a post-hoc self-report on their engagement and attention. The results of the study found a significant difference for participants’ alpha brain activity and engagement, but not for the beta activity and engagement. There was also no significant difference found for participants attention and their alpha or beta activity. A surprising additional finding in the present study was a within-participant consistency for both alpha and beta activity across the two conversations, which is consistent with individual differences stability found in other psychophysiological studies. Overall, the present study has found that alpha activity is necessary for neurological engagement during face-to-face getting acquainted conversations. Therefore, future research is warranted on the use of EEG as an additional tool in face-to-face communication to compliment self-report and measure engagement

Measuring Brain Activation Patterns from Raw Single-Channel EEG during Exergaming: A Pilot Study

Physical and cognitive rehabilitation is deemed crucial to attenuate symptoms and to improve the quality of life in people with neurodegenerative disorders, such as Parkinson's Disease. Among rehabilitation strategies, a novel and popular approach relies on exergaming: the patient performs a motor or cognitive task within an interactive videogame in a virtual environment. These strategies may widely benefit from being tailored to the patient's needs and engagement patterns. In this pilot study, we investigated the ability of a low-cost BCI based on single-channel EEG to measure the user's engagement during an exergame. As a first step, healthy subjects were recruited to assess the system's capability to distinguish between (1) rest and gaming conditions and (2) gaming at different complexity levels, through Machine Learning supervised models. Both EEG and eye-blink features were employed. The results indicate the ability of the exergame to stimulate engagement and the capability of the supervised classification models to distinguish resting stage from game-play(accuracy > 95%). Finally, different clusters of subject responses throughout the game were identified, which could help define models of engagement trends. This result is a starting point in developing an effectively subject-tailored exergaming system

Real-time estimation of EEG-based engagement in different tasks

: Objective.Recent trends in brain-computer interface (BCI) research concern the passive monitoring of brain activity, which aim to monitor a wide variety of cognitive states. Engagement is such a cognitive state, which is of interest in contexts such as learning, entertainment or rehabilitation. This study proposes a novel approach for real-time estimation of engagement during different tasks using electroencephalography (EEG).Approach.Twenty-three healthy subjects participated in the BCI experiment. A modified version of the d2 test was used to elicit engagement. Within-subject classification models which discriminate between engaging and resting states were trained based on EEG recorded during a d2 test based paradigm. The EEG was recorded using eight electrodes and the classification model was based on filter-bank common spatial patterns and a linear discriminant analysis. The classification models were evaluated in cross-task applications, namely when playing Tetris at different speeds (i.e. slow, medium, fast) and when watching two videos (i.e. advertisement and landscape video). Additionally, subjects' perceived engagement was quantified using a questionnaire.Main results.The models achieved a classification accuracy of 90% on average when tested on an independent d2 test paradigm recording. Subjects' perceived and estimated engagement were found to be greater during the advertisement compared to the landscape video (p= 0.025 andp<0.001, respectively); greater during medium and fast compared to slow Tetris speed (p<0.001, respectively); not different between medium and fast Tetris speeds. Additionally, a common linear relationship was observed for perceived and estimated engagement (rrm= 0.44,p<0.001). Finally, theta and alpha band powers were investigated, which respectively increased and decreased during more engaging states.Significance.This study proposes a task-specific EEG engagement estimation model with cross-task capabilities, offering a framework for real-world applications

Neural correlates of flow, boredom, and anxiety in gaming: An electroencephalogram study

Games are engaging and captivating from a human-computer interaction (HCI) perspective as they can facilitate a highly immersive experience. This research examines the neural correlates of flow, boredom, and anxiety during video gaming. A within-subject experimental study (N = 44) was carried out with the use of electroencephalogram (EEG) to assess the brain activity associated with three states of user experience - flow, boredom, and anxiety - in a controlled gaming environment. A video game, Tetris, was used to induce flow, boredom, and anxiety. A 64 channel EEG headset was used to track changes in activation patterns in the frontal, temporal, parietal, and occipital lobes of the players\u27 brains during the experiment. EEG signals were pre-processed and Fast Fourier Transformation values were extracted and analyzed. The results suggest that the EEG potential in the left frontal lobe is lower in the flow state than in the resting and boredom states. The occipital alpha is lower in the flow state than in the resting state. Similarly, the EEG theta in the left parietal lobe is lower during the flow state than the resting state. However, the EEG theta in the frontal-temporal region of the brain is higher in the flow state than in the anxiety state. The flow state is associated with low cognitive load, presence of attention levels, and loss of self-consciousness when compared to resting and boredom states --Abstract, page iii

Towards an integrated framework to measure user engagement with interactive or physical products

Building great products or services is not easy; users want products and services that exceed their expectations and evolve with their needs; it is not just about building the right features. Knowing the user engagement (UE) towards a physical, virtual product or service can give valuable information that could be used as feedback for the design, enhancing its chances of success. In the context of user-centered design, UE is the assessment of the user experience characterized by the study of the individual’s cognitive, affective, and behavioral response to some stimulus, such as a product, a service, or a website. UE considers not only the users’ requirements and wishes but also their perceptions and reactions during and after an interaction with a product, system, or service. Many studies looking to quantify the UE are available. Still, a framework that provides a generic view of the most commonly used methods and metrics to measure UE does not yet exist in the literature. Aiming to understand the UE better, in this research, we developed a conceptual framework summarizing the available metrics and techniques used across different contexts, including good practices of self-report methods and physiological approaches. We expect this study will allow future researchers, developers, and designers to consider the UE as one of the most prominent product/service success indicators and use this guideline to find the more appropriate method, technique, and metric for its measurement

An Exploration of the Feasibility of Functional Near-Infrared Spectroscopy as a Neurofeedback Cueing System for the Mitigation of the Vigilance Decrement

Vigilance is the capacity for observers to maintain attention over extended periods of time, and has most often been operationalized as the ability to detect rare and critical signals (Davies & Parasuraman, 1982; Parasuraman, 1979; Warm, 1984). Humans, however, have natural physical and cognitive limitations that preclude successful long-term vigilance performance and consequently, without some means of assistance, failures in operator vigilance are likely to occur. Such a decline in monitoring performance over time has been a robust finding in vigilance experiments for decades and has been called the vigilance decrement function (Davies & Parasuraman, 1982; Mackworth, 1948). One of the most effective countermeasures employed to maintain effective performance has been cueing: providing the operator with a reliable prompt concerning signal onset probability. Most protocols have based such cues on task-related or environmental factors. The present dissertation examines the efficacy of cueing when nominally based on operator state (i.e., blood oxygenation of cortical tissue) in a novel vigilance task incorporating dynamic displays over three studies. Results pertaining to performance outcomes, physiological measures (cortical blood oxygenation and heart rate variability), and perceived workload and stress are interpreted via Signal Detection Theory and the Resource Theory of vigilance

Noninvasive Physiological Measures And Workload Transitions:an Investigation Of Thresholds Using Multiple Synchronized Sensors

The purpose of this study is to determine under what conditions multiple minimally intrusive physiological sensors can be used together and validly applied for use in areas which rely on adaptive systems including adaptive automation and augmented cognition. Specifically, this dissertation investigated the physiological transitions of operator state caused by changes in the level of taskload. Three questions were evaluated including (1) Do differences exist between physiological indicators when examined between levels of difficulty? (2) Are differences of physiological indicators (which may exist) between difficulty levels affected by spatial ability? (3) Which physiological indicators (if any) account for variation in performance on a spatial task with varying difficulty levels? The Modular Cognitive State Gauge model was presented and used to determine which basic physiological sensors (EEG, ECG, EDR and eye-tracking) could validly assess changes in the utilization of two-dimensional spatial resources required to perform a spatial ability dependent task. Thirty-six volunteers (20 female, 16 male) wore minimally invasive physiological sensing devices while executing a challenging computer based puzzle task. Specifically, participants were tested with two measures of spatial ability, received training, a practice session, an experimental trial and completed a subjective workload survey. The results of this experiment confirmed that participants with low spatial ability reported higher subjective workload and performed poorer when compared to those with high spatial ability. Additionally, there were significant changes for a majority of the physiological indicators between two difficulty levels and most importantly three measures (EEG, ECG and eye-tracking) were shown to account for variability in performance on the spatial task