Prediction of difficulty levels in video games from ongoing EEG

Real-time assessment of mental workload from EEG plays an important role in enhancing symbiotic interaction of human operators in immersive environments. In this study we thus aimed at predicting the difficulty level of a video game a person is playing at a particular moment from the ongoing EEG activity. Therefore, we made use of power modulations in the theta (4–7 Hz) and alpha (8–13 Hz) frequency bands of the EEG which are known to reflect cognitive workload. Since the goal was to predict from multiple difficulty levels, established binary classification approaches are futile. Here, we employ a novel spatial filtering method (SPoC) that finds spatial filters such that their corresponding bandpower dynamics maximally covary with a given target variable, in this case the difficulty level. EEG was recorded from 6 participants playing a modified Tetris game at 10 different difficulty levels. We found that our approach predicted the levels with high accuracy, yielding a mean prediction error of less than one level.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeBMBF, 01GQ0850, Verbundprojekt: Bernstein Fokus Neurotechnologie - Nichtinvasive Neurotechnologie für Mensch-Maschine Interaktio

The brain in flow: a systematic review on the neural basis of the flow state

Background: Flow state is a subjective experience that people report when task performance is experienced as automatic, intrinsically rewarding, optimal and effortless. While this intriguing phenomenon is the subject of a plethora of behavioural studies, only recently researchers have started to look at its neural correlates. Here, we aim to systematically and critically review the existing literature on the neural correlates of the flow state. Methods: Three electronic databases (Web of Science, Scopus and PsycINFO) were searched to acquire information on eligible articles in July, 2021, and updated in March, 2022. Studies that measured or manipulated flow state (through questionnaires or employing experimental paradigms) and recorded associated brain activity with electroencephalography (EEG), functional magnetic resonance (fMRI) or functional near-infrared spectroscopy (fNIRS) or manipulated brain activity with transcranial direct current stimulation (tDCS) were selected. We used the Cochrane Collaboration Risk of Bias 2 (RoB 2) tool to assess the methodological quality of eligible records. Results: In total, 25 studies were included, which involved 471 participants. In general, the studies that experimentally addressed flow state and its neural dynamics seem to converge on the key role of structures linked to attention, executive function and reward systems, giving to the anterior brain areas (e.g., the DLPC, MPFC, IFG) a crucial role in the experience of flow. However, the dynamics of these brain regions during flow state are inconsistent across studies. Discussion: In light of the results, we conclude that the current available evidence is sparse and inconclusive, which limits any theoretical debate. We also outline major limitations of this literature (the small number of studies, the high heterogeneity across them and their important methodological constraints) and highlight several aspects regarding experimental design and flow measurements that may provide useful avenues for future studies on this topic.Spanish Government 20CO1/012863Ministry of Science and Innovation, Spain (MICINN) Spanish Government PID2019-105635GBI00Junta de Andalucia DOC_0022

Emotional Brain-Computer Interfaces

Research in Brain-computer interface (BCI) has significantly increased during the last few years. In addition to their initial role as assisting devices for the physically challenged, BCIs are now proposed for a wider range of applications. As in any HCI application, BCIs can also benefit from adapting their operation to the emotional state of the user. BCIs have the advantage of having access to brain activity which can provide signicant insight into the user's emotional state. This information can be utilized in two manners. 1) Knowledge of the inuence of the emotional state on brain activity patterns can allow the BCI to adapt its recognition algorithms, so that the intention of the user is still correctly interpreted in spite of signal deviations induced by the subject's emotional state. 2) The ability to recognize emotions can be used in BCIs to provide the user with more natural ways of controlling the BCI through affective modulation. Thus, controlling a BCI by recollecting a pleasant memory can be possible and can potentially lead to higher information transfer rates.\ud These two approaches of emotion utilization in BCI are elaborated in detail in this paper in the framework of noninvasive EEG based BCIs

Evaluation of an adaptive game that uses EEG measures validated during the design process as inputs to a Biocybernetic Loop

Biocybernetic adaptation is a form of physiological computing whereby real-time data streaming from the brain and body is used by a negative control loop to adapt the user interface. This article describes the development of an adaptive game system that is designed to maximize player engagement by utilizing changes in real-time electroencephalography (EEG) to adjust the level of game demand. The research consists of four main stages: (1) the development of a conceptual framework upon which to model the interaction between person and system; (2) the validation of the psychophysiological inference underpinning the loop; (3) the construction of a working prototype; and (4) an evaluation of the adaptive game. Two studies are reported. The first demonstrates the sensitivity of EEG power in the (frontal) theta and (parietal) alpha bands to changing levels of game demand. These variables were then reformulated within the working biocybernetic control loop designed to maximize player engagement. The second study evaluated the performance of an adaptive game of Tetris with respect to system behavior and user experience. Important issues for the design and evaluation of closed-loop interfaces are discussed

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeEC/FP7/625991/EU/Hyperscanning 2.0 Analyses of Multimodal Neuroimaging Data: Concept, Methods and Applications/HYPERSCANNING 2.0DFG, 103586207, GRK 1589: Verarbeitung sensorischer Informationen in neuronalen Systeme

Decoding subjective emotional arousal from EEG during an immersive Virtual Reality experience

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation:37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a Long Short-Term Memory recurrent neural network.We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.Competing Interest StatementThe authors have declared no competing interest

Hybridizing 3-dimensional multiple object tracking with neurofeedback to enhance preparation, performance, and learning

Le vaste domaine de l’amélioration cognitive traverse les applications comportementales, biochimiques et physiques. Aussi nombreuses sont les techniques que les limites de ces premières : des études de pauvre méthodologie, des pratiques éthiquement ambiguës, de faibles effets positifs, des effets secondaires significatifs, des couts financiers importants, un investissement de temps significatif, une accessibilité inégale, et encore un manque de transfert. L’objectif de cette thèse est de proposer une méthode novatrice d’intégration de l’une de ces techniques, le neurofeedback, directement dans un paradigme d’apprentissage afin d’améliorer la performance cognitive et l’apprentissage. Cette thèse propose les modalités, les fondements empiriques et des données à l’appui de ce paradigme efficace d’apprentissage ‘bouclé’. En manipulant la difficulté dans une tâche en fonction de l’activité cérébrale en temps réel, il est démontré que dans un paradigme d’apprentissage traditionnel (3-dimentional multiple object tracking), la vitesse et le degré d’apprentissage peuvent être améliorés de manière significative lorsque comparés au paradigme traditionnel ou encore à un groupe de contrôle actif. La performance améliorée demeure observée même avec un retrait du signal de rétroaction, ce qui suggère que les effets de l’entrainement amélioré sont consolidés et ne dépendent pas d’une rétroaction continue. Ensuite, cette thèse révèle comment de tels effets se produisent, en examinant les corrélés neuronaux des états de préparation et de performance à travers les conditions d’état de base et pendant la tâche, de plus qu’en fonction du résultat (réussite/échec) et de la difficulté (basse/moyenne/haute vitesse). La préparation, la performance et la charge cognitive sont mesurées via des liens robustement établis dans un contexte d’activité cérébrale fonctionnelle mesurée par l’électroencéphalographie quantitative. Il est démontré que l’ajout d’une assistance- à-la-tâche apportée par la fréquence alpha dominante est non seulement appropriée aux conditions de ce paradigme, mais influence la charge cognitive afin de favoriser un maintien du sujet dans sa zone de développement proximale, ce qui facilite l’apprentissage et améliore la performance. Ce type de paradigme d’apprentissage peut contribuer à surmonter, au minimum, un des limites fondamentales du neurofeedback et des autres techniques d’amélioration cognitive : le manque de transfert, en utilisant une méthode pouvant être intégrée directement dans le contexte dans lequel l’amélioration de la performance est souhaitée.The domain of cognitive enhancement is vast, spanning behavioral, biochemical and physical applications. The techniques are as numerous as are the limitations: poorly conducted studies, ethically ambiguous practices, limited positive effects, significant side-effects, high financial costs, significant time investment, unequal accessibility, and lack of transfer. The purpose of this thesis is to propose a novel way of integrating one of these techniques, neurofeedback, directly into a learning context in order to enhance cognitive performance and learning. This thesis provides the framework, empirical foundations, and supporting evidence for a highly efficient ‘closed-loop’ learning paradigm. By manipulating task difficulty based on a measure of cognitive load within a classic learning scenario (3-dimentional multiple object tracking) using real-time brain activity, results demonstrate that over 10 sessions, speed and degree of learning can be substantially improved compared with a classic learning system or an active sham-control group. Superior performance persists even once the feedback signal is removed, which suggests that the effects of enhanced training are consolidated and do not rely on continued feedback. Next, this thesis examines how these effects occur, exploring the neural correlates of the states of preparedness and performance across baseline and task conditions, further examining correlates related to trial results (correct/incorrect) and task difficulty (slow/medium/fast speeds). Cognitive preparedness, performance and load are measured using well-established relationships between real-time quantified brain activity as measured by quantitative electroencephalography. It is shown that the addition of neurofeedback-based task assistance based on peak alpha frequency is appropriate to task conditions and manages to influence cognitive load, keeping the subject in the zone of proximal development more often, facilitating learning and improving performance. This type of learning paradigm could contribute to overcoming at least one of the fundamental limitations of neurofeedback and other cognitive enhancement techniques : a lack of observable transfer effects, by utilizing a method that can be directly integrated into the context in which improved performance is sought

EEG-based measurement system for monitoring student engagement in learning 4.0

A wearable system for the personalized EEG-based detection of engagement in learning 4.0 is proposed. In particular, the effectiveness of the proposed solution is assessed by means of the classification accuracy in predicting engagement. The system can be used to make an automated teaching platform adaptable to the user, by managing eventual drops in the cognitive and emotional engagement. The effectiveness of the learning process mainly depends on the engagement level of the learner. In case of distraction, lack of interest or superficial participation, the teaching strategy could be personalized by an automatic modulation of contents and communication strategies. The system is validated by an experimental case study on twenty-one students. The experimental task was to learn how a specific human-machine interface works. Both the cognitive and motor skills of participants were involved. De facto standard stimuli, namely (1) cognitive task (Continuous Performance Test), (2) music background (Music Emotion Recognition-MER database), and (3) social feedback (Hermans and De Houwer database), were employed to guarantee a metrologically founded reference. In within-subject approach, the proposed signal processing pipeline (Filter bank, Common Spatial Pattern, and Support Vector Machine), reaches almost 77% average accuracy, in detecting both cognitive and emotional engagement