Emotional Brain-Computer Interfaces

Research in Brain-computer interface (BCI) has significantly increased during the last few years. In addition to their initial role as assisting devices for the physically challenged, BCIs are now proposed for a wider range of applications. As in any HCI application, BCIs can also benefit from adapting their operation to the emotional state of the user. BCIs have the advantage of having access to brain activity which can provide signicant insight into the user's emotional state. This information can be utilized in two manners. 1) Knowledge of the inuence of the emotional state on brain activity patterns can allow the BCI to adapt its recognition algorithms, so that the intention of the user is still correctly interpreted in spite of signal deviations induced by the subject's emotional state. 2) The ability to recognize emotions can be used in BCIs to provide the user with more natural ways of controlling the BCI through affective modulation. Thus, controlling a BCI by recollecting a pleasant memory can be possible and can potentially lead to higher information transfer rates.\ud These two approaches of emotion utilization in BCI are elaborated in detail in this paper in the framework of noninvasive EEG based BCIs

Plug-in to fear: game biosensors and negative physiological responses to music

The games industry is beginning to embark on an ambitious journey into the world of biometric gaming in search of more exciting and immersive gaming experiences. Whether or not biometric game technologies hold the key to unlock the “ultimate gaming experience” hinges not only on technological advancements alone but also on the game industry’s understanding of physiological responses to stimuli of different kinds, and its ability to interpret physiological data in terms of indicative meaning. With reference to horror genre games and music in particular, this article reviews some of the scientific literature relating to specific physiological responses induced by “fearful” or “unpleasant” musical stimuli, and considers some of the challenges facing the games industry in its quest for the ultimate “plugged-in” experience

Affective level design for a role-playing videogame evaluated by a brain\u2013computer interface and machine learning methods

Game science has become a research field, which attracts industry attention due to a worldwide rich sell-market. To understand the player experience, concepts like flow or boredom mental states require formalization and empirical investigation, taking advantage of the objective data that psychophysiological methods like electroencephalography (EEG) can provide. This work studies the affective ludology and shows two different game levels for Neverwinter Nights 2 developed with the aim to manipulate emotions; two sets of affective design guidelines are presented, with a rigorous formalization that considers the characteristics of role-playing genre and its specific gameplay. An empirical investigation with a brain\u2013computer interface headset has been conducted: by extracting numerical data features, machine learning techniques classify the different activities of the gaming sessions (task and events) to verify if their design differentiation coincides with the affective one. The observed results, also supported by subjective questionnaires data, confirm the goodness of the proposed guidelines, suggesting that this evaluation methodology could be extended to other evaluation tasks

Gamification and Coding to Engage Primary School Students in Learning Mathematics: A Case Study

This paper describes a pilot educational project made in a Primary School in Italy (Scuola Primaria Alessandro Manzoni at Mulazzano, Milan) implemented in 2016 and 2017. The project was born from a specific request: the school aimed at improving the results achieved by students aged 7 during the National Tests for Mathematics since they registered performances lower than the National Average. In this context, we supported teachers providing information tools and methods to improve performances. Our aim was to develop new game-oriented approaches to problem-solving, mixing our different experiences and competences (organization design, information technologies, psychology). We provided a broader spectrum of parameters tools and keys to understand how to achieve an inclusive approach personalized on students, involving them and their teachers in the project. This cooperative approach allowed us to collect interesting observations about learning styles, pointing out the negative impact that standardized processes and instruments can have on self-esteem and consequently on the performance of pupils. We argue that addressing pupils in considering mathematics as continuous research and development can increase their performances in National Tests execution. Children free to realize their own experiments and observations dramatically improve their involvement and curiosity about Mathematics

A conceptual affective design framework for the use of emotions in computer game design

The purpose of this strategy of inquiry is to understand how emotions influence gameplay and to review contemporary techniques to design for them in the aim of devising a model that brings current disparate parts of the game design process together. Emotions sit at the heart of a game player’s level of engagement. They are evoked across many of the components that facilitate gameplay including the interface, the player’s avatar, non-player characters and narrative. Understanding the role of emotion in creating truly immersive and believable environments is critical for game designers. After discussing a taxonomy of emotion, this paper will present a systematic literature review of designing for emotion in computer games. Following this, a conceptual framework for affective design is offered as a guide for the future of computer game design

Emotion Interaction With Virtual Reality Using Hybrid Emotion Classification Technique Toward Brain Signals

Human computer interaction (HCI) considered main aspect in virtual reality (VR) especially in the context of emotion, where users can interact with virtual reality through their emotions and it could be expressed in virtual reality. Last decade many researchers focused on emotion classification in order to employ emotion in interaction with virtual reality, the classification will be done based on Electroencephalogram (EEG) brain signals. This paper provides a new hybrid emotion classification method by combining self- assessment, arousal valence dimension and variance of brain hemisphere activity to classify users’ emotions. Self-assessment considered a standard technique used for assessing emotion, arousal valence emotion dimension model is an emotion classifier with regards to aroused emotions and brain hemisphere activity that classifies emotion with regards to right and left hemisphere. This method can classify human emotions, two basic emotions is highlighted i.e. happy and sad. EEG brain signals are used to interpret the users’ emotional. Emotion interaction is expressed by 3D model walking expression in VR. The results show that the hybrid method classifies the highlighted emotions in different circumstances, and how the 3D model changes its walking style according to the classified users’ emotions. Finally, the outcome is believed to afford new technique on classifying emotions with feedback through 3D virtual model walking expression

EEG-based measurement system for monitoring student engagement in learning 4.0

A wearable system for the personalized EEG-based detection of engagement in learning 4.0 is proposed. In particular, the effectiveness of the proposed solution is assessed by means of the classification accuracy in predicting engagement. The system can be used to make an automated teaching platform adaptable to the user, by managing eventual drops in the cognitive and emotional engagement. The effectiveness of the learning process mainly depends on the engagement level of the learner. In case of distraction, lack of interest or superficial participation, the teaching strategy could be personalized by an automatic modulation of contents and communication strategies. The system is validated by an experimental case study on twenty-one students. The experimental task was to learn how a specific human-machine interface works. Both the cognitive and motor skills of participants were involved. De facto standard stimuli, namely (1) cognitive task (Continuous Performance Test), (2) music background (Music Emotion Recognition-MER database), and (3) social feedback (Hermans and De Houwer database), were employed to guarantee a metrologically founded reference. In within-subject approach, the proposed signal processing pipeline (Filter bank, Common Spatial Pattern, and Support Vector Machine), reaches almost 77% average accuracy, in detecting both cognitive and emotional engagement

An experimental protocol to access immersiveness in video games

In the video game industry, great importance is given to the experience that the user has while playing a game. In particular, this experience benefits from the players' perceived sense of being in the game or immersion. The level of user immersion depends not only on the game's content but also on how the game is displayed, thus on its User Interface (UI) and the Head's-Up Display (HUD). Another factor influencing immersiveness that has been found in the literature is the player's expertise: the more experience the user has with a specific game, the less they need information on the screen to be immersed in the game. Player's level of immersion can be accessed by using both questionnaires of their perceived experience and exploiting their behavioural and physiological responses while playing the target game. Therefore, in this paper, we propose an experimental protocol to access immersiveness of gamers while playing a third-person shooter (Fortnite) with UIs with a standard, a dietetic, and a proposed HUD. A subjective evaluation of the immersion will be provided by completing the Immersive Experience Questionnaire (IEQ), while objective indicators will be provided by face tracking, behaviour and physiological responses analyses. The ultimate goal of this study is to define guidelines for video game UI development that can enhance the players' immersion.Comment: Accepted at the Italian Workshop on Artificial Intelligence for Human Machine Interaction (AIxHMI 2023), November 06, 2023, Rome, Ital

Communicating Spiritual Experience with Video Game Technology.

Michael Highland and Gino Yu “Communicating Spiritual Experience with Video Game Technology” deal with the aspect of experience. They stress that given the interactive nature of video game technology, it is an ideal medium for representing and communicating experience. As the game world is causally dependent on input from individual players, they evoke feelings that are urgent, direct, and personalized. Online virtual spaces therefore provide an environment for people of different faiths to come together in conversation

Classification of Affective Data to Evaluate the Level Design in a Role-Playing Videogame

This paper presents a novel approach to evaluate game level design strategies, applied to role playing games. Following a set of well defined guidelines, two game levels were designed for Neverwinter Nights 2 to manipulate particular emotions like boredom or flow, and tested by 13 subjects wearing a brain computer interface helmet. A set of features was extracted from the affective data logs and used to classify different parts of the gaming sessions, to verify the correspondence of the original level aims and the effective results on people emotions. The very interesting correlations observed, suggest that the technique is extensible to other similar evaluation tasks