Exploring Effects of Background Music in A Serious Game on Attention by Means of EEG Signals in Children

Music and Serious Games are separately useful alternative therapy methods for helping people with a cognitive disorder, including Attention Deficit Hyperactivity Disorder (ADHD). The goal of this thesis is to explore the effect of background music on children with and without ADHD. In this study, a simple Tetris game is designed with Beethoven, Mozart music, and no-music. There are different brainwave techniques for recording; among others, the electroencephalography (EEG) allows for the most efficient use of BCI. We recorded the EEG brain signals of the regular and ADHD subjects who played the Tetris we designed according to our protocol that consists of three trials with three different background music. Attention related Alpha and Beta waves of EEG signals analyzed based on time and time-frequency domain features. The changes in the data over the 1-minute Tetris game sections are investigated with the Short-time Fourier Transform (STFT) method. The results showed that music has a considerable impact on attention of children. When it comes to music types, in general, Mozart music increases Beta waves while decreasing the Alpha band waves for subjects without ADHD. On the other hand, Beethoven music increased both Alpha and Beta band values for children with ADHD

Classification of Affective Data to Evaluate the Level Design in a Role-Playing Videogame

This paper presents a novel approach to evaluate game level design strategies, applied to role playing games. Following a set of well defined guidelines, two game levels were designed for Neverwinter Nights 2 to manipulate particular emotions like boredom or flow, and tested by 13 subjects wearing a brain computer interface helmet. A set of features was extracted from the affective data logs and used to classify different parts of the gaming sessions, to verify the correspondence of the original level aims and the effective results on people emotions. The very interesting correlations observed, suggest that the technique is extensible to other similar evaluation tasks

Affective level design for a role-playing videogame evaluated by a brain\u2013computer interface and machine learning methods

Game science has become a research field, which attracts industry attention due to a worldwide rich sell-market. To understand the player experience, concepts like flow or boredom mental states require formalization and empirical investigation, taking advantage of the objective data that psychophysiological methods like electroencephalography (EEG) can provide. This work studies the affective ludology and shows two different game levels for Neverwinter Nights 2 developed with the aim to manipulate emotions; two sets of affective design guidelines are presented, with a rigorous formalization that considers the characteristics of role-playing genre and its specific gameplay. An empirical investigation with a brain\u2013computer interface headset has been conducted: by extracting numerical data features, machine learning techniques classify the different activities of the gaming sessions (task and events) to verify if their design differentiation coincides with the affective one. The observed results, also supported by subjective questionnaires data, confirm the goodness of the proposed guidelines, suggesting that this evaluation methodology could be extended to other evaluation tasks

Novel Virtual Environment for Alternative Treatment of Children with Cerebral Palsy

Cerebral palsy is a severe condition usually caused by decreased brain oxygenation during pregnancy, at birth or soon after birth. Conventional treatments for cerebral palsy are often tiresome and expensive, leading patients to quit treatment. In this paper, we describe a virtual environment for patients to engage in a playful therapeutic game for neuropsychomotor rehabilitation, based on the experience of the occupational therapy program of the Nucleus for Integrated Medical Assistance (NAMI) at the University of Fortaleza, Brazil. Integration between patient and virtual environment occurs through the hand motion sensor “Leap Motion,” plus the electroencephalographic sensor “MindWave,” responsible for measuring attention levels during task execution. To evaluate the virtual environment, eight clinical experts on cerebral palsy were subjected to a questionnaire regarding the potential of the experimental virtual environment to promote cognitive and motor rehabilitation, as well as the potential of the treatment to enhance risks and/or negatively influence the patient’s development. Based on the very positive appraisal of the experts, we propose that the experimental virtual environment is a promising alternative tool for the rehabilitation of children with cerebral palsy.Cerebral palsy is a severe condition usually caused by decreased brain oxygenation during pregnancy, at birth or soon after birth. Conventional treatments for cerebral palsy are often tiresome and expensive, leading patients to quit treatment. In this paper, we describe a virtual environment for patients to engage in a playful therapeutic game for neuropsychomotor rehabilitation, based on the experience of the occupational therapy program of the Nucleus for Integrated Medical Assistance (NAMI) at the University of Fortaleza, Brazil. Integration between patient and virtual environment occurs through the hand motion sensor “Leap Motion,” plus the electroencephalographic sensor “MindWave,” responsible for measuring attention levels during task execution. To evaluate the virtual environment, eight clinical experts on cerebral palsy were subjected to a questionnaire regarding the potential of the experimental virtual environment to promote cognitive and motor rehabilitation, as well as the potential of the treatment to enhance risks and/or negatively influence the patient’s development. Based on the very positive appraisal of the experts, we propose that the experimental virtual environment is a promising alternative tool for the rehabilitation of children with cerebral palsy

Predictive analysis of auditory attention from physiological signals

In recent years, there has been considerable interest in recording physiological signals from the human body to investigate various responses. Attention is one of the key aspects that physiologists, neuroscientists, and engineers have been exploring. Many theories have been established on auditory and visual selective attention. To date, the number of studies investigating the physiological responses of the human body to auditory attention on natural speech is, surprisingly, very limited, and there is a lack of public datasets. Investigating such physiological responses can open the door to new opportunities, as auditory attention plays a key role in many cognitive functionalities, thus impacting on learning and general task performance. In this thesis, we investigated auditory attention on the natural speech by processing physiological signals such as Electroencephalogram (EEG), Galvanic Skin Response (GSR), and Photoplethysmogram (PPG). An experiment was designed based on the well established dichotic listening task. In the experiment, we presented an audio stimulus under different auditory conditions: background noise level, length, and semanticity of the audio message. The experiment was conducted with 25 healthy, non-native speakers. The attention score was computed by counting the number of correctly identified words in the transcribed text response. All the physiological signals were labeled with their auditory condition and attention score. We formulated four predictive tasks exploiting the collected signals: Attention score, Noise level, Semanticity, and LWR (Listening, Writing, Resting, i.e., the state of the participant). In the first part, we analysed all the user text responses collected in the experiment. The statistical analysis reveals a strong dependency of the attention level on the auditory conditions. By applying hierarchical clustering, we could identify the experimental conditions that have similar effects on attention score. Significantly, the effect of semanticity appeared to vanish under high background noise. Then, analysing the signals, we found that the-state-of-the-art algorithms for artifact removal were inefficient for large datasets, as they require manual intervention. Thus, we introduced an EEG artifact removal algorithm with tuning parameters based on Wavelet Packet Decomposition (WPD). The proposed algorithm operates with two tuning parameters and three modes of wavelet filtering: Elimination, Linear Attenuation, and Soft-thresholding. Evaluating the algorithm performance, we observed that it outperforms state-of-the-art algorithms based on Independent Component Analysis (ICA). The evaluation was based on the spectrum, correlation, and distribution of the signals along with the performance in predictive tasks. We also demonstrate that a proper tuning of the algorithm parameters allows achieving further better results. After applying the artifact removal algorithm on EEG, we analysed the signals in terms of correlation of spectral bands of each electrode and attention score, semanticity, noise level, and state of the participant LWR). Next, we analyse the Event-Related Potential (ERP) on Listening, Writing and Resting segments of EEG signal, in addition to spectral analysis of GSR and PPG. With this thesis, we release the collected experimental dataset in the public domain, in order for the scientific community to further investigate the various auditory processing phenomena and their relation with EEG, GSR and PPG responses. The dataset can be used also to improve predictive tasks or design novel Brain-Computer-Interface (BCI) systems based on auditory attention. We also use the deeplearning approach to exploit the spatial relationship of EEG electrodes and inter-subject dependency of a model. As a domain application, we finally discuss the implications of auditory attention assessment for serious games and propose a 3-dimensional difficulty model to design game levels and dynamically adapt the difficulty to the player status

A Review of Emotion Recognition Using EEG Data and Machine Learning Techniques

Using AI to help humans with handling their emotions and identifying their stress levels in the current stressful lifestyle will greatly help them manage their lifestyle. Using the deep learning techniques, it can be made possible by creating a virtual bot to observe and understand human emotions.   In this paper, the researcher try to review the comments from Reddit that are used, preprocessed and trained using Deep Neural Network to learn the emotions of the user. The inference engine module, which is a hybrid network consisting of convolutional neural network and recurrent neural network, is also interfaced. The model provides a high accuracy of response. The selection of frequency bands plays an important role in discerning patterns of brain-related emotions. This document explores a new method for selecting appropriate thematic bands instead of using fixed bands to detect emotions. A common spatial technique and machine   machines were used to classify the emotional states.  This document describes a number of possible technologies aimed at communication and other applications; however, they represent only a small sample of the extensive future potential of these technologies.  We have also focused on relatively anticipated breakthroughs in the discussion of applications in sensory, BCI technologies; but breakthroughs like the new portable sensor technology, which offers ultra-high-resolution spatial and time-based activity in the brain, opens the door to a much broader range of applications. Keywords: Emotions, EEG, Machine Learning, Deep Learning, Systems and Signals DOI: 10.7176/ISDE/11-4-04 Publication date:August 31st 2020

DeReFrame: a design-research framework to study game mechanics and game aesthetics in an engineering design process

The main aim of this research is to study gaming techniques and elements that may potentially be beneficial to the future development of CAD systems for engineering design, in particular to maintain cognitive engagement. A design-research framework, called DeReFrame, was employed to construct an experimental game-based CAD framework exploring this. This research is based on reviews from the literature and experimental studies and include quantitative and qualitative data analysis methods measuring engineers’ performance and emotional responses. The thesis presents the construction process of the framework (DeReframe) to study a set of game mechanics and game aesthetics in an engineering design process and compare this with the traditional CAD. The framework was used to design and implement a game-based CAD system, called ICAD which was embedded with the following game mechanics of Directional Goals, Progression, Performance-Feedback and Rewards-Achievement. The DeReFrame and ICAD evolved through the experimental studies. In each case, selected game mechanics were at the core of each interaction and iteration which gave rise to feelings of progress, competence and mastery. The final results from the DeReFrame framework and ICAD indicated that gamified approaches should be included in engineering design with CAD: in particular the game mechanics of performance feedback and rewards-achievements influence engineers’ behaviour by supporting them within the problem-solving process creating an engaging-challenging interaction. In conclusion, this research has shown that a framework, that includes both engineering requirements and gamified aspects into consideration, cam serve as a basis for implementing game-based CAD to facilitate performance by providing engaging experiences for engineers

Music, computing and health: A roadmap for the current and future roles of music technology for health care and well-being.

Health and self-regulatio