A taxonomy and state of the art revision on affective games

Affective Games are a sub-field of Affective Computing that tries to study how to design videogames that are able to react to the emotions expressed by the player, as well as provoking desired emotions to them. To achieve those goals it is necessary to research on how to measure and detect human emotions using a computer, and how to adapt videogames to the perceived emotions to finally provoke them to the players. This work presents a taxonomy for research on affective games centring on the aforementioned issues. Here we devise as well a revision of the most relevant published works known to the authors on this area. Finally, we analyse and discuss which important research problem are yet open and might be tackled by future investigations in the area of Affective GamesThis work has been co-funded by the following research projects: EphemeCH (TIN2014-56494-C4-{1,4}-P) and DeepBio (TIN2017-85727-C4-3-P) by Spanish Ministry of Economy and Competitivity, under the European Regional Development Fund FEDER, and Justice Programme of the European Union (2014–2020) 723180 – RiskTrack – JUST-2015-JCOO-AG/JUST-2015-JCOO-AG-

A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Physiopad: development of a non-invasive game controller toolkit to study physiological responses for Game User Research

Os jogos afectivos usam as respostas fisiológicas do jogador para criar um ambiente adequado ao estado emocional do utilizador. A investigação destes jogos tem sido explorada nos últimos anos. Estas experiências, contudo, ainda requerem sistemas complexos e difíceis de utilizar. Nesta dissertação, é proposta a construção de um dispositivo capaz de ler dados fisiológicos de forma não invasiva e que seja de fácil utilização. Este aparelho faz a leitura do ritmo cardíaco e dos níveis de excitação do jogador, além disso foi criado um software para interligar com o dispositivo. Utilizando um comando da PlayStation 3 e um BITalino, o dispositivo é capaz de fazer a aquisição do sinal PPG e sinal EDA durante o jogo. O software analisa os sinais do comando, calcula o ritmo cardíaco e mede os níveis de excitação em tempo real. Foi realizada uma experiência utilizando biofeedback positivo e negativo, com o objectivo de testar a integração entre o software e o hardware. Não será no imediato que os dispositivos deste género sejam disponibilizados comercialmente. Os resultados são, no entanto, promissores. O cálculo do ritmo cardíaco em tempo real tem apenas uma diferença de 5 batimentos por minuto em relação ao ritmo cardíaco real do jogador. Apesar de os testes com o EDA serem inconclusivos, pode-se verificar que foi possível construir um sistema para ler os dados fisiológicos sendo mais económico do que os seus pares, sem comprometer a fiabilidade dos dados.Affective games are a genre of games that use the physiological responses from the player to adapt the gameplay to a more enjoyable emotional state and experience. Physiological responses and affective games have been studied vastly over the years. However, the setups used in these interventions are very intrusive and are complex to set up. In this project, it is purposed to build a non-invasive and easy-to-set-up toolkit that records physiological data. This toolkit records the player's heart rate and arousal levels and was decomposed into software and hardware. Using a PS3 game controller replica and a BITalino, a physiological game controller which can record heart rate and arousal during gameplay was built. The software interfaces with the gamepad, processes the physiological signals and sends this information to the game. An experiment with a positive biofeedback condition and negative biofeedback condition was conducted. This experiment showed that even though more work must be done until these type of devices could be commercially available, the results are promising. This toolkit’s heart rate values, when compared with other more traditional acquisition devices, were very similar, being on average only 5 BMP lower than the actual heart rate, proving that is possible to build more affordable non-invasive physiological hardware without compromising the signal's accuracy

Facial Electromyography-based Adaptive Virtual Reality Gaming for Cognitive Training

Cognitive training has shown promising results for delivering improvements in human cognition related to attention, problem solving, reading comprehension and information retrieval. However, two frequently cited problems in cognitive training literature are a lack of user engagement with the training programme, and a failure of developed skills to generalise to daily life. This paper introduces a new cognitive training (CT) paradigm designed to address these two limitations by combining the benefits of gamification, virtual reality (VR), and affective adaptation in the development of an engaging, ecologically valid, CT task. Additionally, it incorporates facial electromyography (EMG) as a means of determining user affect while engaged in the CT task. This information is then utilised to dynamically adjust the game's difficulty in real-time as users play, with the aim of leading them into a state of flow. Affect recognition rates of 64.1% and 76.2%, for valence and arousal respectively, were achieved by classifying a DWT-Haar approximation of the input signal using kNN. The affect-aware VR cognitive training intervention was then evaluated with a control group of older adults. The results obtained substantiate the notion that adaptation techniques can lead to greater feelings of competence and a more appropriate challenge of the user's skills

Anxiety reducing through a neurofeedback serious game with dynamic difficulty adjustment

Presently, society has to deal with a large number of mental issues. Anxiety disorder is a serious concern, affecting millions of people’s lives and, although methods to tackle the problem currently exist, these main treatments are being linked to some issues and improvements must be found. One of the alternatives is Neurofeedback, a biofeedback treatment, completely non-invasive and showing impressive results so far. It uses a neuroheadset equipment to read the neural activity of the brain, giving the user visual feedback about it. The purpose this, is to train the users’ brain in specific regions and frequencies, allowing the subjects to learn how to voluntarily control its neural activity, even outside of the session. Current applications using this method might be too simple, which can become tedious and disengaging. Serious games can help with these issues, since it can bring enjoyment and engagement while doing this type of treatment. The interest in games’ capabilities in education has been increasing over the past years, since it has been proved that games are an excellent tool for education and skill learning. Joining these concepts of game and neurofeedback, this project aims to create a serious game prototype, applying the current treatment knowledge. The development process of a new game with neuroheadset integration, capable of reading the neural activity of the user while playing and giving the appropriate feedback, will be described in the present document. Since studies proved that a good balance between challenge and skill increases the learning performance, a dynamic difficulty adjustment system is implemented within the game, allowing the game to adapt itself to each user’s skill individually, and keeping the user in a challenging, motivating zone. At the end of the document, the results of pilot test on a few subjects are shown.Na sociedade actual o número de problemas relacionados com perturbações mentais tem sido cada vez mais relevante, sendo esse o caso da ansiedade. O distúrbio de ansiedade é um problema que atinge milhões de pessoas e, embora existam métodos para combater este problema, estudos comprovam que estes têm algumas lacunas que podem trazer outros problemas associados, sendo portanto necessário procurar melhorias aos métodos actuais. Uma das alternativas tem apresentado excelentes resultados e denomina-se Neurofeedback. Este é um tratamento de biofeedback, nãoinvasivo e que utiliza um equipamento neuroheadset para capturar a actividade neuronal, apresentando indicações visuais sobre o comportamento do utilizador. Isto é feito com o objectivo de treinar o cérebro do utilizador, em regiões e frequências específicas, para que este seja capaz de controlar voluntariamente a sua actividade neuronal. As aplicações actualmente utilizadas com este intuito podem se tornar aborrecidas e monótonas devido à sua simplicidade. Um jogo sério pode ajudar com estes problemas, uma vez que é capaz de trazer divertimento e motivação para este tipo de tratamento. O crescente interesse nas capacidades educativas dos jogos sérios, tem identificado estes como excelentes ferramentas para a educação. Este projecto pretende portanto criar um protótipo de um jogo sério, aplicando os conceitos de neurofeedback. Neste documento, é apresentado o processo de desenvolvimento de um novo jogo com integração de um neuroheadset, capaz de identificar a actividade neuronal do jogador dando respostas adequadas. Uma vez que estudos comprovam que um bom balanço entre desafio apresentado e técnica do utilizador aumenta a capacidade de aprendizagem, foi implementado também um sistema de ajuste de dificuldade dinâmica, permitindo uma adaptação do jogo a cada indivíduo e mantendo este numa zona motivante de equilíbrio entre desafio e proficiência. No final serão apresentados os resultados de um teste piloto efectuado em alguns indivíduos

Enhancing video game performance through an individualized biocybernetic system

Biocybernetic systems are physiological software systems that explicitly utilize physiological signals to control or adapt software functionality (Pope et al., 1995.) These systems have tremendous potential for innovation in human computer interaction by using physiological signals to infer a user\u27s emotional and mental states (Allanson & Fairclough, 2004; Fairclough, 2008). Nevertheless, development of these systems has been ultimately hindered by two fundamental challenges. First, these systems make generalizations about physiological indicators of cognitive states across populations when, in fact, relationships between physiological responses and cognitive states are specific to each individual (Andreassi, 2006). Second, they often employ largely inconsistent retrospective techniques to subjectively infer user\u27s mental state (Fairclough, 2008). An individualized biocybernetic system was developed to address the fundamental challenges of biocybernetic research. This system was used to adapt video game difficulty through real-time classifications of physiological responses to subjective appraisals. A study was conducted to determine the system\u27s ability to improve player\u27s performance. The results provide evidence of significant task performance increase and higher attained task difficulty when players interacted with the game using the system than without. This work offers researchers with an alternative method for software adaptation by conforming to the individual characteristics of each user