A framework for the manipulation of video game elements using the player's biometric data

A seguinte dissertação foca-se na utilização dos vários sinais biométricos produzidos pelo ser humano (neste caso específico, os do jogador) para manipulação dos vários elementos existentes em um vídeo jogo. Estes elementos são constituídos pelas mecânicas de jogo, banda sonora, elementos visuais, inteligência artificial dos inimigos e personagens não jogáveis, diálogos da história e sequências da narrativa.Para tal, apresenta-se uma framework para o desenvolvimento de uma estrutura de suporte destinada a jogos que pretendam utilizar este tipo de mecanismo de interacção, bem como a implementação de um protótipo que utilize a mesma.The following dissertation focuses on the use of several biometric signals produced by the human being (in this specific case, the player) to manipulate several elements present in a game. These elements are composed by the game mechanics, soundtrack, visual assets, enemies and NPCs' artificial intelligence, dialogues and narrative sequences.To do such, this work presents a framework for the development of a supporting structure meant for games that wish to use this type of interaction mechanism as well as the implementation of a prototype using the said framework

Tune in to your emotions: a robust personalized affective music player

The emotional power of music is exploited in a personalized affective music player (AMP) that selects music for mood enhancement. A biosignal approach is used to measure listeners’ personal emotional reactions to their own music as input for affective user models. Regression and kernel density estimation are applied to model the physiological changes the music elicits. Using these models, personalized music selections based on an affective goal state can be made. The AMP was validated in real-world trials over the course of several weeks. Results show that our models can cope with noisy situations and handle large inter-individual differences in the music domain. The AMP augments music listening where its techniques enable automated affect guidance. Our approach provides valuable insights for affective computing and user modeling, for which the AMP is a suitable carrier application

Autonomous Assessment of Videogame Difficulty Using Physiological Signals

Given the well-explored relation between challenge and involvement in a task, (e.g., as described in Csikszentmihalyi’s theory of flow), it could be argued that the presence of challenge in videogames is a core element that shapes player experiences and should, therefore, be matched to the player’s skills and attitude towards the game. However, handling videogame difficulty, is a challenging problem in game design, as too easy a task can lead to boredom and too hard can lead to frustration. Thus, by exploring the relationship between difficulty and emotion, the current work intends to propose an artificial intelligence model that autonomously predicts difficulty according to the set of emotions elicited in the player. To test the validity of this approach, we developed a simple puzzle-based Virtual Reality (VR) videogame, based on the Trail Making Test (TMT), and whose objective was to elicit different emotions according to three levels of difficulty. A study was carried out in which physiological responses as well as player self- reports were collected during gameplay. Statistical analysis of the self-reports showed that different levels of experience with either VR or videogames didn’t have a measurable impact on how players performed during the three levels. Additionally, the self-assessed emotional ratings indicated that playing the game at different difficulty levels gave rise to different emotional states. Next, classification using a Support Vector Machine (SVM) was performed to verify if it was possible to detect difficulty considering the physiological responses associated with the elicited emotions. Results report an overall F1-score of 68% in detecting the three levels of difficulty, which verifies the effectiveness of the adopted methodology and encourages further research with a larger dataset.Dada a relação bem explorada entre desafio e envolvimento numa tarefa (p. ex., con- forme descrito na teoria do fluxo de Csikszentmihalyi), pode-se argumentar que a pre- sença de desafio em videojogos é um elemento central que molda a experiência do jogador e deve, portanto, ser compatível com as habilidades e a atitude que jogador exibe perante o jogo. No entanto, saber como lidar com a dificuldade de um videojogo é um problema desafiante no design de jogos, pois uma tarefa muito fácil pode gerar tédio e muito di- fícil pode levar à frustração. Assim, ao explorar a relação entre dificuldade e emoção, o presente trabalho pretende propor um modelo de inteligência artificial que preveja de forma autônoma a dificuldade de acordo com o conjunto de emoções elicitadas no jogador. Para testar a validade desta abordagem, desenvolveu-se um jogo de puzzle em Realidade Virtual (RV), baseado no Trail Making Test (TMT), e cujo objetivo era elicitar diferentes emoções tendo em conta três níveis de dificuldade. Foi realizado um estudo no qual se recolheram as respostas fisiológicas, juntamente com os autorrelatos dos jogado- res, durante o jogo. A análise estatística dos autorelatos mostrou que diferentes níveis de experiência com RV ou videojogos não tiveram um impacto mensurável no desempenho dos jogadores durante os três níveis. Além disso, as respostas emocionais auto-avaliadas indicaram que jogar o jogo em diferentes níveis de dificuldade deu origem a diferentes estados emocionais. Em seguida, foi realizada a classificação por intermédio de uma Má- quina de Vetores de Suporte (SVM) para verificar se era possível detectar dificuldade, considerando as respostas fisiológicas associadas às emoções elicitadas. Os resultados re- latam um F1-score geral de 68% na detecção dos três níveis de dificuldade, o que verifica a eficácia da metodologia adotada e incentiva novas pesquisas com um conjunto de dados maior

Affective Game Computing: A Survey

This paper surveys the current state of the art in affective computing principles, methods and tools as applied to games. We review this emerging field, namely affective game computing, through the lens of the four core phases of the affective loop: game affect elicitation, game affect sensing, game affect detection and game affect adaptation. In addition, we provide a taxonomy of terms, methods and approaches used across the four phases of the affective game loop and situate the field within this taxonomy. We continue with a comprehensive review of available affect data collection methods with regards to gaming interfaces, sensors, annotation protocols, and available corpora. The paper concludes with a discussion on the current limitations of affective game computing and our vision for the most promising future research directions in the field

Sports ingroup love does not make me like the sponsor’s beverage but gets me buying it

Previous literature has shown that social identity influences consumer decision-making towards branded products. However, its influence on ones' own sensory perception of an ingroup (or outgroup) associated brand's product (i.e. sponsor) is seldom documented and little understood. Here, we investigate the impact of social identity (i.e. team identification) with a football team on the sensorial experience and willingness to buy a beverage, said to be sponsoring the ingroup or the outgroup team. Ninety subjects participated in one of three sensorial experience conditions (matched identity: ingroup beverage; mismatched identity: outgroup beverage; control: no group preference). Each participant tasted the new sponsoring beverage and answered a questionnaire about their subjective sensorial experience of the beverage. EEG and BVP were synchronously collected throughout. Analyses revealed that team identification does not influence subjective responses and only slightly modulates physiological signals. All participants reported high valence and arousal values while physiological signals consistently translated negative affects across groups, which showed that participants reported to be happy/excited about trying the beverage while their physiological signals showed that they were feeling sad/depressed/angry. Crucially, despite a similar sensorial experience, and similar socially desirable report of the subjective experience, only participants in the matched identity group demonstrate higher willingness to buy, showing that the level of team identification, but not taste or beverage quality, influences willingness to buy the said sponsor's product.info:eu-repo/semantics/publishedVersio

A Framework for Students Profile Detection

Some of the biggest problems tackling Higher Education Institutions are students’ drop-out and academic disengagement. Physical or psychological disabilities, social-economic or academic marginalization, and emotional and affective problems, are some of the factors that can lead to it. This problematic is worsened by the shortage of educational resources, that can bridge the communication gap between the faculty staff and the affective needs of these students. This dissertation focus in the development of a framework, capable of collecting analytic data, from an array of emotions, affects and behaviours, acquired either by human observations, like a teacher in a classroom or a psychologist, or by electronic sensors and automatic analysis software, such as eye tracking devices, emotion detection through facial expression recognition software, automatic gait and posture detection, and others. The framework establishes the guidance to compile the gathered data in an ontology, to enable the extraction of patterns outliers via machine learning, which assist the profiling of students in critical situations, like disengagement, attention deficit, drop-out, and other sociological issues. Consequently, it is possible to set real-time alerts when these profiles conditions are detected, so that appropriate experts could verify the situation and employ effective procedures. The goal is that, by providing insightful real-time cognitive data and facilitating the profiling of the students’ problems, a faster personalized response to help the student is enabled, allowing academic performance improvements

Establishing a Framework for the development of Multimodal Virtual Reality Interfaces with Applicability in Education and Clinical Practice

The development of Virtual Reality (VR) and Augmented Reality (AR) content with multiple sources of both input and output has led to countless contributions in a great many number of fields, among which medicine and education. Nevertheless, the actual process of integrating the existing VR/AR media and subsequently setting it to purpose is yet a highly scattered and esoteric undertaking. Moreover, seldom do the architectures that derive from such ventures comprise haptic feedback in their implementation, which in turn deprives users from relying on one of the paramount aspects of human interaction, their sense of touch. Determined to circumvent these issues, the present dissertation proposes a centralized albeit modularized framework that thus enables the conception of multimodal VR/AR applications in a novel and straightforward manner. In order to accomplish this, the aforesaid framework makes use of a stereoscopic VR Head Mounted Display (HMD) from Oculus Rift©, a hand tracking controller from Leap Motion©, a custom-made VR mount that allows for the assemblage of the two preceding peripherals and a wearable device of our own design. The latter is a glove that encompasses two core modules in its innings, one that is able to convey haptic feedback to its wearer and another that deals with the non-intrusive acquisition, processing and registering of his/her Electrocardiogram (ECG), Electromyogram (EMG) and Electrodermal Activity (EDA). The software elements of the aforementioned features were all interfaced through Unity3D©, a powerful game engine whose popularity in academic and scientific endeavors is evermore increasing. Upon completion of our system, it was time to substantiate our initial claim with thoroughly developed experiences that would attest to its worth. With this premise in mind, we devised a comprehensive repository of interfaces, amid which three merit special consideration: Brain Connectivity Leap (BCL), Ode to Passive Haptic Learning (PHL) and a Surgical Simulator