A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Gaze-directed gameplay in first person computer games

The use of eye tracking systems in computer games is still at an early stage. Commercial eye trackers and researches have been focusing in gaze-oriented gameplay as an alternative to traditional input devices. This dissertation proposes to investigate the advantages and disadvantages of the use of these systems in computer games. For it, instead of using eye tracking as a simple direct control input, it is proposed to use it in order to control the attention of the player’s avatar (e.g., if the player notices an obstacle in the way, the avatar will notice it too and avoid it) and the game’s procedural content generation (e.g., spawn obstacles in the opposite side of the screen to where the player’s attention is focused). To demonstrate the value of this proposal, it was developed and is herein presented the first-person shooter "Zombie Runner". Tests showed that the implementation meets the stipulated technical requirements and that, although it still needs improvements in terms of precision and robustness, eye tracking technology can be successfully used to to make the player experience more immersive and challenging.A utilização de sistemas de rastreamento ocular em jogos de computador ainda se encontra numa fase embrionária. Aparelhos de rastreamento ocular comerciais e pesquisas na área têm-se focado em jogabilidade à base da atenção visual como uma alternativa a métodos de entrada tradicionais. Esta dissertação propõe-se a investigar as vantages e desvantagens do uso destes sistemas em jogos de computador. Para isso, invés de se usar rastreamento ocular apenas como um método directo de entrada, é proposto usá-lo para controlar a atenção do personagem do jogo (e.g., se o jogador reparar num obstáculo, a personagem também repara e desvia-se do mesmo) assim como afectar a geração procedimental do jogo (e.g., gerar obstáculos no lado oposto ao qual o jogador tem a sua atenção focada). Para demonstrar o valor desta proposta, foi desenvolvido e aqui apresentado o jogo de tiros em primeira pessoa "Zombie Runner". Os testes demonstraram que a implementação cumpre os requisitos técnicos estipulados e que, apesar de ainda carecer de melhorias em termos de precisão e robustez, a tecnologia para rastreamento ocular pode ser utilizada com sucesso para tornar a experiência do jogador mais imersiva e desafiante

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

The Use of Learning Analytics Interactive Dashboards in Serious Games: A Review of the Literature

The learning analytics in serious games, corresponds to a subject in increasing demand in the educational field. In this context, there is a need to study how data visualizations found in the literature are adopted in learning analytics in serious games. This paper presents a Systematic Literature Review (SLR) on how the evolution of studies associated with the use of learning analytics interactive dashboards in serious games is processed, seeking to investigate the characteristics of using dashboards for viewing educational data. A bibliometric analysis was carried out in which 75 relevant studies were selected from the Scopus, Web of Science, and IEEExplore databases. From the data analysis, it was observed that in the current literature there is a reduced number of studies containing the main actors in the learning process, as follows: teachers/instructors, students/participants, game developers/designers, and managers/researchers. In the vast majority of investigated studies, data visualization algorithms are used, where the main focus takes into account only actors, such as teachers/instructors and students/participants

A interação jogador e videojogo na construção da experiência de jogo

Doutoramento em Informação e Comunicação em Plataformas DigitaisAmong the many discussions and studies related to video games, one of the most recurrent, widely debated and important relates to the experience of playing video games. The gameplay experience – as appropriated in this study – is the result of the interplay between two essential elements: a video game and a player. Existing studies have explored the resulting experience of video game playing from the perspective of the video game or the player, but none appear to equally balance both of these elements. The study presented here contributes to the ongoing debate with a gameplay experience model. The proposed model, which looks to equally balance the video game and the player elements, considers the gameplay experience to be both an interactive experience (related to the process of playing the video game) and an emotional experience (related to the outcome of playing the video game). The mutual influence of these two experiences during video game play ultimately defines the gameplay experience. To this gameplay experience contributes several dimensions, related to both the video game and player: the video game includes a mechanics, interface and narrative dimension; the player includes a motivations, expectations and background dimension. Also, the gameplay experience is initially defined by a gameplay situation, conditioned by an ambient in which gameplay takes place and a platform on which the video game is played. In order to initially validate the proposed model and attempt to show a relationship among the multiple model dimensions, a multi-case study was carried out using two different video games and player samples. In one study, results show significant correlations between multiple model dimensions, and evidence that video game related changes influence player motivations as well as player visual behavior. In specific player related analysis, results show that while players may be different in terms of background and expectations regarding the game, their motivation to play are not necessarily different, even if their performance in the game is weak. While further validation is necessary, this model not only contributes to the gameplay experience debate, but also demonstrates in a given context how player and video game dimensions evolve during video game play.Entre as muitas discussões e estudos relacionados com os videojogos, um dos mais recorrentes, amplamente debatido e importante relaciona-se com a experiência de jogar videojogos. A experiência de jogo - como empregado neste estudo - é o resultado da interação entre dois elementos essenciais: um videojogo e um jogador. Os estudos existentes têm explorado a experiência resultante do ato de jogar a partir da perspectiva do videojogo ou do jogador, mas nenhum parece igualmente equilibrar estes dois elementos. O estudo aqui apresentado contribui para o debate em curso com um modelo da experiência de jogo. O modelo proposto, que procura equilibrar de forma igual os elementos videojogo e jogador, considera a experiência de jogo como uma experiência interativa (relacionada com o processo de jogar o videojogo) e uma experiência emocional (relacionada com o resultado de jogar o videojogo). A influência mútua destas duas experiências durante o ato de jogar define a experiência de jogo. Para esta experiência de jogo contribuem várias dimensões, relacionadas com o videojogo e o jogador: o videojogo inclui a dimensão da mecânica, da interface e narrativa; o jogador inclui a dimensão das motivações, expectativas e background. Além disso, a experiência de jogo é inicialmente definida por uma situação de jogo, condicionada por um ambiente em que o jogo se realiza e uma plataforma na qual se joga. Para inicialmente validar o modelo e tentar mostrar uma relação entre as múltiplas dimensões do modelo proposto, um estudo multicaso foi concretizado utilizando dois videojogos e amostras diferentes. Num dos estudos, os resultados mostram correlações significativas entre as múltiplas dimensões do modelo, e evidências de que alterações ao videojogo podem influenciar as motivações do jogador e o seu comportamento visual. Numa análise relacionada com características dos jogadores, os resultados mostram que os jogadores, embora possam ser diferentes em termos de experiência e expectativas em relação ao jogo, a sua motivação para jogar não é necessariamente diferente, mesmo que o seu desempenho no jogo seja fraco. Embora uma validação contínua do modelo seja necessária, este modelo não só contribui para o debate da experiência de jogo, mas também mostra num determinado contexto como as dimensões do jogador e videojogo evoluem durante o ato de jogar

Customizing Experiences for Mobile Virtual Reality

A criação manual de conteúdo para um jogo é um processo demorado e trabalhoso que requer um conjunto de habilidades diversi cado (normalmente designers, artistas e programadores) e a gestão de diferentes recursos (hardware e software especializados). Dado que o orçamento, tempo e recursos são frequentemente muito limitados, os projetos poderiam bene ciar de uma solução que permitisse poupar e investir noutros aspectos do desenvolvimento. No contexto desta tese, abordamos este desa o sugerindo a criação de pacotes especí cos para a geração de conteúdo per sonalizável, focados em aplicações de Realidade Virtual (RV) móveis. Esta abordagem divide o problema numa solução com duas facetas: em primeiro lugar, a Geração Procedural de Conteúdo, alcançada através de métodos convencionais e pela utilização inovadora de Grandes Modelos de Lin guagem (normalmente conhecidos por Large Language Models). Em segundo lugar, a Co-Criação de Conteúdo, que enfatiza o desenvolvimento colaborativo de conteúdo. Adicionalmente, dado que este trabalho se foca na compatibilidade com RV móvel, as limitações de hardware associadas a capacetes de RV autónomos (standalone VR Headsets) e formas de as ultrapassar são também abordadas. O conteúdo será gerado utilizando métodos actuais em geração procedural e facilitando a co-criação de conteúdo pelo utilizador. A utilização de ambas estas abordagens resulta em ambi entes, objectivos e conteúdo geral mais re-jogáveis com muito menos desenho. Esta abordagem está actualmente a ser aplicada no desenvolvimento de duas aplicações de RV distintas. A primeira, AViR, destina-se a oferecer apoio psicológico a indivíduos após a perda de uma gravidez. A se gunda, EmotionalVRSystem, visa medir as variações nas respostas emocionais dos participantes induzidas por alterações no ambiente, utilizando tecnologia EEG para leituras precisas

Serious Games and Mixed Reality Applications for Healthcare

Virtual reality (VR) and augmented reality (AR) have long histories in the healthcare sector, offering the opportunity to develop a wide range of tools and applications aimed at improving the quality of care and efficiency of services for professionals and patients alike. The best-known examples of VR–AR applications in the healthcare domain include surgical planning and medical training by means of simulation technologies. Techniques used in surgical simulation have also been applied to cognitive and motor rehabilitation, pain management, and patient and professional education. Serious games are ones in which the main goal is not entertainment, but a crucial purpose, ranging from the acquisition of knowledge to interactive training.These games are attracting growing attention in healthcare because of their several benefits: motivation, interactivity, adaptation to user competence level, flexibility in time, repeatability, and continuous feedback. Recently, healthcare has also become one of the biggest adopters of mixed reality (MR), which merges real and virtual content to generate novel environments, where physical and digital objects not only coexist, but are also capable of interacting with each other in real time, encompassing both VR and AR applications.This Special Issue aims to gather and publish original scientific contributions exploring opportunities and addressing challenges in both the theoretical and applied aspects of VR–AR and MR applications in healthcare

Exploiting physiological changes during the flow experience for assessing virtual-reality game design.

Immersive experiences are considered the principal attraction of video games. Achieving a healthy balance between the game's demands and the user's skills is a particularly challenging goal. However, it is a coveted outcome, as it gives rise to the flow experience – a mental state of deep concentration and game engagement. When this balance fractures, the player may experience considerable disinclination to continue playing, which may be a product of anxiety or boredom. Thus, being able to predict manifestations of these psychological states in video game players is essential for understanding player motivation and designing better games. To this end, we build on earlier work to evaluate flow dynamics from a physiological perspective using a custom video game. Although advancements in this area are growing, there has been little consideration given to the interpersonal characteristics that may influence the expression of the flow experience. In this thesis, two angles are introduced that remain poorly understood. First, the investigation is contextualized in the virtual reality domain, a technology that putatively amplifies affective experiences, yet is still insufficiently addressed in the flow literature. Second, a novel analysis setup is proposed, whereby the recorded physiological responses and psychometric self-ratings are combined to assess the effectiveness of our game's design in a series of experiments. The analysis workflow employed heart rate and eye blink variability, and electroencephalography (EEG) as objective assessment measures of the game's impact, and self-reports as subjective assessment measures. These inputs were submitted to a clustering method, cross-referencing the membership of the observations with self-report ratings of the players they originated from. Next, this information was used to effectively inform specialized decoders of the flow state from the physiological responses. This approach successfully enabled classifiers to operate at high accuracy rates in all our studies. Furthermore, we addressed the compression of medium-resolution EEG sensors to a minimal set required to decode flow. Overall, our findings suggest that the approaches employed in this thesis have wide applicability and potential for improving game designing practices