Feedback-Based Gameplay Metrics and Gameplay Performance Segmentation: An audio-visual approach for assessing player experience.

Gameplay metrics is a method and approach that is growing in popularity amongst the game studies research community for its capacity to assess players’ engagement with game systems. Yet, little has been done, to date, to quantify players’ responses to feedback employed by games that conveys information to players, i.e., their audio-visual streams. The present thesis introduces a novel approach to player experience assessment - termed feedback-based gameplay metrics - which seeks to gather gameplay metrics from the audio-visual feedback streams presented to the player during play. So far, gameplay metrics - quantitative data about a game state and the player's interaction with the game system - are directly logged via the game's source code. The need to utilise source code restricts the range of games that researchers can analyse. By using computer science algorithms for audio-visual processing, yet to be employed for processing gameplay footage, the present thesis seeks to extract similar metrics through the audio-visual streams, thus circumventing the need for access to, whilst also proposing a method that focuses on describing the way gameplay information is broadcast to the player during play. In order to operationalise feedback-based gameplay metrics, the present thesis introduces the concept of gameplay performance segmentation which describes how coherent segments of play can be identified and extracted from lengthy game play sessions. Moreover, in order to both contextualise the method for processing metrics and provide a conceptual framework for analysing the results of a feedback-based gameplay metric segmentation, a multi-layered architecture based on five gameplay concepts (system, game world instance, spatial-temporal, degree of freedom and interaction) is also introduced. Finally, based on data gathered from game play sessions with participants, the present thesis discusses the validity of feedback-based gameplay metrics, gameplay performance segmentation and the multi-layered architecture. A software system has also been specifically developed to produce gameplay summaries based on feedback-based gameplay metrics, and examples of summaries (based on several games) are presented and analysed. The present thesis also demonstrates that feedback-based gameplay metrics can be conjointly analysed with other forms of data (such as biometry) in order to build a more complete picture of game play experience. Feedback based game-play metrics constitutes a post-processing approach that allows the researcher or analyst to explore the data however they wish and as many times as they wish. The method is also able to process any audio-visual file, and can therefore process material from a range of audio-visual sources. This novel methodology brings together game studies and computer sciences by extending the range of games that can now be researched but also to provide a viable solution accounting for the exact way players experience games

Capturing and Scaffolding the Complexities of Self-Regulation During Game-Based Learning

Game-based learning environments (GBLEs) can offer students with engaging interactive instructional materials while also providing a research platform to investigate the dynamics and intricacies of effective self-regulated learning (SRL). Past research has indicated learners are often unable to monitor and regulate their cognitive and metacognitive processes within GBLEs accurately and effectively on their own due mostly to the open-ended nature of these environments. The future design and development of GBLEs and embedded scaffolds, therefore, require a better understanding of the discrepancies between the affordances of GBLEs and the required use of SRL. Specifically, how to incorporate interdisciplinary theories and concepts outside of traditional educational, learning, and psychological sciences literature, how to utilize process data to measure SRL processes during interactions with instructional materials accounting for the dynamics of leaners\u27 SRL, and how to improve SRL-driven scaffolds to be individualized and adaptive based on the level of agency GBLEs provide. Across four studies, this dissertation investigates learners\u27 SRL while they learn about microbiology using CRYSTAL ISLAND, a GBLE, building upon each other by enhancing the type of data collected, analytical methodologies used, and applied theoretical models and theories. Specifically, this dissertation utilizes a combination of traditional statistical approaches (i.e., linear regression models), non-linear statistical approaches (i.e., growth modeling), and non-linear dynamical theory (NDST) approaches (aRQA) with process trace data to contribute to the field\u27s current understanding of the dynamics and complexities of SRL. Furthermore, this dissertation examines how limited agency can act as an implicit scaffold during game-based learning to promote the use of SRL processes and increase learning outcomes

"This isn't what war is like" : an ethnographic account of ArmA3

This thesis examines the social practices of the Armed Assault 3 (ArmA 3) gaming community and their attempts to recreate a realistic combat experience online. Using an ethnographic approach, I explore the numerous military simulation (milsim) gaming practices employed by the community, many of which relied heavily on modeling and simulations processes. I contend that these practices were a response to the ‘gaps’ between the ‘real’ and the ‘virtual,’ which disrupted the gaming community’s ability to achieve the desired combat experience. An analysis of these practices makes evident what was deemed necessary for a meaningful and realistic online experience by a diverse community, as well as the new layers of gaps produced by the gamers themselves.Social Science and Humanities Research Counci

Evaluating scaffolding in serious games with children

In scaffolding, full support (guidance) is given to the learner to support weakness and withdrawn bit by bit as learner knowledge fortifies (fading) (Martens & Maciuszek, 2013) . According to Puntambekar & Hubscher (2005), the attributes of scaffolding include diagnosis, calibration and fading. Research work on scaffolding in serious games – games with other purposes other than entertainment, has mainly focused on diagnosis and calibration often referred to in this field as player modelling and adaptivity respectively. There is barely any empirical study investigating fading this in these games. Instead of fading which is the gradual removal of scaffolding, an all-or-nothing approach is often used. The all-or-nothing could lead to cognitive overload in children. For children to have a pleasurable gameplay, it is important the cognitive load is managed effectively. The fundamental question asked in this thesis is “To what extent can scaffolding-fading improve children’s gameplay experience and knowledge gain?” This is broken down to four research questions – 1. Does the gradual removal of guidance improve children’s gameplay experience? 2. What dimensions of gameplay experience are impacted and to what extent are they impacted by the gradual removal of guidance? 3. Would guidance fading during gameplay improve knowledge gain? 4. What effect would inappropriate guidance-fading have on gameplay? A game in which the scaffolding can be manipulated is designed for this study. A comparative study methodology with a controlled experiment, comparing gameplay in both the gradual removal and the all-or-nothing mode, is employed with the aim of measuring gameplay experience and knowledge gain in these modes. Analytics was also employed to capture performance-related gameplay metrics. These methods were combined for a more substantial explanation of findings. The key contributions made include – 1. Appropriately implementing guidance-fading for the first time in a game AND highlighting the relevance of this scaffolding mode to serious gamepla

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

A Comparative Study of in-Game and out-Game Assessment for Storyline-based Games

Serious games have the potential to complement existing teaching methods by motivating and providing a more enjoyable experience for the players or by simulating events that would be otherwise difficult to reproduce in the classroom. Despite their potential, little is known about how the games could be used not only for teaching but also as assessment tools. This research addresses this gap. We present an in-game assessment method which assesses the learning objectives included in the game without the need for a separate intervention. We evaluate the proposed method and we show that there is no statistically significant difference in participants being assessed through a questionnaire outside the game and the integrated game assessment method. Moreover, we looked at whether the player experience has been affected by the changes needed in the game design and the players' preferences for different types of assessment. Most participants preferred being assessed through the game. They also felt that the assessment has overall improved their game experience

Conceptual Framework for Designing Virtual Field Trip Games

This thesis aimed to provide designing models to explore an alternative solution for a field trip when it becomes impossible for several reasons such as the limitation of cost and time. Virtual field trip games are relatively new means to create virtual field trips in game environments through adding game aspects to learning aspects to enhance the learning experience. The simple combining of game and learning aspects will not guarantee the desired effect of virtual field trips. Theoretical and logical connections should be established to form interweave between both aspects. This thesis proposes a designing framework by establishing three links between game design aspects and learning aspects. The three links are constructed by modelling: the experiential learning theory (ELT), the gameplay, and the game world. ELT modelling quantifies the theory into the internal economy mechanic and balances the levels of game task difficulty with the player’s ability through game machinations, game modelling links the learning process to gameplay, and world modelling connects field environment to game environment. The internal economy mechanic and its components (resources, internal mechanic, feedback loop), formulating equations to define generic player’s interactions and identify indicators to capture evidence of achievements via a mathematical (evaluation) model. The game modelling includes skill models to design two important high-order skills (decision-making and teamwork) and connects them to the evaluation model. The game world is modelled through defining its variables and relationships’ rules to connect both environments (game and field) expanding the evaluation model. The framework is supported by essential learning theories (ELT, task-based learning, some aspects of social learning) and pedagogical aspects (assessment, feedback, field-based structure, high-order skills) and connected to the key game elements (interaction, multimodal presentation, control of choice…etc) of field-based learning along with suitable game mechanics. The two research studies that were conducted as part of this thesis found that the designing framework is useful, usable, and provides connections between learning and game aspects and the designed VFTG based on the framework improved learning performance along with providing motivation and presence. This suggests the effectiveness of the framework

12th Annual Undergraduate Student Symposium

The Undergraduate Student Symposium, sponsored by the Farquhar College of Arts and Sciences, presents student projects through presentations, papers, and poster displays. The event serves as a “showcase” demonstrating the outstanding scholarship of undergraduate students at NSU. The Symposium is open to undergraduate students from all disciplines. Projects cover areas of student scholarship ranging from the experimental and the applied to the computational, theoretical, artistic, and literary. They are taken from class assignments as well as from independent projects. The projects do not have to be complete; presentations can represent any stage in the concept’s evolution, from proposal and literature review to fully completed and realized scholarly work. As in past symposia, the definition of scholarship will be sufficiently broad to include work presented in the biological and physical sciences, the social and behavioral sciences, computer science, mathematics, arts and humanities, education, and business. This is the eleventh annual Undergraduate Student Symposium

Study and experimentation of cognitive decline measurements in a virtual reality environment

À l’heure où le numérique s’est totalement imposé dans notre quotidien, nous pouvons nous demander comment évolue notre bien-être. La réalité virtuelle hautement immersive permet de développer des environnements propices à la relaxation qui peuvent améliorer les capacités cognitives et la qualité de vie de nombreuses personnes. Le premier objectif de cette étude est de réduire les émotions négatives et améliorer les capacités cognitives des personnes souffrant de déclin cognitif subjectif (DCS). À cette fin, nous avons développé un environnement de réalité virtuelle appelé Savannah VR, où les participants ont suivi un avatar à travers une savane. Nous avons recruté dix-neuf personnes atteintes de DCS pour participer à l’expérience virtuelle de la savane. Le casque Emotiv Epoc a capturé les émotions des participants pendant toute l’expérience virtuelle. Les résultats montrent que l’immersion dans la savane virtuelle a réduit les émotions négatives des participants et que les effets positifs ont continué par la suite. Les participants ont également amélioré leur performance cognitive. La confusion se manifeste souvent au cours de l’apprentissage lorsque les élèves ne comprennent pas de nouvelles connaissances. C’est un état qui est également très présent chez les personnes atteintes de démence à cause du déclin de leurs capacités cognitives. Détecter et surmonter la confusion pourrait ainsi améliorer le bien-être et les performances cognitives des personnes atteintes de troubles cognitifs. Le deuxième objectif de ce mémoire est donc de développer un outil pour détecter la confusion. Nous avons mené deux expérimentations et obtenu un modèle d’apprentissage automatique basé sur les signaux du cerveau pour reconnaître quatre niveaux de confusion (90% de précision). De plus, nous avons créé un autre modèle pour reconnaître la fonction cognitive liée à la confusion (82 % de précision).At a time when digital technology has become an integral part of our daily lives, we can ask ourselves how our well-being is evolving. Highly immersive virtual reality allows the development of environments that promote relaxation and can improve the cognitive abilities and quality of life of many people. The first aim of this study is to reduce the negative emotions and improve the cognitive abilities of people suffering from subjective cognitive decline (SCD). To this end, we have developed a virtual reality environment called Savannah VR, where participants followed an avatar across a savannah. We recruited nineteen people with SCD to participate in the virtual savannah experience. The Emotiv Epoc headset captured their emotions for the entire virtual experience. The results show that immersion in the virtual savannah reduced the negative emotions of the participants and that the positive effects continued afterward. Participants also improved their cognitive performance. Confusion often occurs during learning when students do not understand new knowledge. It is a state that is also very present in people with dementia because of the decline in their cognitive abilities. Detecting and overcoming confusion could thus improve the well-being and cognitive performance of people with cognitive impairment. The second objective of this paper is, therefore, to develop a tool to detect confusion. We conducted two experiments and obtained a machine learning model based on brain signals to recognize four levels of confusion (90% accuracy). In addition, we created another model to recognize the cognitive function related to the confusion (82% accuracy)

From multiscale modeling to metamodeling of geomechanics problems

In numerical simulations of geomechanics problems, a grand challenge consists of overcoming the difficulties in making accurate and robust predictions by revealing the true mechanisms in particle interactions, fluid flow inside pore spaces, and hydromechanical coupling effect between the solid and fluid constituents, from microscale to mesoscale, and to macroscale. While simulation tools incorporating subscale physics can provide detailed insights and accurate material properties to macroscale simulations via computational homogenizations, these numerical simulations are often too computational demanding to be directly used across multiple scales. Recent breakthroughs of Artificial Intelligence (AI) via machine learning have great potential to overcome these barriers, as evidenced by their great success in many applications such as image recognition, natural language processing, and strategy exploration in games. The AI can achieve super-human performance level in a large number of applications, and accomplish tasks that were thought to be not feasible due to the limitations of human and previous computer algorithms. Yet, machine learning approaches can also suffer from overfitting, lack of interpretability, and lack of reliability. Thus the application of machine learning into generation of accurate and reliable surrogate constitutive models for geomaterials with multiscale and multiphysics is not trivial. For this purpose, we propose to establish an integrated modeling process for automatic designing, training, validating, and falsifying of constitutive models, or "metamodeling". This dissertation focuses on our efforts in laying down step-by-step the necessary theoretical and technical foundations for the multiscale metamodeling framework. The first step is to develop multiscale hydromechanical homogenization frameworks for both bulk granular materials and granular interfaces, with their behaviors homogenized from subscale microstructural simulations. For efficient simulations of field-scale geomechanics problems across more than two scales, we develop a hybrid data-driven method designed to capture the multiscale hydro-mechanical coupling effect of porous media with pores of various different sizes. By using sub-scale simulations to generate database to train material models, an offline homogenization procedure is used to replace the up-scaling procedure to generate path-dependent cohesive laws for localized physical discontinuities at both grain and specimen scales. To enable AI in taking over the trial-and-error tasks in the constitutive modeling process, we introduce a novel “metamodeling” framework that employs both graph theory and deep reinforcement learning (DRL) to generate accurate, physics compatible and interpretable surrogate machine learning models. The process of writing constitutive models is simplified as a sequence of forming graph edges with the goal of maximizing the model score (a function of accuracy, robustness and forward prediction quality). By using neural networks to estimate policies and state values, the computer agent is able to efficiently self-improve the constitutive models generated through self-playing. To overcome the obstacle of limited information in geomechanics, we improve the efficiency in utilization of experimental data by a multi-agent cooperative metamodeling framework to provide guidance on database generation and constitutive modeling at the same time. The modeler agent in the framework focuses on evaluating all modeling options (from domain experts’ knowledge or machine learning) in a directed multigraph of elasto-plasticity theory, and finding the optimal path that links the source of the directed graph (e.g., strain history) to the target (e.g., stress). Meanwhile, the data agent focuses on collecting data from real or virtual experiments, interacts with the modeler agent sequentially and generates the database for model calibration to optimize the prediction accuracy. Finally, we design a non-cooperative meta-modeling framework that focuses on automatically developing strategies that simultaneously generate experimental data to calibrate model parameters and explore weakness of a known constitutive model until the strengths and weaknesses of the constitutive law on the application range can be identified through competition. These tasks are enabled by a zero-sum reward system of the metamodeling game and robust adversarial reinforcement learning techniques