The platformer experience dataset

Player modeling and estimation of player experience have become very active research fields within affective computing, human computer interaction, and game artificial intelligence in recent years. For advancing our knowledge and understanding on player experience this paper introduces the Platformer Experience Dataset (PED) - the first open-access game experience corpus - that contains multiple modalities of user data of Super Mario Bros players. The open-access database aims to be used for player experience capture through context-based (i.e. game content), behavioral and visual recordings of platform game players. In addition, the database contains demographical data of the players and self-reported annotations of experience in two forms: ratings and ranks. PED opens up the way to desktop and console games that use video from webcameras and visual sensors and offer possibilities for holistic player experience modeling approaches that can, in turn, yield richer game personalization.peer-reviewe

Predicting Player Engagement in Tom Clancy's The Division 2: A Multimodal Approach via Pixels and Gamepad Actions

This paper introduces a large scale multimodal corpus collected for the purpose of analysing and predicting player engagement in commercial-standard games. The corpus is solicited from 25 players of the action role-playing game Tom Clancy's The Division 2, who annotated their level of engagement using a time-continuous annotation tool. The cleaned and processed corpus presented in this paper consists of nearly 20 hours of annotated gameplay videos accompanied by logged gamepad actions. We report preliminary results on predicting long-term player engagement based on in-game footage and game controller actions using Convolutional Neural Network architectures. Results obtained suggest we can predict the player engagement with up to 72% accuracy on average (88% at best) when we fuse information from the game footage and the player's controller input. Our findings validate the hypothesis that long-term (i.e. 1 hour of play) engagement can be predicted efficiently solely from pixels and gamepad actions.Comment: 8 pages, accepted for publication and presentation at 2023 25th ACM International Conference on Multimodal Interaction (ICMI

General general game AI

Arguably the grand goal of artificial intelligence research is to produce machines with general intelligence: the capacity to solve multiple problems, not just one. Artificial intelligence (AI) has investigated the general intelligence capacity of machines within the domain of games more than any other domain given the ideal properties of games for that purpose: controlled yet interesting and computationally hard problems. This line of research, however, has so far focused solely on one specific way of which intelligence can be applied to games: playing them. In this paper, we build on the general game-playing paradigm and expand it to cater for all core AI tasks within a game design process. That includes general player experience and behavior modeling, general non-player character behavior, general AI-assisted tools, general level generation and complete game generation. The new scope for general general game AI beyond game-playing broadens the applicability and capacity of AI algorithms and our understanding of intelligence as tested in a creative domain that interweaves problem solving, art, and engineering.peer-reviewe

Deep Unsupervised Multi-View Detection of Video Game Stream Highlights

We consider the problem of automatic highlight-detection in video game streams. Currently, the vast majority of highlight-detection systems for games are triggered by the occurrence of hard-coded game events (e.g., score change, end-game), while most advanced tools and techniques are based on detection of highlights via visual analysis of game footage. We argue that in the context of game streaming, events that may constitute highlights are not only dependent on game footage, but also on social signals that are conveyed by the streamer during the play session (e.g., when interacting with viewers, or when commenting and reacting to the game). In this light, we present a multi-view unsupervised deep learning methodology for novelty-based highlight detection. The method jointly analyses both game footage and social signals such as the players facial expressions and speech, and shows promising results for generating highlights on streams of popular games such as Player Unknown's Battlegrounds

Analysis and application of rhythm in the design of 2D platformer levels

Abstract. The video game industry has grown quickly from its humble beginnings to one of the largest entertainment industries in the world. Fuelled by the continuous advancements in technology, the quality and quantity of content in AAA video games continues to rise along with customer expectations. But with the ever-higher ambitions, the development budgets and durations rise with them, making the cycle unsustainable on the long run. Procedural content generation is a technique that has the potential of helping break the cycle. The automatic generation of game content, such as levels, could help game developers reach the desired quantity of content with a fraction of the time and money required. However, commercial applications of procedural content generation so far have been largely limited in scope and lacking in quality, with the more successful cases being found in smaller budget indie games. In this study, the possibility to use the idea of rhythm in guiding procedural level generation towards better quality was studied. Using a design science research approach, the gameplay rhythm of original Super Mario Bros. levels was extracted and used to build a rhythm-based procedural 2D platformer level generator. The nature of the generated levels was investigated by computational metrics, and the quality of them was evaluated by a series of playtests. It was found that the existing platformer levels included an extractable rhythm. The rhythm-based level generator that was built upon the found rhythm data produced levels that were closely on par with the original levels, indicating that rhythm has potential applications in informing how a procedural content generator could create more meaningful and higher quality content. Finally, this experimental approach in incorporating music theory to procedural content generation opens up many interesting new avenues for future research

EMOTIONS RECOGNITION IN VIDEO GAME PLAYERS USING PHYSIOLOGICAL INFORMATION

Video games are interactive software able to arouse different kinds of emotions in players. Usually, the game designer tries to define a set of game features able to enjoy, engage, and/or educate the consumers. Through the gameplay, the narrative, and the game environment, a video game is able to interact with players' intellect and emotions. Thanks to the technological developments of the last years, the gaming industry has grown to become one of the most important entertainment markets. The scientific community and private companies have put a lot of efforts on the technical aspects as well as on the interaction aspects between the players and the video game. Considering the game design, many theories have been proposed to define some guidelines to design games able to arouse specific emotions in consumers. They mainly use interviews or observations in order to deduce the goodness of their approach through qualitative data. There are some works based on empirical studies aimed at studying the emotional states directly on players, using quantitative data. However, these researches usually consider the data analysis as a classification problem involving, mainly, the game events. Our goal is to understand how the feelings, experienced by the players, can be automatically deducted, and how these emotional states can be used to improve the game quality. In order to pursue this purpose, we have measured the mental states using physiological signals in order to return a set of quantitative values used to identify the players emotions. The most common ways to identify emotions are: to use a discrete set of labels (e.g., joy, anger), or to assess them inside an n-dimensional vector space. Albeit the most natural way to describe the emotions is to represent them through their name, the latter approach provides a quantitative result that can be used to define the new game status. In this thesis, we propose a framework aimed at an automatic assessment, using physiological data, of emotions in a 2-dimensional space, structured by valence and arousal vectors. The former may vary between pleasure and displeasure, while the latter defines the level of physiological activation. As a consequence, we have considered as most effective to infer the players\u2019 mental states, the following physiological data: electrocardiography (ECG), electromyography on 5 facial muscles (Facial EMG), galvanic skins response (GSR), and respiration intensity/rate. We have recorded a video, during a set of game sessions, of the player's face and of her gameplay. To acquire the affective information, we have shown the recorded video and audio to the player, and we have asked to self-assess her/his emotional state over the entire game on the valence and arousal vectors presented above. Starting from this framework, we have conducted two sets of experiments. In the first experiment, our aim was to validate the procedure. We have collected the data of 10 participants while playing at 4 platform games. We have also analyzed the data to identify the emotion pattern of the player during the gaming sessions. The analysis has been conducted in two directions: individual analysis (to find the physiological pattern of an individual player), and collective analysis (to find the generic patterns of the sample population). The goal of the second experiment has been to create a dataset of physiological information of 33 players, and to extend the data analysis and the results provided by the pilot study. We have asked the participants to play at 2 racing games in two different environments: on a standard monitor and using a head mounted display for Virtual Reality. After we have collected the information useful to the dataset creation, we have analyzed the data focusing on individual analysis. In both analyses, the self-assessment and the physiological data have been used in order to infer the emotional state of the players in each moment of the game sessions, and to build a prediction model of players' emotions using Machine Learning techniques. Therefore, the three main contributions of this thesis are: to design a novel framework for study the emotions of video game players, to develop an open-source architecture and a set of software able to acquire the physiological signals and the affective states, to create an affective dataset using racing video games as stimuli, to understand which physiological conditions could be the most relevant in order to determine the players' emotions, and to propose a method for the real-time prediction of a player's mental state during a video game session. The results to suggest that it is possible to design a model that fits with player's characteristics, predicting her emotions. It could be an effective tool available to game designers who can introduce innovative features to their games

What Does Bach Have in Common with World 1-1: Automatic Platformer Gestalt Analysis

Platformer level generation has often used a beat metaphor to relate to how players interact with level geometry. However, this conceptualization of beats is different from the musical concept of `beat', limiting the utility of theories and tools developed in music analysis for platformer levels. A gameplay gestalt, a pattern of interaction that the player enacts or performs in order to make progress in a game, may fit the beat metaphor. By taking a very similar lens and viewing players playing platformer levels as enacting a series of gameplay gestalts through time, gestalt music analysis (GMA) does fit into the platformer domain. This paper details work on transforming a GMA model to work with the Platformer Experience Dataset (PED), and some promising first results of the transformed model