Narrative Transformation and Music in Mediatized Moral Space in Charlie Brooker's Bandersnatch

Black Mirror’s Bandersnatch has brought interactive film to the forefront of ludo-entertainment discourse. Released through the Netflix entertainment platform in December 2018, the film features a non-linear script that the viewer guides through a series of choices. By virtue of design, Bandersnatch places a larger emphasis on narratological elements of the interactive experience than gameplay. This paper deconstructs narrative transformations as modeled by David Lewin’s connected-operation network model, functions of music in interactive media, and morality in mediatized spaces for a comprehensive close reading of Bandersnatch as writer Charlie Brooker’s commentary on free will

Game Scoring: Towards a Broader Theory

“Game scoring,” that is, the act of composing music for and through gaming, is distinct from other types of scoring. To begin with, unlike other scoring activities, game scoring depends on — in fact, it arguably is — software programming. The game scorer’s choices are thus first-and-foremost limited by available gaming technology, and the “programmability” of their musical ideas given that technology, at any given historical moment. Moreover, game scores are unique in that they must allow for an unprecedented level of musical flexibility, given the high degree of user interactivity the video game medium enables and encourages. As such, game scoring necessarily constitutes an at least partially aleatoric compositional activity, the final score being determined as much through gameplay as traditional composition. This thesis demonstrates this through case studies of the Nintendo Entertainment System sound hardware configuration, and game scores, including the canonic score for Super Mario Bros. (1985)

Relaciones y grados de dependencia entre la música y la imagen en los videojuegos: Aproximación a su análisis formal desde la jugabilidad

The creation of a specific vocabulary for video games and game design analysis poses a challenge shared by the analysis of the correlation between image and music in them. Concepts from filmmaking, graphic design and other media alone would be insufficient; the interactivity and non linear nature of the video game medium are both their main methodologic issue and distinctive characteristics. In this article we'll try to link music, image and their means of communication between them and with the player, from the gameplay and playability perspective.La generación de un vocabulario específico para el análisis de los videojuegos y el game design es una dificultad de la que tampoco escapa el análisis de la relación imagen-música dentro de ellos. Los conceptos extraídos del cine, del diseño gráfico, o de otros medios lineales serían insuficientes para afrontar esto; la interactividad y no linealidad del medio son tanto su principal dificultad metodológica como sus características distintivas. En este artículo intentaremos vincular la imagen y música y sus maneras de comunicarse entre sí y con el jugador, desde la jugabilidad misma.A geração de um vocabulário específico para o análise dos videogames e o game design é uma dificuldade da qual também não escapa o análise da relação imagem-música dentro deles. Os conceitos extraídos do cinema, do desenho gráfico, ou de outros medios lineaís seriam insuficientes para afrontar isto; a interatividade e no linealidade do médio são tanto a su principale dificuldade metodológica quanto as suas características distintivas. Neste artigo tentaremos vincular a imagem e a música e as suas maneiras de se comunicar entre elas e com o jogador, desde a jogabilidade mesma

Influencia de la banda sonora en la evaluación de personajes de una película

Desde el inicio del cine, las bandas sonoras se caracterizan por cumplir diversas funciones que hacen compleja la experiencia del espectador. De este modo, se puede decir que éstas promueven la relación con el personaje principal de una película. Utilizando el método experimental, esta tesis explora cómo el tipo de música (drama o thriller) afecta la evaluación de personajes de películas en una muestra de estudiantes universitarios de Lima Metropolitana (n=30). Se realizaron análisis en tres niveles: a nivel videos, a nivel condiciones y a nivel intrasujeto. Los resultados demuestran que la música de una película puede llegar a generar diferencias en la agradabilidad del personaje, las intenciones que se le atribuyen, la seguridad que sienten los espectadores de saber lo que el personaje esté pensando o sintiendo y en nivel de identificación con el personaje. Siendo las respuestas atribuidas a personajes del género dramático las más positivas (mayor agradabilidad, mejores intenciones y mayor identificación).Since the beginning of cinema, movie scores have been known for performing certain functions that make the spectator’s experience more complex, some of these promote the establishment of a relationship with the main character. Using an experimental design, this study explores how different types of music (taken from drama and thriller genre) affect the evaluation of movie characters in a sample of students from a private university in Lima Metropolitana (n=30). Analyses were carried out in three levels: considering each video as a unit of analysis, across conditions (drama, thriller and without music) and within-subjects. Results show that movie music can produce differences in character likeability, interpretations of the character´s intentions, certainty of what the character is thinking or feeling and identification with the character. Responses attributed to melodramatic characters were more positive (higher likeability, better intentions and greater identification).Tesi

Adaptive Music Generation for Computer Games

This dissertation explores a novel approach to game music that addresses the limitations of conventional game music systems in supporting a dynamically changing narrative. In the proposed approach, the music is generated automatically based on a set of variable input parameters corresponding to emotional musical features. These are then tied to narrative parameters in the game, so that the features and emotions of the music are perceived to continuously adapt to the game's changing narrative. To investigate this approach, an algorithmic music generator was developed which outputs a stream of chords based on several input parameters. The parameters control different aspects of the music, including the transition matrix of a Markov model used to stochastically generate the chords, and can be adjusted continuously in real time. A tense first-person game was then configured to control the generator's input parameters to reflect the changing tension of its narrative---for example, as the narrative tension of the game increases, the generated music becomes more dissonant and the tempo increases. The approach was empirically evaluated primarily by having participants play the game under a variety of conditions, comparing them along several subjective dimensions. The participants' skin conductance was also recorded. The results indicate that the condition with the dynamically varied music described above was both rated and felt as the most tense and exciting, and, for participants who said they enjoy horror games and films, also rated as the most preferable and fun. Another study with music experts then demonstrated that the proposed approach produced smoother musical transitions than crossfades, the approach conventionally used in computer games. Overall, the findings suggest that dynamic music can have a significant positive impact on game experiences, and that generating it algorithmically based on emotional musical features is a viable and effective approach

Atmosphere & Challenge: An Exploration of Dissonant Player Experiences

Dissonance means an unusual combination of any two things. Two dissonant experiences in video games which could lead to undesirable player states are thematic dissonance and difficulty dissonance. Thematic dissonance potentially annoys players by breaking the atmosphere, and difficulty dissonance by preventing players with low skill from progressing past unbalanced challenges, resulting in rage-quits. This thesis seeks to deepen the understanding of dissonant experiences in video games through two experiments measuring the player experience as affected by different audio and practice conditions respectively. Results indicate that the experience colloquially referred to as a rage-quit is directly affected by avatar death events and game-specific skill and is related to lower levels of heart rate variability (HRV) and higher levels of electrodermal activity (EDA), which implicates feelings of stress. This project successfully advances the definition of video game atmosphere as the level of subjective thematic fit or association between the audio and visual components of a game’s setting, and indicates that musical thematic dissonance may lead to higher intensity negative valence facial events

Automatic Generation of Dynamic Musical Transitions in Computer Games

In video games, music must often change quickly from one piece to another due to player interaction, such as when moving between different areas. This quick change in music can often sound jarring if the two pieces are very different from each other. Several transition techniques have been used in industry such as the abrupt cut transition, crossfading, horizontal resequencing and vertical reorchestration among others. However, while several claims are made about their effectiveness (or lack thereof), none of these have been experimentally tested. To investigate how effective each transition technique is, this dissertation empirically evaluates each technique in a study informed by music psychology. This is done based on several features identified as being important for successful transitions. The obtained results led to a novel approach to musical transitions in video games by investigating the use of a multiple viewpoint system, with viewpoints being modelled using Markov models. This algorithm allowed the seamless generation of music that could serve as a transition between two composed pieces of music. While transitions in games normally occur over a zone boundary, the algorithm presented in this dissertation takes place over a transition region, giving the generated music enough time to transition. This novel approach was evaluated in a bespoke video game environment, where participants navigated through several pairs of different game environments and rated the resulting musical transitions. The results indicate that the generated transitions perform as well as crossfading, a technique commonly used in the industry. Since crossfading is not always appropriate, being able to use generated transitions gives composers another tool in their toolbox. Furthermore, the principled approach taken opens up avenues for further research

CREATING A COHERENT SCORE: THE MUSIC OF SINGLE-PLAYER FANTASY COMPUTER ROLE-PLAYING GAMES

This thesis provides a comprehensive exploration into the music of the ludic genre (Hourigan, 2005) known as a Computer Role-Playing Game (CRPG) and its two main sub-divisions: Japanese and Western Role-Playing Games (JRPGs & WRPGs). It focuses on the narrative category known as genre fiction, concentrating on fantasy fiction (Turco, 1999) and seeks to address one overall question: How do fantasy CRPG composers incorporate the variety of musical material needed to create a coherent score across the JRPG and WRPG divide? Seven main chapters form the thesis text. Chapter One provides an introduction to the thesis, detailing the research contributions in addition to outlining a variety of key terms that must be understood to continue with the rest of the text. A database accompanying this thesis showcases the vast range of CRPGs available; a literature review tackles relevant existing materials. Chapters Two and Three seek to provide the first canonical history of soundtracks used in CRPGs by dissecting typical narrative structures for games so as to provide context to their musical scores. Through analysis of existing game composer interviews, cultural influences are revealed. Chapters Four and Five mirror one another with detailed discussion respectively regarding JRPG and WRPG music including the influence that anime and Hollywood cinema have had upon them. In Chapter Six, the use of CRPG music outside of video games is explored, particularly the popularity of JRPG soundtracks in the concert hall. Chapter Seven concludes the thesis, summarising research contributions achieved and areas for future work. Throughout these chapters, the core task is to explain how the two primary sub-genres of CRPGs parted ways and why the music used to accompany these games differs so drastically

Towards Balancing Fun and Exertion in Exergames: Exploring the Impact of Movement-Based Controller Devices, Exercise Concepts, Game Adaptivity and Player Modes on Player Experience and Training Intensity in Different Exergame Settings

Physical inactivity remains one of the biggest societal challenges of the 21st century. The gaming industry and the fitness sector have responded to this alarming fact by introducing game-based or gamified training scenarios and thus established the promising trend of exergaming. Exergames – games controlled by active (whole) body movements – have been extolled as potential attractive and effective training tools. However, the majority of the exergames do not meet the required intensity or effectiveness, nor do they induce the intended training adherence or long-term motivation. One reason for this is that the evaluated exergames were often not co-designed with the user group to meet their specific needs and preferences, nor were they co-designed with an interdisciplinary expert team of game designers (to ensure a good gaming experience) and sports scientists (for a great training experience). Accordingly, the research results from studies with these exergames are rather limited. To fully exploit the potential of these innovative movement tools and to establish them as attractive and effective training approach, it is necessary to understand and explore both the underlying interdisciplinary theories and concepts as well as possible design approaches and their impact on the game and training experience. This dissertation aims to contribute to a better understanding of well-balanced exergame design. It explores and evaluates how different movement-based control devices, exercise concepts, game adaptations, and player modes influence the attractiveness and effectiveness of exergames. The work provides theoretical and practical contributions to the problem area of effective and attractive exergames. For this purpose, a research and development (R&D) approach with iterative phases was followed. As preliminary work for the contributions of this dissertation, exergames were approached from a theoretical perspective. Underlying multidisciplinary theories and concepts of exergames from relevant fields were analyzed and a generic framework was built, which structured the findings based on three interdependent dimensions: the player, the game controller, and the virtual game scenario. Some commercially available exergames were explored to verify the theory-based assumption that the interposition of technology brings specific transformations in the coupling of perception and action that do not occur in real sports situations. Among other things, the comparative pilot study showed that two different controllers (one gesture-based and one haptic device), which allowed for different physical input, were likely to induce diverse gameplay experiences (e.g., higher feeling of flow and self-location when playing with the haptic device) with differently skilled players. However, certain design-specific differences in the two exergame conditions meant that these results could only be interpreted as a first trend. To overcome the limitations of this preliminary study approach (e.g., unequal game design of the commercial exergames and very sports-specific movement concept), Plunder Planet, an adaptive exergame environment, was iteratively designed with and for children and allowed for a single- and cooperative multiplayer experience with two different controller devices. The user-centered design was further informed by insights from the growing body of related R&D work in the field of exergames. The first study presented in this dissertation compared the subjectively experienced attractiveness and effectiveness of Plunder Planet when played with different motion-based controllers. Besides a generally great acceptance of the exergame, it was found that the haptic full-body motion controller provided physical guidance and a more cognitively and coordinatively challenging workout, which was more highly rated by experienced gamers with fewer athletic skills. The gesture-based Kinect sensor felt more natural, allowed more freedom of movement, and provided a rather physically intense but cognitively less challenging workout, which was more highly rated by athletic players with less gameplay experience. Furthermore, experiments were made with an exploratory adaptive algorithm that enabled the cognitive and the physical challenge of the exergame to be manually adapted in real-time based on the player’s fitness and gaming skills. The first and the second study also compared an adaptive with a non-adaptive single player version of Plunder Planet. It could be shown that the (well-balanced) adaptive version of the exergame was better valued than the non-adaptive version with regard to the experienced and measured attractiveness (motivation, game flow, spatial presence experience, balance of cognitive and physical challenge) and effectiveness (heart rate, physical exertion, balance of cognitive and physical challenge) by differently skilled players. Finally, and contrary to the findings from related work, the results of the third study proved that the specifically designed controller technology could be used as an “enabler”, “supporter” and “shaper” of bodily interplay in social exergaming. Based on these promising findings, the goal became to further explore the effectiveness of exergames, refine the adaptive game difficulty algorithm, and explore further attractiveness- and motivation-boosting design approaches. Therefore, the ExerCube, a physically immersive and adaptive fitness game setting, was developed. It was iteratively designed with and for adults and allowed for cooperatively and competitive exergame experiences. With its physically immersive game setup, the ExerCube combines a mixed version of the advantages of both previously tested controllers. A coordinatively and cognitively challenging functional workout protocol with scalable intensity (moderate to high) was developed and the subjective experience of the ExerCube training was compared with a conventional functional training with a personal trainer. The fourth study showed that the game-based training gave signs of reaching a similar intensity to the personal training, but was more highly rated for flow, motivation, and enjoyment. Based on this exploratory comparison of the ExerCube with a personal trainer session, valuable avenues for further design could be identified. Among other things, it could be proved that the player’s focus during the ExerCube session was more on the game than on the own body. Players experienced stronger physical exertion and social pressure with the personal trainer and a stronger cognitive exertion and involvement with the ExerCube. Furthermore, a refined version of the previously tested adaptive game difficulty algorithm was implemented and automated for the first time for purpose of this study. Again it was shown that the adaptive version had benefits with regard to subjectively experienced attractiveness (motivation, game flow, balance of cognitive and physical challenge) and effectiveness (physical exertion, balance of cognitive and physical challenge) compared to the non-adaptive version. In order to further enhance the gaming experience, experiments were also conducted with sound designs and an adaptive audio design with adaptive background music and sound feedback was implemented. It was found to be a promising and beneficial add-on for a user-centered attractive exergame design. To inform the design of a multiplayer version of the ExerCube, different social play mechanics were explored in the fifth study. This resulted in differently balanced experiences of fun, and in physical as well as cognitive exertion. As the preliminary comparative evaluation of the subjectively experienced effectiveness and attractiveness of an ExerCube session and a personal trainer session could prove the general feasibility of the concept and revealed the first indications of the intensity of the ExerCube’s training concept, the objectively measured effectiveness of a single ExerCube session with a functional high-intensity interval training (fHIIT) with a personal trainer was compared in a final sixth study, and after another design iteration. Again, the subjectively experienced attractiveness of both conditions was assessed. It could be shown that the ExerCube is a feasible training device for training at fHIIT-level. While physical exertion was slightly lower than in the conventional fHIIT condition, the ExerCube condition’s average heart rate values reached the fHIIT threshold and also yielded significantly better results for flow, enjoyment, and motivation. The ExerCube training also resulted in a subjectively experienced higher cognitive load (dual-domain training). To sum up, it can be stated that this dissertation provides valuable and fundamental research contributions to the promising field of exergames as attractive and effective training tools. Furthermore, important contributions to design questions in this field could be developed. Since this field is still relatively unexplored, the work presented creates a sound basis for future R&D work in this area

Understanding Game Scoring: Software Programming, Aleatoric Composition and Mimetic Music Technology

Game scoring, that is, the act of composing music for and through gaming, is distinct from other types of scoring. To begin with, unlike other scoring activities, game scoring depends on — in fact, it arguably is — software programming. The game scorer‘s choices are thus first-and-foremost limited by available gaming technology, and the programmability of their musical ideas given that technology, at any given historical moment. Moreover, game scores are unique in that they must allow for an unprecedented level of musical flexibility, given the high degree of user interactivity the video game medium enables and encourages. As such, game scoring necessarily constitutes an at least partially aleatoric compositional activity, the final score being determined as much through gameplay as traditional composition. This dissertation demonstrates how game scoring is software programming that is structured by gaming technology, and that constitutes a unique kind of aleatoric composition, through case studies of the Nintendo Entertainment System sound hardware configuration, and game scores, including the canonic score for Super Mario Bros. (1985)