Narrative-Driven Camera Control for Cinematic Replay of Computer Games

International audienceThis paper presents a system that generates cinematic replays for dialogue-based 3D video games. The system exploits the narrative and geometric information present in these games and automatically computes camera framings and edits to build a coherent cinematic replay of the gaming session. We propose a novel importance-driven approach to cinematic replay. Rather than relying on actions performed by characters to drive the cinematography (as in idiom-based approaches), we rely on the importance of characters in the narrative. We first devise a mechanism to compute the varying importance of the characters. We then map importances of characters with different camera specifications, and propose a novel technique that (i) automatically computes camera positions satisfying given specifications, and (ii) provides smooth camera motions when transitioning between different specifications. We demonstrate the features of our system by implementing three camera behaviors (one for master shots, one for shots on the player character, and one for reverse shots). We present results obtained by interfacing our system with a full-fledged serious game (Nothing for Dinner) containing several hours of 3D animated content

Mosaic narrative a poetics of cinematic new media narrative

This thesis proposes the Poetics of Mosaic Narrative as a tool for theorising the creation and telling of cinematic stories in a digital environment. As such the Poetics of Mosaic Narrative is designed to assist creators of new media narrative to design dramatically compelling screen based stories by drawing from established theories of cinema and emerging theories of new media. In doing so it validates the crucial element of cinematic storytelling in the digital medium, which due to its fragmentary, variable and re-combinatory nature, affords the opportunity for audience interaction. The Poetics of Mosaic Narrative re-asserts the dramatic and cinematic nature of narrative in new media by drawing upon the dramatic theory of Aristotle’s Poetics, the cinematic theories of the 1920s Russian Film Theorists and contemporary Neo-Formalists, the narrative theories of the 1960s French Structuralists, and the scriptwriting theories of contemporary cinema. In particular it focuses on the theory and practice of the prominent new media theorist, Lev Manovich, as a means of investigating and creating a practical poetics. The key element of the Poetics of Mosaic Narrative is the expansion of the previously forgotten and undeveloped Russian Formalist concept of cinematurgy which is vital to the successful development of new media storytelling theory and practice. This concept, as originally proposed but not elaborated by Kazansky, encompasses the notion of the creation of cinematic new media narrative as a mosaic – integrally driven by the narrative systems of plot, as well as the cinematic systems of visual style created by the techniques of cinema- montage, cinematography and mise-en-scene

Narratives of ocular experience in interactive 360° environments

The purpose of this research project was to examine how immersive digital virtual technologies have the potential to expand the genre of interactive film into new forms of audience engagement and narrative production. Aside from addressing the limitations of interactive film, I have explored how interactive digital narratives can be reconfigured in the wake of immersive media. My contribution to knowledge stems from using a transdisciplinary synthesis of the interactive systems in film and digital media art, which is embodied in the research framework and theoretical focal point that I have titled Cynematics (chapter 2). Using a methodology that promotes iterative experimentation I developed a series of works that allowed me to practically explore the limitations of interactive film systems that involve non-haptic user interaction. This is evidenced in the following series of works: Virtual Embodiment, Narrative Maze, Eye Artefact Interactions and Routine Error - all of which are discussed in chapter 4 of this thesis. Each of these lab experiments collectively build towards the development of novel interactive 360° film practices. Funneling my research towards these underexplored processes I focused on virtual gaze interaction (chapters 4-6), aiming to define and historically contextualise this system of interaction, whilst critically engaging with it through my practice. It is here that gaze interaction is cemented as the key focus of this thesis. The potential of interactive 360° film is explored through the creation of three core pieces of practice, which are titled as follows: Systems of Seeing (chapter 5), Mimesis (chapter 6), Vanishing Point (chapter 7). Alongside the close readings in these chapters and the theoretical developments explored in each are the interaction designs included in the appendix of the thesis. These provide useful context for readers unable to experience these site-specific installations as virtual reality applications. After creating these systems, I established terms to theoretically unpack some of the processes occurring within them. These include Datascape Mediation (chapter 2), which frames agency as a complex entanglement built on the constantly evolving relationships between human and machine - and Live-Editing Practice (chapter 7), which aims to elucidate how the interactive 360° film practice designed for this research leads to new way of thinking about how we design, shoot and interact with 360° film. Reflecting on feedback from exhibiting Mimesis I decided to define and evaluate the key modes of virtual gaze interaction, which led to the development of a chapter and concept referred to as The Reticle Effect (chapter 6). This refers to how a visual overlay that is used to represent a user's line of sight not only shapes their experience of the work, but also dictates their perception of genre. To navigate this, I combined qualitative and quantitative analysis to explore user responses to four different types of gaze interaction. In preparing to collect this data I had to articulate these different types of interaction, which served to demarcate the difference between each of these types of gaze interaction. Stemming from this I used questionnaires, thematic analysis and data visualisation to explore the use and response to these systems. The results of this not only supports the idea of the reticle effect, but also gives insight into how these different types of virtual gaze interaction shape whether these works are viewed as games or as types of interactive film. The output of this allowed me to further expand on interactive 360° film as a genre of immersive media and move beyond the realm of interactive film into new technological discourses, which serves to validate the nascent, yet expansive reach of interactive 360° film as a form of practice. The thesis is concluded by framing this research within the wider discourse of posthuman theory as given that the technologies of immersive media perpetuate a state of extended human experience - how we interact and consider the theories that surround these mediums needs to be considered in the same way. The practice and theory developed throughout this thesis contribute to this discourse and allow for new ways of considering filmic language in the wake of interactive 360° film practice

Cinematic games : the aesthetic influence of cinema on video games

During its first decade, Game Studies debate mainly revolved around the juxtaposition between two perspectives: the one of ludology and the one of narratology, each positing a primary quality of video games against the other. The study of the relationship between cinema and video games got somehow caught in the crossfire between these two fields. In this work, I investigate the extent to which representation in video games is connected to cinema and its representational codes. A number of authors before challenged this assumption, theorising models that only partially connect the cinematic form to video games. Such investigations have always started from the ludologically educated assumption that video games are different from cinema and, therefore, for the premises of this comparison to be considered “vitiated”, only tangentially useful due to the irreconcilably different nature of the two media. The adjective “cinematic” is a concept constantly evoked in cultural discourses concerning video games. Magazines, reviewers, critics, but also designers, artists, users and commentators (even scholars) often summon the idea of cinematic games in the attempt of describing some peculiar features that share affinities with films and suggesting that video games possess the aura of the big screen. Cinematic games are born at the crossroads between interactive movies and video games, for which the cinematic expression is retained by means of audiovisual representation while keeping the action in the hands of the player. Due to the vast scale of the subject, my work focuses on relatively recent developments in game design which have yet to be fully investigated, and seeks to extend existing attempts to apply the tools of film theory to Game Studies. A secondary value of this work is an annotation on the disengagement of moving image scholars with video games, and it partly serves as an invocation for this to change