Interaction and Expressivity in Video Games: Harnessing the Rhetoric of Film

The film-maker uses the camera and editing creatively, not simply to present the action of the film but also to set up a particular relation between the action and the viewer. In 3D video games with action controlled by the player, the pseudo camera is usually less creatively controlled and has less effect on the player’s appreciation of and engagement with the game. This paper discusses methods of controlling games by easy and intuitive interfaces and use of an automated virtual camera to increase the appeal of games for users

A Trip to the Moon: Personalized Animated Movies for Self-reflection

Self-tracking physiological and psychological data poses the challenge of presentation and interpretation. Insightful narratives for self-tracking data can motivate the user towards constructive self-reflection. One powerful form of narrative that engages audience across various culture and age groups is animated movies. We collected a week of self-reported mood and behavior data from each user and created in Unity a personalized animation based on their data. We evaluated the impact of their video in a randomized control trial with a non-personalized animated video as control. We found that personalized videos tend to be more emotionally engaging, encouraging greater and lengthier writing that indicated self-reflection about moods and behaviors, compared to non-personalized control videos

Machinima Filmmaking: The Integration of Immersive Technology for Collaborative Machinima Filmmaking

This Digital Media MS project proposes to create a flexible, intuitive, and low-entry barrier virtual cinematography tool that will enable participants engaged in human computer interaction (HCI) activities to quickly stage, choreograph, rehearse, and capture performances in real-time. Heretofore, Machinima developers have used limited forms of expressive input devices to puppeteer characters and record in-game live performances, using a gamepad, keyboard, mouse, and joystick to produce content. This has stagnated Machinima development because machinimators have not embraced the current evolution of input devices to create, capture, and edit content, thereby omitting game engine programming possibilities that could exploit new 3D compositing techniques and alternatives to strengthen interactivity, collaboration, and efficiency in cinematic pipelines. Our system, leveraging consumer-affordable hardware and software, advances the development of Machinima production by providing a foundation for alternative cinematic practices, to create a seamless form of human computer interaction and to positively convince more people toward Machinima filmmaking. We propose to produce an Unreal Engine 4 system plugin which integrates virtual reality and eye tracking, via the Oculus Rift DK2 and Tobii EyeX, respectively. The plugin will enable two people, in the roles of Director and Performer, to navigate and interact within a virtual 3D space to productively affect collaborative Machinima filmmaking.M.S., Digital Media -- Drexel University, 201

Machinima Filmmaking: The Integration of Immersive Technology for Collaborative Machinima Filmmaking

This Digital Media MS project proposes to create a flexible, intuitive, and low-entry barrier virtual cinematography tool that will enable participants engaged in human computer interaction (HCI) activities to quickly stage, choreograph, rehearse, and capture performances in real-time. Heretofore, Machinima developers have used limited forms of expressive input devices to puppeteer characters and record in-game live performances, using a gamepad, keyboard, mouse, and joystick to produce content. This has stagnated Machinima development because machinimators have not embraced the current evolution of input devices to create, capture, and edit content, thereby omitting game engine programming possibilities that could exploit new 3D compositing techniques and alternatives to strengthen interactivity, collaboration, and efficiency in cinematic pipelines. Our system, leveraging consumer-affordable hardware and software, advances the development of Machinima production by providing a foundation for alternative cinematic practices, to create a seamless form of human computer interaction and to positively convince more people toward Machinima filmmaking. We propose to produce an Unreal Engine 4 system plugin which integrates virtual reality and eye tracking, via the Oculus Rift DK2 and Tobii EyeX, respectively. The plugin will enable two people, in the roles of Director and Performer, to navigate and interact within a virtual 3D space to productively affect collaborative Machinima filmmaking.M.S., Digital Media -- Drexel University, 201

Mosaic narrative a poetics of cinematic new media narrative

This thesis proposes the Poetics of Mosaic Narrative as a tool for theorising the creation and telling of cinematic stories in a digital environment. As such the Poetics of Mosaic Narrative is designed to assist creators of new media narrative to design dramatically compelling screen based stories by drawing from established theories of cinema and emerging theories of new media. In doing so it validates the crucial element of cinematic storytelling in the digital medium, which due to its fragmentary, variable and re-combinatory nature, affords the opportunity for audience interaction. The Poetics of Mosaic Narrative re-asserts the dramatic and cinematic nature of narrative in new media by drawing upon the dramatic theory of Aristotle’s Poetics, the cinematic theories of the 1920s Russian Film Theorists and contemporary Neo-Formalists, the narrative theories of the 1960s French Structuralists, and the scriptwriting theories of contemporary cinema. In particular it focuses on the theory and practice of the prominent new media theorist, Lev Manovich, as a means of investigating and creating a practical poetics. The key element of the Poetics of Mosaic Narrative is the expansion of the previously forgotten and undeveloped Russian Formalist concept of cinematurgy which is vital to the successful development of new media storytelling theory and practice. This concept, as originally proposed but not elaborated by Kazansky, encompasses the notion of the creation of cinematic new media narrative as a mosaic – integrally driven by the narrative systems of plot, as well as the cinematic systems of visual style created by the techniques of cinema- montage, cinematography and mise-en-scene

Towards a Smart Drone Cinematographer for Filming Human Motion

Affordable consumer drones have made capturing aerial footage more convenient and accessible. However, shooting cinematic motion videos using a drone is challenging because it requires users to analyze dynamic scenarios while operating the controller. In this thesis, our task is to develop an autonomous drone cinematography system to capture cinematic videos of human motion. We understand the system's filming performance to be influenced by three key components: 1) video quality metric, which measures the aesthetic quality -- the angle, the distance, the image composition -- of the captured video, 2) visual feature, which encapsulates the visual elements that influence the filming style, and 3) camera planning, which is a decision-making model that predicts the next best movement. By analyzing these three components, we designed two autonomous drone cinematography systems using both heuristic-based methods and learning-based methods.For the first system, we designed an Autonomous CinemaTography system -- "ACT" by proposing a viewpoint quality metric focusing on the visibility of the 3D human skeleton of the subject. We expanded the application of human motion analysis and simplified manual control by assisting viewpoint selection using a through-the-lens method. For the second system, we designed an imitation-based system that learns the artistic intention of the cameramen through watching professional aerial videos. We designed a camera planner that analyzes the video contents and previous camera motion to predict future camera motion. Furthermore, we propose a planning framework, which can imitate a filming style by ``seeing" only one single demonstration video of such style. We named it ``one-shot imitation filming." To the best of our knowledge, this is the first work that extends imitation learning to autonomous filming. Experimental results in both simulation and field test exhibit significant improvements over existing techniques and our approach managed to help inexperienced pilots capture cinematic videos

The Prose Storyboard Language: A Tool for Annotating and Directing Movies: (Version 2.0, Revised and Illustrated Edition)

International audienceThe prose storyboard language is a formal language for describing movies shot by shot, where each shot is described with a unique sentence. The language uses a simple syntax and limited vocabulary borrowed from working practices in traditional movie-making and is intended to be readable both by machines and humans. The language has been designed over the last ten years to serve as a high-level user interface for intelligent cinematography and editing systems. In this new paper, we present the latest evolution of the language, and the results of an extensive annotation exercise showing the benefits of the language in the task of annotating the sophisticated cinematography and film editing of classic movies

Putting the Video Back in Video Games: Opportunities and Challenges for Visual Studies Approaches to Video Game Analysis

I argue for the application of visual studies in video game analysis. This approach presents opportunities for the intersectional analysis of video game visuals as games are brought into dialogue with other visual media like film and painting. This approach also presents challenges due to ontological distinctions between different media as well as due to academic divisions regarding the study of different art and media objects. Despite the challenges presented, a visual studies approach is particularly useful as a critical window into contemporary visual culture at large. I outline the fields of game studies and visual studies, marking their distinctions as well as the areas in which they overlap. I provide examples of visual studies approaches to video game analysis through an emphasis on the visual characteristics of video games. As visual studies is generally considered an interdisciplinary endeavor, I contextualize my analyses through comparisons with other visual media, in particular finding intersections with art history and film studies. Specifically, I argue that perspective is an integral visual trait of many video games, relating the use of linear perspective and isometric perspective, used in some genres of games, with the development of perspective in painting. I examine various cinematic techniques used in video games and discuss their ideological potency. I also cover ways in which video games subvert conventional norms, such as through self-reflexivity, to open up novel avenues for visual expression