Telling interactive stories: A practice-based investigation into new media interactive storytelling

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Telling Interactive Stories is a practice-based thesis, which theoretically and practically probes the field of digital fictional interactive storytelling. The submission takes the form of the interactive cinema installation Crossed Lines together with a written element of the thesis which interrogates historical, contextual, theoretical, technical and critical aspects of the field of interactive narrative using new media. Crossed Lines is an original fictional interactive AV piece, amalgamating multiform plots, a multi-screen viewing environment, an interactive interface and an interactive story navigation form. The installation tells the stories of nine characters in a way that the viewer can constantly explore and switch between all nine forms, using a telephone keypad and handset as an interface, and can simultaneously observe all characters’ presence between the nine remote locations. Several research methodologies are utilised to analyse and evaluate the installation. Quantitative methodologies include the use of user tracking systems where the computational output of the installation provides measurements and timings of user choices and behaviours. Qualitative methodologies include theoretical and visual analysis, and in depth analysis of user responses using interviews, questionnaires, video recordings and cuttingedge eye-tracking technologies

HeadOn: Real-time Reenactment of Human Portrait Videos

We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and eye gaze. Given a short RGB-D video of the target actor, we automatically construct a personalized geometry proxy that embeds a parametric head, eye, and kinematic torso model. A novel real-time reenactment algorithm employs this proxy to photo-realistically map the captured motion from the source actor to the target actor. On top of the coarse geometric proxy, we propose a video-based rendering technique that composites the modified target portrait video via view- and pose-dependent texturing, and creates photo-realistic imagery of the target actor under novel torso and head poses, facial expressions, and gaze directions. To this end, we propose a robust tracking of the face and torso of the source actor. We extensively evaluate our approach and show significant improvements in enabling much greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at Siggraph'1

Machinima And Video-based Soft Skills Training

Multimedia training methods have traditionally relied heavily on video based technologies and significant research has shown these to be very effective training tools. However production of video is time and resource intensive. Machinima (pronounced \u27muh-sheen-eh-mah\u27) technologies are based on video gaming technology. Machinima technology allows video game technology to be manipulated into unique scenarios based on entertainment or training and practice applications. Machinima is the converting of these unique scenarios into video vignettes that tell a story. These vignettes can be interconnected with branching points in much the same way that education videos are interconnected as vignettes between decision points. This study addressed the effectiveness of machinima based soft-skills education using avatar actors versus the traditional video teaching application using human actors. This research also investigated the difference between presence reactions when using avatar actor produced video vignettes as compared to human actor produced video vignettes. Results indicated that the difference in training and/or practice effectiveness is statistically insignificant for presence, interactivity, quality and the skill of assertiveness. The skill of active listening presented a mixed result indicating the need for careful attention to detail in situations where body language and facial expressions are critical to communication. This study demonstrates that a significant opportunity exists for the exploitation of avatar actors in video based instruction

The New Reflexivity: Puzzle Films, Found Footage, and Cinematic Narration in the Digital Age

“The New Reflexivity” tracks two narrative styles of contemporary Hollywood production that have yet to be studied in tandem: the puzzle film and the found footage horror film. In early August 1999, near the end of what D.N. Rodowick refers to as “the summer of digital paranoia,” two films entered the wide-release U.S. theatrical marketplace and enjoyed surprisingly massive financial success, just as news of the “death of film” circulated widely. Though each might typically be classified as belonging to the horror genre, both the unreliable “puzzle film” The Sixth Sense and the fake-documentary “found footage film” The Blair Witch Project stood as harbingers of new narrative currents in global cinema. This dissertation looks closely at these two films, reading them as illustrative of two decidedly millennial narrative styles, styles that stepped out strikingly from the computer-generated shadows cast by big-budget Hollywood. The industrial shift to digital media that coincides with the rise of these films in the late 90s reframed the cinematic image as inherently manipulable, no longer a necessary index of physical reality. Directors become image-writers, constructing photorealistic imagery from scratch. Meanwhile, DVDs and online paratexts encourage cinephiles to digitize, to attain and interact with cinema in novel ways. “The New Reflexivity” reads The Sixth Sense and The Blair Witch Project as reflexive allegories of cinema’s and society’s encounters with new digital media. The most basic narrative tricks and conceits of puzzle films and found footage films produce an unusually intense and ludic engagement with narrative boundaries and limits, thus undermining the naturalized practices of classical Hollywood narration. Writers and directors of these films treat recorded events and narrative worlds as reviewable, remixable, and upgradeable, just as Hollywood digitizes and tries to keep up with new media. Though a great deal of critical attention has been paid to both puzzle and found footage films separately, no lengthy critical survey has yet been undertaken that considers these movies in terms of their shared formal and thematic concerns. Rewriting the rules of popular cinematic narration, these films encourage viewers to be suspicious of what they see onscreen, to be aware of the possibility of unreliable narration, or CGI and the “Photoshopped.” Urgent to film and cultural studies, “The New Reflexivity” suggests that these genres’ complicitous critique of new media is decidedly instructive for a networked society struggling with what it means to be digital

Designing virtual spaces: redefining radio art through digital control

Radio Art is a composition practice that is constantly evolving. Artists share a commonality to redefine, reinvent, and repurpose analogue radio. It is an art that often bends to the will of antiqued technology, celebrating a wide pallet of found sounds. This research extends the boundaries of the art form by exploring Radio Art through sonic-centric lens and establishing a consistent and reproducible compositional framework. By shifting radio from a found object to an instrument, I have deconstructed its sonic aesthetics into two parallel materials for composition, gestural noise and broadcast signal. When tuning an analogue radio to a signal, relationships between these materials unfold. Contrast is a term found throughout my research. Contrast is embodied throughout radio and its history; radio is used as both a scientific communication device and for artistic expression. it is a symbol of democracy and oppression. Radio produces broadcast noise and signal, creating poetic reception, such as control and chaos, anxiety and ecstasy, distance and closeness. This research explores the characteristics of these forces and materials as a symbiotic relationship of unfolding radiophonic behaviours. A major focus of this research is the control of analogue radio through deconstruction and composition. I embarked on a twenty-four-month development period to build a Digital Audio Workstation called Radiophonic Environmental Designer, (RED). RED enables composers to create virtual radiophonic environments that are navigated by rotating the dial. Material is positioned along a horizon, and tuning behaviours sculpted. There is also a physical interface embedded into an analogue radio shell to control the virtual tuning, namely, Broadcast Link-up Environment, (BLUE). BLUE is an ad-on program offering an online digital platform for the diffusion of Radio Art. Using an internet connection and gyroscope technology that is built into most smart phones, a radiophonic environment is interacted through a purpose-built website. In my creative practice, analogue radio has been redesigned by adopting digital technological practices to control, edit and model it’s unique sound. In doing so, I reflect upon relationships between analogue and digital design principles through an extensive study on virtual analogue software and interfaces

Youth, Technology, and DIY: Developing Participatory Competencies in Creative Media Production

Traditionally, educational researchers and practitioners have focused on the development of youths’ critical understanding of media as a key aspect of new media literacies. The 21st Century media landscape suggests an extension of this traditional notion of literacy – an extension that sees creative designs, ethical considerations, and technical skills as part of youth's expressive and intellectual engagement with media as participatory competencies. These engagements with media are also part of a growing Do-It-Yourself, or DIY, movement involving arts, crafts, and new technologies. The purpose of this chapter is to provide a framework and a language for understanding the multiple DIY practices in which youth engage while producing media. In the review, we will first provide a historical overview of the shifting perspectives of two related fields—new media literacies and computer literacy —before outlining the general trends in DIY media cultures that see youth moving towards becoming content creators. We then introduce how a single framework allows us to consider different participatory competencies in DIY under one umbrella. Special attention will be given to the digital practices of remixing, reworking, and repurposing popular media among disadvantaged youth. We will conclude with considerations of equity, access, and participation in after-school settings and possible implications for K-12 education

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Simple identification tools in FishBase

Simple identification tools for fish species were included in the FishBase information system from its inception. Early tools made use of the relational model and characters like fin ray meristics. Soon pictures and drawings were added as a further help, similar to a field guide. Later came the computerization of existing dichotomous keys, again in combination with pictures and other information, and the ability to restrict possible species by country, area, or taxonomic group. Today, www.FishBase.org offers four different ways to identify species. This paper describes these tools with their advantages and disadvantages, and suggests various options for further development. It explores the possibility of a holistic and integrated computeraided strategy