A Lightweight Intelligent Virtual Cinematography System for Machinima Production

Machinima is a low-cost alternative to full production filmmaking. However, creating quality cinematic visualizations with existing machinima techniques still requires a high degree of talent and effort. We introduce a lightweight artificial intelligence system, Cambot, that can be used to assist in machinima production. Cambot takes a script as input and produces a cinematic visualization. Unlike other virtual cinematography systems, Cambot favors an offline algorithm coupled with an extensible library of specific modular and reusable facets of cinematic knowledge. One of the advantages of this approach to virtual cinematography is a tight coordination between the positions and movements of the camera and the actors

'Breaking the glass': preserving social history in virtual environments

New media technologies play an important role in the evolution of our society. Traditional museums and heritage sites have evolved from the ‘cabinets of curiosity’ that focused mainly on the authority of the voice organising content, to the places that offer interactivity as a means to experience historical and cultural events of the past. They attempt to break down the division between visitors and historical artefacts, employing modern technologies that allow the audience to perceive a range of perspectives of the historical event. In this paper, we discuss virtual reconstruction and interactive storytelling techniques as a research methodology and educational and presentation practices for cultural heritage sites. We present the Narrating the Past project as a case study, in order to illustrate recent changes in the preservation of social history and guided tourist trails that aim to make the visitor’s experience more than just an architectural walk through

Portmanteau worlds: hosting multiple worldviews in virtual environments.

Intending to persuade (or sell), conventional architectural representation often hides conflicting opinions, discourages participation and culls possible futures. Dissatisfied with this situation, I consider an approach that aims to emphasize co-presence of multiple voices, disclose power relationships, demonstrate lines of resistance and present existing or possible places as politically charged networks of enacted relationships. Motivated by the capabilities of interactive narrative, the paper considers polyphonic potentials of virtual environments. In spite of their interactive characteristics, virtual environments can impose preconceived worldviews as forcefully as any others. However, generative capabilities of computational media can also support construction of multiple interpretations that emerge simultaneously

Portmanteau worlds: hosting multiple worldviews in virtual environments.

Intending to persuade (or sell), conventional architectural representation often hides conflicting opinions, discourages participation and culls possible futures. Dissatisfied with this situation, I consider an approach that aims to emphasize co-presence of multiple voices, disclose power relationships, demonstrate lines of resistance and present existing or possible places as politically charged networks of enacted relationships. Motivated by the capabilities of interactive narrative, the paper considers polyphonic potentials of virtual environments. In spite of their interactive characteristics, virtual environments can impose preconceived worldviews as forcefully as any others. However, generative capabilities of computational media can also support construction of multiple interpretations that emerge simultaneously

Narratives of ocular experience in interactive 360° environments

The purpose of this research project was to examine how immersive digital virtual technologies have the potential to expand the genre of interactive film into new forms of audience engagement and narrative production. Aside from addressing the limitations of interactive film, I have explored how interactive digital narratives can be reconfigured in the wake of immersive media. My contribution to knowledge stems from using a transdisciplinary synthesis of the interactive systems in film and digital media art, which is embodied in the research framework and theoretical focal point that I have titled Cynematics (chapter 2). Using a methodology that promotes iterative experimentation I developed a series of works that allowed me to practically explore the limitations of interactive film systems that involve non-haptic user interaction. This is evidenced in the following series of works: Virtual Embodiment, Narrative Maze, Eye Artefact Interactions and Routine Error - all of which are discussed in chapter 4 of this thesis. Each of these lab experiments collectively build towards the development of novel interactive 360° film practices. Funneling my research towards these underexplored processes I focused on virtual gaze interaction (chapters 4-6), aiming to define and historically contextualise this system of interaction, whilst critically engaging with it through my practice. It is here that gaze interaction is cemented as the key focus of this thesis. The potential of interactive 360° film is explored through the creation of three core pieces of practice, which are titled as follows: Systems of Seeing (chapter 5), Mimesis (chapter 6), Vanishing Point (chapter 7). Alongside the close readings in these chapters and the theoretical developments explored in each are the interaction designs included in the appendix of the thesis. These provide useful context for readers unable to experience these site-specific installations as virtual reality applications. After creating these systems, I established terms to theoretically unpack some of the processes occurring within them. These include Datascape Mediation (chapter 2), which frames agency as a complex entanglement built on the constantly evolving relationships between human and machine - and Live-Editing Practice (chapter 7), which aims to elucidate how the interactive 360° film practice designed for this research leads to new way of thinking about how we design, shoot and interact with 360° film. Reflecting on feedback from exhibiting Mimesis I decided to define and evaluate the key modes of virtual gaze interaction, which led to the development of a chapter and concept referred to as The Reticle Effect (chapter 6). This refers to how a visual overlay that is used to represent a user's line of sight not only shapes their experience of the work, but also dictates their perception of genre. To navigate this, I combined qualitative and quantitative analysis to explore user responses to four different types of gaze interaction. In preparing to collect this data I had to articulate these different types of interaction, which served to demarcate the difference between each of these types of gaze interaction. Stemming from this I used questionnaires, thematic analysis and data visualisation to explore the use and response to these systems. The results of this not only supports the idea of the reticle effect, but also gives insight into how these different types of virtual gaze interaction shape whether these works are viewed as games or as types of interactive film. The output of this allowed me to further expand on interactive 360° film as a genre of immersive media and move beyond the realm of interactive film into new technological discourses, which serves to validate the nascent, yet expansive reach of interactive 360° film as a form of practice. The thesis is concluded by framing this research within the wider discourse of posthuman theory as given that the technologies of immersive media perpetuate a state of extended human experience - how we interact and consider the theories that surround these mediums needs to be considered in the same way. The practice and theory developed throughout this thesis contribute to this discourse and allow for new ways of considering filmic language in the wake of interactive 360° film practice

Interactive Fiction in Cinematic Virtual Reality: Epistemology, Creation and Evaluation

This dissertation presents the Interactive Fiction in Cinematic Virtual Reality (IFcVR), an interactive digital narrative (IDN) that brings together the cinematic virtual reality (cVR) and the creation of virtual environments through 360\ub0 video within an interactive fiction (IF) structure. This work is structured in three components: an epistemological approach to this kind of narrative and media hybrid; the creation process of IFcVR, from development to postproduction; and user evaluation of IFcVR. In order to set the foundations for the creation of interactive VR fiction films, I dissect the IFcVR by investigating the aesthetics, narratological and interactive notions that converge and diverge in it, proposing a medium-conscious narratology for this kind of artefact. This analysis led to the production of an IFcVR functional prototype: \u201cZENA\u201d, the first interactive VR film shot in Genoa. ZENA\u2019s creation process is reported proposing some guidelines for interactive and immersive film-makers. In order to evaluate the effectiveness of the IFcVR as an entertaining narrative form and a vehicle for diverse types of messages, this study also proposes a methodology to measure User Experience (UX) on IFcVR. The full evaluation protocol gathers both qualitative and quantitative data through ad hoc instruments. The proposed protocol is illustrated through its pilot application on ZENA. Findings show interactors' positive acceptance of IFcVR as an entertaining experience

Applications of CSP solving in computer games (camera control)

While camera control systems of commercial 3D games have improved greatly in recent years, they are not as fully developed as are other game components such as graphics and physics engines. Bourne and Sattar (2006) have proposed a reactive constraint based third person perspective camera control system. We have extended the capability of their system to handle occlusion while following the main character, and have used camera cuts to find appropriate camera positions for a few difficult situations. We have developed a reactive constraint based third person perspective chase camera control system to follow a character in a 3D environment. The camera follows the character from (near) optimal positions defined by a camera profile. The desired values of the height and distance constraints of the camera profile are changed appropriately whenever the character enters a semi-enclosed or an enclosed area, and the desired value of the orientation constraint of the camera profile is changed incrementally whenever theoptimal camera view is obstructed. Camera cuts are used whenever the main character backs up to a wall or any other obstructions, or comes out of a semi-enclosed or an enclosed area. Two auxiliary cameras to observe the main camera positions from top and side views have been added. The chase camera control system achieved real-time performance while following the main character in a typical 3D environment, and maintained an optimal view based on a user specified/selected camera profile

Toward Intelligent Support of Authoring Machinima Media Content: Story and Visualization

The Internet and the availability of authoring tools have enabled a greater community of media content creators, including nonexperts. However, while media authoring tools often make it technically feasible to generate, edit and share digital media artifacts, they do not guarantee that the works will be valuable or meaningful to the community at large. Therefore intelligent tools that support the authoring and creative processes are especially valuable. In this paper, we describe two intelligent support tools for the authoring and production of machinima. Machinima is a technique for producing computer-animated movies through the manipulation of computer game technologies. The first system we describe, ReQUEST, is an intelligent support tool for the authoring of plots. The second system, Cambot, produces machinima from a pre-authored script by manipulating virtual avatars and a virtual camera in a 3D graphical environment

Intelligent Camera Control Using Behavior Trees

Automatic camera systems produce very basic animations for virtual worlds. Users often view environments through two types of cameras: a camera that they control manually, or a very basic automatic camera that follows their character, minimizing occlusions. Real cinematography features much more variety producing more robust stories. Cameras shoot establishing shots, close-ups, tracking shots, and bird’s eye views to enrich a narrative. Camera techniques such as zoom, focus, and depth of field contribute to framing a particular shot. We present an intelligent camera system that automatically positions, pans, tilts, zooms, and tracks events occurring in real-time while obeying traditional standards of cinematography. We design behavior trees that describe how a single intelligent camera might behave from low-level narrative elements assigned by “smart events”. Camera actions are formed by hierarchically arranging behavior sub-trees encapsulating nodes that control specific camera semantics. This approach is more modular and particularly reusable for quickly creating complex camera styles and transitions rather then focusing only on visibility. Additionally, our user interface allows a director to provide further camera instructions, such as prioritizing one event over another, drawing a path for the camera to follow, and adjusting camera settings on the fly.We demonstrate our method by placing multiple intelligent cameras in a complicated world with several events and storylines, and illustrate how to produce a well-shot “documentary” of the events constructed in real-time