Examining the role of smart TVs and VR HMDs in synchronous at-a-distance media consumption

This article examines synchronous at-a-distance media consumption from two perspectives: How it can be facilitated using existing consumer displays (through TVs combined with smartphones), and imminently available consumer displays (through virtual reality (VR) HMDs combined with RGBD sensing). First, we discuss results from an initial evaluation of a synchronous shared at-a-distance smart TV system, CastAway. Through week-long in-home deployments with five couples, we gain formative insights into the adoption and usage of at-a-distance media consumption and how couples communicated during said consumption. We then examine how the imminent availability and potential adoption of consumer VR HMDs could affect preferences toward how synchronous at-a-distance media consumption is conducted, in a laboratory study of 12 pairs, by enhancing media immersion and supporting embodied telepresence for communication. Finally, we discuss the implications these studies have for the near-future of consumer synchronous at-a-distance media consumption. When combined, these studies begin to explore a design space regarding the varying ways in which at-a-distance media consumption can be supported and experienced (through music, TV content, augmenting existing TV content for immersion, and immersive VR content), what factors might influence usage and adoption and the implications for supporting communication and telepresence during media consumption

Sound for Fantasy and Freedom

Sound is an integral part of our everyday lives. Sound tells us about physical events in the environ- ment, and we use our voices to share ideas and emotions through sound. When navigating the world on a day-to-day basis, most of us use a balanced mix of stimuli from our eyes, ears and other senses to get along. We do this totally naturally and without effort. In the design of computer game experiences, traditionally, most attention has been given to vision rather than the balanced mix of stimuli from our eyes, ears and other senses most of us use to navigate the world on a day to day basis. The risk is that this emphasis neglects types of interaction with the game needed to create an immersive experience. This chapter summarizes the relationship between sound properties, GameFlow and immersive experience and discusses two projects in which Interactive Institute, Sonic Studio has balanced perceptual stimuli and game mechanics to inspire and create new game concepts that liberate users and their imagination

Towards 3-D Sound: Spatial Presence and the Space Vacuum

This chapter demonstrates the evolution of relationships between sound design and music in cinematic representations of the interstellar space vacuum. Mera provides a framework for understanding how audiences believe they are physically present in the represented environment and argues that, in the late 2000s, we move towards three-dimensional (3-D) sound, an aesthetic and technical extension of the superfield and the ultrafield as defined by Chion and Kerins, respectively. 3-D Sound’s primary characteristic is the emancipation of music from a fixed sound-stage spatialization, resulting in greater fluidity between sound design and music. This chapter examines the relationship between two types of spatial presence, articulating both the audience’s suspension of disbelief within a film’s narrative world and the spatial presence of sound and music within a multichannel cinema environme

Experiencing Cinematic VR: Where Theory and Practice Converge in the Tribeca Film Festival Cinema360

Cinematic virtual reality (VR) production has reached enough capacity to support a festival. This paper offers a theoretical framework of VR narrative structure to critically examine one such festival in cinematic VR. The spotlight here is on the fifteen entries in the 2020 Tribeca Film Festival Cinema360. Findings suggest that although the field of cinematic VR has advanced substantially in recent years in terms of narrative design and user experience, there is still a considerable distance for VR storytellers to travel to fully utilize the nature and potential of the developing medium of virtual reality

'Girlfriends and Strawberry Jam’: Tagging Memories, Experiences, and Events for Future Retrieval

In this short paper we have some preliminary thoughts about tagging everyday life events in order to allow future retrieval of events or experiences related to events. Elaboration of these thoughts will be done in the context of the recently started Network of Excellence PetaMedia (Peer-to-Peer Tagged Media) and the Network of Excellence SSPNet (Social Signal Processing), to start in 2009, both funded by the European Commission's Seventh Framework Programme. Descriptions of these networks will be given later in this paper

Is the World After All Just a Dream?

DocPerform is a multi and interdisciplinary research project based at City, University of London. Led by members of the Department of Library & Information Science, it comprises scholars and practitioners from the fields of performing arts and library & information science. The project concerns conceptual, methodological and technological innovations in the documentation of performance, and the extent to which performance may itself be considered to be a document. The collection of papers in this special issue of Proceedings from the Document Academy are selected from the second DocPerform Symposium, held at City, University of London, 6–7 November 2017. This editorial introduces those papers and provides disciplinary and historical context for DocPerform

Designing Interactive and Immersive Multimodal Installations for People with Disability

We developed an end-to-end co-creative methodology for designing interactive and immersive multisensory virtual reality experiences with a particular focus on people with disability. Our method draws on what is called “design thinking” to provide a backbone to our approach. This embraces three stages, an empathic first stage, followed by an ideation phase, during which the thematic context is elaborated, and then an iterative exploration phase during which the initial concept is refined and the implementation is achieved. Furthermore, the “cognitive design” methodology developed by one of us led us to an approach incorporating all sensory modalities, not just the audio and visual modalities (that is, it includes odor, tactile, taste and proprioceptive stimuli), in order to deliver an experience that fully enhances the user’s sense of embodiment, and also led us to place the user’s experience at the heart of the installation. Users participate in the design process through co-design protocols. We showcase the application of this methodology in a detailed way for the construction of an interactive and immersive VR installation for people with disabilities

Exploring the Influence of Virtual Reality on Adults With Indications of Early Stage Dementia

Virtual reality (VR) has the potential to transport people to compelling worlds and bring positive spiritual benefits to persons with dementia and their caregivers. The purpose of this qualitative phenomenological study was to understand the influence of VR on participants with indications of early stage dementia. This study explored how five elderly participants in a Seattle-based Program of All-Inclusive Care for the Elderly (PACE) program experienced immersive virtual reality by means of a Windows Mixed Reality headset. Using an episode from the 2016 VR series, theBlu—a colorful and immersive underwater experience with soothing audio accompaniment—each participant spent up to one hour with the researcher viewing the VR content and engaging in a pre- and post- interview to discuss their spirituality and VR experience. The participants’ responses were audio recorded, transcribed, and coded from a spiritual care perspective, searching for moments of joy and connection. The six emerging themes included: connection to positive feelings, connection to beauty, connection to the present moment, connection to past memories, a desire for future engagement, and participant recommendations for VR content. The findings demonstrated that immersive VR offers the possibility for people to explore a myriad of exciting and awe-inspiring worlds, bringing beauty and peace while offering relational connection

Evaluation Report of Prospero’s Island: an Immersive Approach to Literacy at Key Stage 3.

Prospero's Island is an immersive theatre project created by Punchdrunk Enrichment and sponsored by Learning Partner, London Borough of Hackney (Hackney Learning Trust). The project sought to inspire and motivate students’ engagement with the English curriculum, and to develop an immersive approach to teaching literacy that would improve students’ learning. Prospero’s Island took place in a secondary academy in Hackney, London over two school terms (autumn 2014-spring 2015). The project was embedded in existing schemes of work, and included the following elements: • An immersive theatre installation for Year 7-8 students (aged 11-13 years); this took the form of an interactive game based on The Tempest; over a two-week period groups of students participated in this experience for a morning or afternoon (autumn term); • A Teaching and Learning Day (TALD) and eight twilight CPD sessions on immersive learning techniques for school staff and teachers across London (autumn term); • A return to the installation for one lesson, led by English teachers (autumn term); • Follow-on work by teachers to develop immersive learning in English lessons (spring term); • An independent evaluation of the project (autumn and spring terms)