Sound for Fantasy and Freedom

Sound is an integral part of our everyday lives. Sound tells us about physical events in the environ- ment, and we use our voices to share ideas and emotions through sound. When navigating the world on a day-to-day basis, most of us use a balanced mix of stimuli from our eyes, ears and other senses to get along. We do this totally naturally and without effort. In the design of computer game experiences, traditionally, most attention has been given to vision rather than the balanced mix of stimuli from our eyes, ears and other senses most of us use to navigate the world on a day to day basis. The risk is that this emphasis neglects types of interaction with the game needed to create an immersive experience. This chapter summarizes the relationship between sound properties, GameFlow and immersive experience and discusses two projects in which Interactive Institute, Sonic Studio has balanced perceptual stimuli and game mechanics to inspire and create new game concepts that liberate users and their imagination

Presenting in Virtual Worlds: Towards an Architecture for a 3D Presenter explaining 2D-Presented Information

Entertainment, education and training are changing because of multi-party interaction technology. In the past we have seen the introduction of embodied agents and robots that take the role of a museum guide, a news presenter, a teacher, a receptionist, or someone who is trying to sell you insurances, houses or tickets. In all these cases the embodied agent needs to explain and describe. In this paper we contribute the design of a 3D virtual presenter that uses different output channels to present and explain. Speech and animation (posture, pointing and involuntary movements) are among these channels. The behavior is scripted and synchronized with the display of a 2D presentation with associated text and regions that can be pointed at (sheets, drawings, and paintings). In this paper the emphasis is on the interaction between 3D presenter and the 2D presentation

Between Art and Social Science: scenic composition as a methodological device

The scenic composition (SC) is a methodological device enabling the synthesis and articulation of researchers' own complex experiences of events witnessed during data collection. Positioned between art and social science, it makes use of literary conventions to synthesise "experience near" accounts of data for interpretation. This article explains how the SC is composed by drawing on associative thinking and illustrates its use within a specific case study. The conceptual basis of the SC is discussed with reference to the work of LORENZER, WINNICOTT and BION. This is the first study in which four compositions, each by a different researcher, have been used to provide a multi-faceted view of a complex event, a live webcast. The compositions are presented along with researchers' reflections. Common themes and significant differences relating to life situations, histories and dispositions of the researchers emerge. The differences were expressed through choice of literary genres, which are common cultural resources. We ask what was achieved through the use of SCs compared with a thematic analysis of the webcast, and find that apart from synthesising and presentational functions, they give access to a multi-sensory range of researchers' experiences, including unconscious elements which were then available for reflexive interpretation by an interpretation panel

An interoceptive predictive coding model of conscious presence

We describe a theoretical model of the neurocognitive mechanisms underlying conscious presence and its disturbances. The model is based on interoceptive prediction error and is informed by predictive models of agency, general models of hierarchical predictive coding and dopaminergic signaling in cortex, the role of the anterior insular cortex (AIC) in interoception and emotion, and cognitive neuroscience evidence from studies of virtual reality and of psychiatric disorders of presence, specifically depersonalization/derealization disorder. The model associates presence with successful suppression by top-down predictions of informative interoceptive signals evoked by autonomic control signals and, indirectly, by visceral responses to afferent sensory signals. The model connects presence to agency by allowing that predicted interoceptive signals will depend on whether afferent sensory signals are determined, by a parallel predictive-coding mechanism, to be self-generated or externally caused. Anatomically, we identify the AIC as the likely locus of key neural comparator mechanisms. Our model integrates a broad range of previously disparate evidence, makes predictions for conjoint manipulations of agency and presence, offers a new view of emotion as interoceptive inference, and represents a step toward a mechanistic account of a fundamental phenomenological property of consciousness

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

A place within a place: Toward new understandings on the enactment of contemporary imaginative play practices and places

This study was an intergenerational inquiry into imaginative play practices and places. Using narrative inquiry, it explored the childhood imaginative play practices and places of four families who resided in and around the suburbs of Melbourne, Australia. Each family had three generations who participated in the study; these included grandparents, parents, primary school children and pre-school children- a total of sixteen participants. Positioned within a qualitative, interpretative research paradigm, the narrative inquiry approach prompted participants’ subjective stories as socially constructed knowledge about their childhood experiences of imaginative play..

E-Drama: Facilitating Online Role-play using an AI Actor and Emotionally Expressive Characters.

This paper describes a multi-user role-playing environment, e-drama, which enables groups of people to converse online, in scenario driven virtual environments. The starting point of this research – edrama – is a 2D graphical environment in which users are represented by static cartoon figures. An application has been developed to enable integration of the existing edrama tool with several new components to support avatars with emotionally expressive behaviours, rendered in a 3D environment. The functionality includes the extraction of affect from open-ended improvisational text. The results of the affective analysis are then used to: (a) control an automated improvisational AI actor – EMMA (emotion, metaphor and affect) that operates a bit-part character in the improvisation; (b) drive the animations of avatars using the Demeanour framework in the user interface so that they react bodily in ways that are consistent with the affect that they are expressing. Finally, we describe user trials that demonstrate that the changes made improve the quality of social interaction and users’ sense of presence. Moreover, our system has the potential to evolve normal classroom education for young people with or without learning disabilities by providing 24/7 efficient personalised social skill, language and career training via role-play and offering automatic monitoring

Creating emotionally aware performance environments: a phenomenological exploration of inferred and invisible data space

Merged with duplicate record 10026.1/2674 on 13.03.2017 by CS (TIS)The practical research undertaken for this thesis - the building of interactive and non-interactive environments for performance - posits a radical recasting of the performing body in physical and digital space. The choreographic and thematic context of the performance work has forced us', as makers, to ask questions about the nature of digital interactivity which in turn feeds the work theoretically, technically and thematically. A computer views (and attempts to interpret) motion information through a video camera, and, by way of a scripting language, converts that information into MIDI' data. As the research has developed, our company has been able to design environments which respond sensitivelyto particular artistic / performance demands. I propose to show in this research that is it possible to design an interactive system that is part of a phenomenological performance space, a mechanical system with an ontological heart. This represents a significant shift in thinking from existing systems, is at the heart of the research developments and is what I consider to be one of the primary outcomes of this research, outcomes that are original and contribute to the body of knowledge in this area. The phenomenal system allows me to use technology in a poetic way, where the poetic aesthetic is dominant - it responds to the phenomenal dancer, rather than merely to the `physico-chemical' (Merleau-Ponty 1964 pp. 10-I I) dancer. Other artists whose work attempts phenomenological approaches to working with technology and the human body are referenced throughout the writing