Gesture, Rhetoric, and Digital Storytelling

This proposal seeks to develop new theory and technology for digital interactive narratives based on cross-cultural storytelling traditions that utilize non-verbal communication to convey meaning. Digital interactive narratives are of increasing cultural relevance including: new genres of electronic literature, virtual museums and memorials, educational computer games, and computer-assisted generation of scholarly documents. We propose a new form of storytelling that changes emotional tone, theme, perspective, other subjective elements based upon gestural input via touch-screen and motion-sensitive devices. This proposal builds upon the Principal Investigator's significant body of work in developing humanistically grounded interactive narrative and artificial intelligence. The Principal Investigator has found that literary, cultural, and performance studies theories of orature (oral literature) and accounts of non-verbal communication from animation studies provide models that can be adapted to interactive narratives

Desire Lines: Open Educational Collections, Memory and the Social Machine

This paper delineates the initial ideas around the development of the Co-Curate North East project. The idea of computerised machines which have a social use and impact was central to the development of the project. The project was designed with and for schools and communities as a digital platform which would collect and aggregate ‘memory’ resources and collections around local area studies and social identity. It was a co-curation process supported by museums and curators which was about the ‘meshwork’ between ‘official’ and ‘unofficial’ archives and collections and the ways in which materials generated from within the schools and community groups could themselves be re-narrated and exhibited online as part of self-organised learning experiences. This paper looks at initial ideas of social machines and the ways in machines can be used in identity and memory studies. It examines ideas of navigation and visualisation of data and concludes with some initial findings from the early stages of the project about the potential for machines and educational work

Spectators’ aesthetic experiences of sound and movement in dance performance

In this paper we present a study of spectators’ aesthetic experiences of sound and movement in live dance performance. A multidisciplinary team comprising a choreographer, neuroscientists and qualitative researchers investigated the effects of different sound scores on dance spectators. What would be the impact of auditory stimulation on kinesthetic experience and/or aesthetic appreciation of the dance? What would be the effect of removing music altogether, so that spectators watched dance while hearing only the performers’ breathing and footfalls? We investigated audience experience through qualitative research, using post-performance focus groups, while a separately conducted functional brain imaging (fMRI) study measured the synchrony in brain activity across spectators when they watched dance with sound or breathing only. When audiences watched dance accompanied by music the fMRI data revealed evidence of greater intersubject synchronisation in a brain region consistent with complex auditory processing. The audience research found that some spectators derived pleasure from finding convergences between two complex stimuli (dance and music). The removal of music and the resulting audibility of the performers’ breathing had a significant impact on spectators’ aesthetic experience. The fMRI analysis showed increased synchronisation among observers, suggesting greater influence of the body when interpreting the dance stimuli. The audience research found evidence of similar corporeally focused experience. The paper discusses possible connections between the findings of our different approaches, and considers the implications of this study for interdisciplinary research collaborations between arts and sciences

Towards immersive generosity: The need for a novel framework to explore large audiovisual archives through embodied experiences in immersive environments

This article proposes an innovative framework to explore large audiovisual archives using Immersive Environments to place users inside a dataset and create an embodied experience. It starts by outlining the need for such a novel interface to meet the needs of archival scholars and the GLAM sector, and discusses issues in the current modes of access, mostly restrained to traditional information retrieval systems based on metadata. The paper presents the concept of ``generous interfaces" as a preliminary approach to address these issues, and argues some of the key reasons why employing Immersive Visual Storytelling might benefit such frameworks. The theory of embodiment is leveraged to justify this claim, showing how a more embodied understanding of a collection can result in a stronger engagement for the public. By placing users as actors in the experience rather than mere spectators, the emergence of narrative is driven by their interactions, with benefits in terms of engagement with the public and understanding of the cultural component. The framework we propose is applied to two existing installations to analyze them in-depth and critique them, highlighting the key directions to pursue for further development.Comment: This is the pre-published version (after peer-review

The Mundane Computer: Non-Technical Design Challenges Facing Ubiquitous Computing and Ambient Intelligence

Interdisciplinary collaboration, to include those who are not natural scientists, engineers and computer scientists, is inherent in the idea of ubiquitous computing, as formulated by Mark Weiser in the late 1980s and early 1990s. However, ubiquitous computing has remained largely a computer science and engineering concept, and its non-technical side remains relatively underdeveloped. The aim of the article is, first, to clarify the kind of interdisciplinary collaboration envisaged by Weiser. Second, the difficulties of understanding the everyday and weaving ubiquitous technologies into the fabric of everyday life until they are indistinguishable from it, as conceived by Weiser, are explored. The contributions of Anne Galloway, Paul Dourish and Philip Agre to creating an understanding of everyday life relevant to the development of ubiquitous computing are discussed, focusing on the notions of performative practice, embodied interaction and contextualisation. Third, it is argued that with the shift to the notion of ambient intelligence, the larger scale socio-economic and socio-political dimensions of context become more explicit, in contrast to the focus on the smaller scale anthropological study of social (mainly workplace) practices inherent in the concept of ubiquitous computing. This can be seen in the adoption of the concept of ambient intelligence within the European Union and in the focus on rebalancing (personal) privacy protection and (state) security in the wake of 11 September 2001. Fourth, the importance of adopting a futures-oriented approach to discussing the issues arising from the notions of ubiquitous computing and ambient intelligence is stressed, while the difficulty of trying to achieve societal foresight is acknowledged