Gesture-sound causality from the audience’s perspective: : investigating the aesthetic experience of performances with digital musical instruments.

In contrast to their traditional, acoustic counterparts, digital musical instruments (DMIs) rarely feature a clear, causal relationship between the performer’s actions and the sounds produced. They often function simply as systems for controlling digital sound synthesis, triggering computer-generated audio. This study aims to shed light on how the level of perceived causality of DMI designs impacts audience members’ aesthetic responses to new DMIs. In a preliminary survey, 49 concert attendees listed adjectives that described their experience of a number of DMI performances. In a subsequent experiment, 31 participants rated video clips of performances with DMIs with causal and acausal mapping designs using the eight most popular adjectives from the preliminary survey. The experimental stimuli were presented in their original version and in a manipulated version with a reduced level of gesture-sound causality. The manipulated version was created by placing the audio track of one section of the recording over the video track of a different section. It was predicted that the causal DMIs would be rated more positively, with the manipulation having a stronger effect on the ratings for the causal DMIs. Our results confirmed these hypotheses, and indicate that a lack of perceptible causality does have a negative impact on ratings of DMI performances. The acausal group received no significant difference in ratings between original and manipulated clips. We posit that this result arises from the greater understanding that clearer gesture-sound causality offers spectators. The implications of this result for DMI design and practice are discussed. (PsycINFO Database Record (c) 2018 APA, all rights reserved

Exploring the Motivations for Building New Digital Musical Instruments

Over the past four decades, the number, diversity and complexity of digital musical instruments (DMIs) has increased rapidly. There are very few constraints on DMI design as such systems can be easily reconfigured, offering near limitless flexibility for music-making. Given that new acoustic musical instruments have in many cases been created in response to the limitations of available technologies, what motivates the development of new DMIs? We conducted an interview study with ten designers of new DMIs, in order to explore 1) the motivations electronic musicians may have for wanting to build their own instruments; and 2) the extent to which these motivations relate to the context in which the artist works and performs (academic vs. club settings). We found that four categories of motivation were mentioned most often: M1: wanting to bring greater embodiment to the activity of performing and producing electronic music; M2: wanting to improve audience experiences of DMI performances; M3: wanting to develop new sounds, and M4: wanting to build responsive systems for improvisation. There were also some detectable trends in motivation according to the context in which the artists work and perform. Our results offer the first systematically gathered insights into the motivations for new DMI design. It appears that the challenges of controlling digital sound synthesis drive the development of new DMIs, rather than the shortcomings of any one particular design or existing technology

Almost Human: The Study of Physical Processes and the Performance of a Prosthetic Digital Spine

Almost Human is an investigation of interdisciplinary performance through music that looks to the self to try to further understand subjective performance practices in expression, gesture and sonic output. This text presents experimental methods of examining and creating music through kinaesthetic and electronic-assisted means within instrumental, dance and interactive works. The extraction of affective, performative and sonic properties from these works aids in unlocking the relationship between the choreographic, physical and conceptual object. The first part of the text explores and illustrates multimodal approaches to analysing, capturing, measuring and archiving the moving musician and dancer in an assortment of performative settings. It focuses on a series of works for solo cello, as well as interdisciplinary pieces which positions movement and embodied expressivity at the forefront of the discussion. The second part is dedicated to the aesthetic, conceptual and utilitarian content of a new interactive work for cellist/mover, and a prosthetic digital spine. Here, relationships are combined to showcase the permeability of the body, as well as its expressive content. The conceptual object, The Spine, serves as a generator to help expand musical and artistic possibilities. Its inclusion in the work aids in refocusing my relationship to movement and sound for creation and performance, but also aesthetically, it adds to the growing canon of experimental ventures in conceptualising expressivity. Beyond the text, the portfolio of Almost Human includes an auditory and visual chronicle of the process between the years 2012-14, which is used to assist the reader in further understanding the performative practice and findings

The Show Must Go Wrong: Towards an understanding of audience perception of error in digital musical instrument performance

PhDThis thesis is about DMI (digital musical instrument) performance, its audiences, and their perception of error. The goal of this research is to improve current understanding of how audiences perceive DMI performance, where performers and their audiences often have no shared, external frame of reference with which to judge the musical output. Further complicating this audience-performer relationship are human-computer interaction (HCI) issues arising from the use of a com- puter as a musical instrument. In current DMI literature, there is little direct inquiry of audience perception on these issues. Error is an aspect of this kind of audience perception. Error, a condition reached by stepping out of bounds, appears at first to be a simple binary quantity, but the location and nature of those boundaries change with con- text. With deviation the locus of style and artistic progress, understanding how audiences perceive error has the potential to lend important insight to the cultural mechanics of DMI performance. In this thesis I describe the process of investigating audience perception and unpacking these issues through three studies. Each study examines the relative effects of various factors on audience perception — instrument familiarity and musical style, gesture size, and visible risk — using a novel methodology combining real-time data collected by mobile phone, and post- hoc data in the form of written surveys. The results have implications for DMI and HCI researchers as well as DMI performers and composers, and contribute insights on these confounding factors from the audience’s perspective as well as important insights on audience perception of error in this context. Further, through this thesis I contribute a practical method and tool that can be used to continue this audience-focused work in the future.This work was funded by the Engineering and Physical Sciences Research Council (EPSRC) as part of the Doctoral Training Centre in Media and Arts Technology at Queen Mary University of London (ref: EP/G03723X/1)

Liveness Through the Lens of Agency and Causality

Liveness is a well-known problem with Digital Musical Instruments (DMIs). When used in performances, DMIs provide less visual information than acoustic instruments, preventing the audience from understanding how the musicians influence the music. In this paper, we look at this issuethrough the lens of causality. More specifically, we investigate the attribution of causality by an external observer to a performer, relying on the theory of apparent mental causation. We suggest that the perceived causality between a performer’s gestures and the musical result is central to liveness. We present a framework for assessing attributed causality and agency to a performer, based on a psychological heory which suggests three criteria for inferred causality. These criteria then provide the basis of an experimentalstudy investigating the effect of visual augmentations on audience’s inferred causality. The results provide insights onhow the visual component of performances with DMIs impacts the audience’s causal inferences about the performer.In particular we show that visual augmentations help highlight the influence of the musician when parts of the musicare automated, and help clarify complex mappings between gestures and sounds. Finally we discuss the potential wider implications for assessing liveness in the design of new musical interfaces