Improvisation Pedagogy: An Epistemological Perspective of the 4‘E’ Model within Digital Musical Instruments

Recent years have witnessed the appearance of many new digital musical instruments (DMIs) and other interfaces for musical expression (NIME). This paper highlights a well-established music educational background theory that we believe may help DMI developers and users better understand DMIs in the context of music cognition and education. From an epistemological perspective, we present the paradigm of enactive music cognition related to improvisation in the context of the skills and needs of 21st century music learners. We hope this can lead to a deeper insertion of DMIs into music education, as well as to new DMIs to be ideated, prototyped and developed within these concepts and theories in mind. We specifically address the theory generally known as the 4E model of cognition (embodied, embedded, extended and enactive) within DMIs. The concept of autopoiesis is also described. Finally, we present some concrete cases of DMIs and NIMEs, and we describe how the experience of musical improvisation with them may be seen through the prism of such theories

Design Strategies for Adaptive Social Composition: Collaborative Sound Environments

In order to develop successful collaborative music systems a variety of subtle interactions need to be identified and integrated. Gesture capture, motion tracking, real-time synthesis, environmental parameters and ubiquitous technologies can each be effectively used for developing innovative approaches to instrument design, sound installations, interactive music and generative systems. Current solutions tend to prioritise one or more of these approaches, refining a particular interface technology, software design or compositional approach developed for a specific composition, performer or installation environment. Within this diverse field a group of novel controllers, described as ‘Tangible Interfaces’ have been developed. These are intended for use by novices and in many cases follow a simple model of interaction controlling synthesis parameters through simple user actions. Other approaches offer sophisticated compositional frameworks, but many of these are idiosyncratic and highly personalised. As such they are difficult to engage with and ineffective for groups of novices. The objective of this research is to develop effective design strategies for implementing collaborative sound environments using key terms and vocabulary drawn from the available literature. This is articulated by combining an empathic design process with controlled sound perception and interaction experiments. The identified design strategies have been applied to the development of a new collaborative digital instrument. A range of technical and compositional approaches was considered to define this process, which can be described as Adaptive Social Composition. Dan Livingston

Composing with Sound-Objects: A Methodology

Technology presents us with the ability to record and manipulate the entire universe of sound in a musical composition. As a result, composers are faced with an overwhelming—often paralyzing—amount of available musical options. My methodology focuses on how a sound-object informs the organization, collection, manipulation, and culmination of a work of electronic music. I believe that breaking down the number of choices to manageable bite-sized portions helps minimize ambiguity, and imposing limits on musical parameters helps the composer focus on productive musical options. This is a methodology where the sound-object holds primacy over the work and serves as the motivic touchstone from which to make all compositional decisions.Part one of the dissertation provides a definition of a sound-object and an historical overview. Part two is my methodology, which is divided into three working stages: onset, continuant, and termination. The onset stage discusses a compositional approach to organizing a piece of music based on the sound-object as motivic touchstone; it introduces the organizational process according to functional considerations as well as conceptual approaches. The continuant stage is the composer’s playground where sound is transformed. It includes the technical and practical approaches used to assess the many parameters of a sound-object, as well as how the object itself informs the transformations. Additionally, the continuant stage represents an approach to composition and improvisation—informed by the sound-object—that uses acoustic instruments. Finally, the termination stage brings all these elements together in order to finish the piece. This stage explores how the sound-object can inform the structure of the piece at the subsequent levels of event, phrase, section, and overall form. I will demonstrate this methodology by explaining how I composed five original pieces with sound-objects—Acoustic Memories (2019), Modular Voices (2019), Interconnected (2019), Synthetic Objects (2019), and Gucci Concrète (2019). Through the framework presented in this methodology, composers of electronic music will better understand this flexible medium of composition, by moving beyond the traditional grid of discrete pitches and rhythms, in order to control the entire universe of sound for their palette of inspiration

Interaction Design for Digital Musical Instruments

The thesis aims to elucidate the process of designing interactive systems for musical performance that combine software and hardware in an intuitive and elegant fashion. The original contribution to knowledge consists of: (1) a critical assessment of recent trends in digital musical instrument design, (2) a descriptive model of interaction design for the digital musician and (3) a highly customisable multi-touch performance system that was designed in accordance with the model. Digital musical instruments are composed of a separate control interface and a sound generation system that exchange information. When designing the way in which a digital musical instrument responds to the actions of a performer, we are creating a layer of interactive behaviour that is abstracted from the physical controls. Often, the structure of this layer depends heavily upon: 1. The accepted design conventions of the hardware in use 2. Established musical systems, acoustic or digital 3. The physical configuration of the hardware devices and the grouping of controls that such configuration suggests This thesis proposes an alternate way to approach the design of digital musical instrument behaviour – examining the implicit characteristics of its composite devices. When we separate the conversational ability of a particular sensor type from its hardware body, we can look in a new way at the actual communication tools at the heart of the device. We can subsequently combine these separate pieces using a series of generic interaction strategies in order to create rich interactive experiences that are not immediately obvious or directly inspired by the physical properties of the hardware. This research ultimately aims to enhance and clarify the existing toolkit of interaction design for the digital musician

Electrifying Opera, Amplifying Agency: Designing a performer-controlled interactive audio system for opera singers

This artistic research project examines the artistic, technical, and pedagogical challenges of developing a performer-controlled interactive technology for real-time vocal processing of the operatic voice. As a classically trained singer-composer, I have explored ways to merge the compositional aspects of transforming electronic sound with the performative aspects of embodied singing. I set out to design, develop, and test a prototype for an interactive vocal processing system using sampling and audio processing methods. The aim was to foreground and accommodate an unamplified operatic voice interacting with the room's acoustics and the extended disembodied voices of the same performer. The iterative prototyping explored the performer's relationship to the acoustic space, the relationship between the embodied acoustic voice and disembodied processed voice(s), and the relationship to memory and time. One of the core challenges was to design a system that would accommodate mobility and allow interaction based on auditory and haptic cues rather than visual. In other words, a system allowing the singer to control their sonic output without standing behind a laptop. I wished to highlight and amplify the performer's agency with a system that would enable nuanced and variable vocal processing, be robust, teachable, and suitable for use in various settings: solo performances, various types and sizes of ensembles, and opera. This entailed mediating different needs, training, and working methods of both electronic music and opera practitioners. One key finding was that even simple audio processing could achieve complex musical results. The audio processes used were primarily combinations of feedback and delay lines. However, performers could get complex musical results quickly through continuous gestural control and the ability to route signals to four channels. This complexity sometimes led to surprising results, eliciting improvisatory responses also from singers without musical improvisation experience. The project has resulted in numerous vocal solo, chamber, and operatic performances in Norway, the Netherlands, Belgium, and the United States. The research contributes to developing emerging technologies for live electronic vocal processing in opera, developing the improvisational performance skills needed to engage with those technologies, and exploring alternatives for sound diffusion conducive to working with unamplified operatic voices. Links: Exposition and documentation of PhD research in Research Catalogue: Electrifying Opera, Amplifying Agency. Artistic results. Reflection and Public Presentations (PhD) (2023): https://www.researchcatalogue.net/profile/show-exposition?exposition=2222429 Home/Reflections: https://www.researchcatalogue.net/view/2222429/2222460 Mapping & Prototyping: https://www.researchcatalogue.net/view/2222429/2247120 Space & Speakers: https://www.researchcatalogue.net/view/2222429/2222430 Presentations: https://www.researchcatalogue.net/view/2222429/2247155 Artistic Results: https://www.researchcatalogue.net/view/2222429/222248

End-user action-sound mapping design for mid-air music performance

How to design the relationship between a performer’s actions and an instrument’s sound response has been a consistent theme in Digital Musical Instrument (DMI) research. Previously, mapping was seen purely as an activity for DMI creators, but more recent work has exposed mapping design to DMI musicians, with many in the field introducing soware to facilitate end-user mapping, democratising this aspect of the DMI design process. This end-user mapping process provides musicians with a novel avenue for creative expression, and offers a unique opportunity to examine how practising musicians approach mapping design.Most DMIs suffer from a lack of practitioners beyond their initial designer, and there are few that are used by professional musicians over extended periods. The Mi.Mu Gloves are one of the few examples of a DMI that is used by a dedicated group of practising musicians, many of whom use the instrument in their professional practice, with a significant aspect of creative practice with the gloves being end-user mapping design. The research presented in this dissertation investigates end-user mapping practice with the Mi.Mu Gloves, and what influences glove musicians’ design decisions based on the context of their music performance practice, examining the question: How do end-users of a glove-based mid-air DMI design action–sound mapping strategies for musical performance?In the first study, the mapping practice of existing members of the Mi.Mu Glove community is examined. Glove musicians performed a mapping design task, which revealed marked differences in the mapping designs of expert and novice glove musicians, with novices designing mappings that evoked conceptual metaphors of spatial relationships between movement and music, while more experienced musicians focused on designing ergonomic mappings that minimised performer error.The second study examined the initial development period of glove mapping practice. A group of novice glove musicians were tracked in a longitudinal study. The findings supported the previous observation that novices designed mappings using established conceptual metaphors, and revealed that transparency and the audience’s ability to perceive their mappings was important to novice glove musicians. However, creative mapping was hindered by system reliability and the novices’ poorly trained posture recognition.The third study examined the mapping practice of expert glove musicians, who took part in a series of interviews. Findings from this study supported earlier observations that expert glove musicians focus on error minimisation and ergonomic, simple controls, but also revealed that the expert musicians embellished these simple controls with performative ancillary gestures to communicate aesthetic meaning. The expert musicians also suffered from system reliability, and had developed a series of gestural techniques to mitigate accidental triggering.The fourth study examined the effects of system-related error in depth. A laboratory study was used to investigate how system-related errors impacted a musician’s ability to acquire skill with the gloves, finding that a 5% rate of system error had a significant effect on skill acquisition.Learning from these findings, a series of design heuristics are presented, applicable for use in the fields of DMI design, mid-air interaction design and end-user mapping design