Exciting Instrumental Data: Toward an Expanded Action-Oriented Ontology for Digital Music Performance

Musical performance using digital musical instruments has obfuscated the relationship between observable musical gestures and the resultant sound. This is due to the sound producing mechanisms of digital musical instruments being hidden within the digital music making system. The difficulty in observing embodied artistic expression is especially true for musical instruments that are comprised of digital components only. Despite this characteristic of digital music performance practice, this thesis argues that it is possible to bring digital musical performance further within our action-oriented ontology by understanding the digital musician through the lens of Lévi-Strauss’ notion of the bricoleur. Furthermore, by examining musical gestures with these instruments through a multi-tiered analytical framework that accounts for the physical computing elements necessarily present in all digital music making systems, we can further understand and appreciate the intricacies of digital music performance practice and culture

Interaction and the Art of User-Centered Digital Musical Instrument Design

This thesis documents the formulation of a research-based practice in multimedia art, technology and digital musical instrument design. The primary goal of my research was to investigate the principles and methodologies involved in the structural design of new interactive digital musical instruments aimed at performance by members of the general public, and to identify ways that the design process could be optimized to increase user adoption of these new instruments. The research was performed over three years and moved between studies at the University of Maine, internships in New York, and specialized research at the Input Devices and Music Interaction Laboratory at McGill University. My work is presented in two sections. The first covers early studies in user interaction and exploratory works in web and visual design, sound art, installation, and music performance. While not specifically tied to the research topic of user adoption of digital musical instruments, this work serves as the conceptual and technical background for the dedicated work to follow. The second section is dedicated to focused research on digital musical instrument design through two major projects carried out as a Graduate Research Trainee at McGill University. The first was the design and prototype of the Noisebox, a new digital musical instrument. The purpose of this project was to learn the various stages of instrument design through practical application. A working prototype has been presented and tested, and a second version is currently being built. The second project was a user study that surveyed musicians about digital musical instrument use. It asked questions about background, instrument choice, music styles played, and experiences with and attitudes towards new digital musical instruments. Based on the results of the two research projects, a model of digital musical instrument design is proposed that adopts a user-centered focus, soliciting user input and feedback throughout the design process from conception to final testing. This approach aims to narrow the gap between conceptual design of new instruments and technologies and the actual musicians who would use them

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Moving sounds and sonic moves : exploring interaction quality of embodied music mediation technologies through a user-centered perspective

This research project deals with the user-experience related to embodied music mediation technologies. More specifically, adoption and policy problems surrounding new media (art) are considered, which arise from the usability issues that to date pervade new interfaces for musical expression. Since the emergence of new wireless mediators and control devices for musical expression, there is an explicit aspiration of the creative industries and various research centers to embed such technologies into different areas of the cultural industries. The number of applications and their uses have exponentially increased over the last decade. Conversely, many of the applications to date still suffer from severe usability problems, which not only hinder the adoption by the cultural sector, but also make culture participants take a rather cautious, hesitant, or even downright negative stance towards these technologies. Therefore, this thesis takes a vantage point that is in part sociological in nature, yet has a link to cultural studies as well. It combines this with a musicological frame of reference to which it introduces empirical user-oriented approaches, predominantly taken from the field of human-computer-interaction studies. This interdisciplinary strategy is adopted to cope with the complex nature of digital embodied music controlling technologies. Within the Flanders cultural (and creative) industries, opportunities of systems affiliated with embodied interaction are created and examined. This constitutes an epistemological jigsaw that looks into 1) “which stakeholders require what various levels of involvement, what interactive means and what artistic possibilities?”, 2) “the way in which artistic aspirations, cultural prerequisites and operational necessities of (prospective) users can be defined?”, 3) “how functional, artistic and aesthetic requirements can be accommodated?”, and 4) “how quality of use and quality of experience can be achieved, quantified, evaluated and, eventually, improved?”. Within this multi-facetted problem, the eventual aim is to assess the applicability of the foresaid technology, both from a theoretically and empirically sound basis, and to facilitate widening and enhancing the adoption of said technologies. Methodologically, this is achieved by 1) applied experimentation, 2) interview techniques, 3) self-reporting and survey research, 4) usability evaluation of existing devices, and 5) human-computer interaction methods applied – and attuned – to the specific case of embodied music mediation technologies. Within that scope, concepts related to usability, flow, presence, goal assessment and game enjoyment are scrutinized and applied, and both task- and experience-oriented heuristics and metrics are developed and tested. In the first part, covering three chapters, the general context of the thesis is given. In the first chapter, an introduction to the topic is offered and the current problems are enumerated. In the second chapter, a broader theoretical background is presented of the concepts that underpin the project, namely 1) the paradigm of embodiment and its connection to musicology, 2) a state of the arts concerning new interfaces for musical expression, 3) an introduction into HCI-usability and its application domain in systematic musicology, 4) an insight into user-centered digital design procedures, and 5) the challenges brought about by e-culture and digitization for the cultural-creative industries. In the third chapter, the state of the arts concerning the available methodologies related to the thesis’ endeavor is discussed, a set of literature-based design guidelines are enumerated and from this a conceptual model is deduced which is gradually presented throughout the thesis, and fully deployed in the “SoundField”-project (as described in Chapter 9). The following chapters, contained in the second part of the thesis, give a quasi-chronological overview of how methodological concepts have been applied throughout the empirical case studies, aimed specifically at the exploration of the various aspects of the complex status quaestionis. In the fourth chapter, a series of application-based tests, predominantly revolving around interface evaluation, illustrate the complex relation between gestural interfaces and meaningful musical expression, advocating a more user-centered development approach to be adopted. In the fifth chapter, a multi-purpose questionnaire dubbed “What Moves You” is discussed, which aimed at creating a survey of the (prospective) end-users of embodied music mediation technologies. Therefore, it primarily focused on cultural background, musical profile and preferences, views on embodied interaction, literacy of and attitudes towards new technology and participation in digital culture. In the sixth chapter, the ethnographical studies that accompanied the exhibition of two interactive art pieces, entitled "Heart as an Ocean" & "Lament", are discussed. In these studies, the use of interview and questionnaire methodologies together with the presentation and reception of interactive art pieces, are probed. In the seventh chapter, the development of the collaboratively controlled music-game “Sync-In-Team” is presented, in which interface evaluation, presence, game enjoyment and goal assessment are the pivotal topics. In the eighth chapter, two usability studies are considered, that were conducted on prototype systems/interfaces, namely a heuristic evaluation of the “Virtual String” and a usability metrics evaluation on the “Multi-Level Sonification Tool”. The findings of these two studies in conjunction with the exploratory studies performed in association with the interactive art pieces, finally gave rise to the “SoundField”-project, which is recounted in full throughout the ninth chapter. The integrated participatory design and evaluation method, presented in the conceptual model is fully applied over the course of the “SoundField”-project, in which technological opportunities and ecological validity and applicability are investigated through user-informed development of numerous use cases. The third and last part of the thesis renders the final conclusions of this research project. The tenth chapter sets out with an epilogue in which a brief overview is given on how the state of the arts has evolved since the end of the project (as the research ended in 2012, but the research field has obviously moved on), and attempts to consolidate the implications of the research studies with some of the realities of the Flemish cultural-creative industries. Chapter eleven continues by discussing the strengths and weaknesses of the conceptual model throughout the various stages of the project. Also, it comprises the evaluation of the hypotheses, how the assumptions that were made held up, and how the research questions eventually could be assessed. Finally, the twelfth and last chapter concludes with the most important findings of the project. Also, it discusses some of the implications on cultural production, artistic research policy and offers an outlook on future research beyond the scope of the “SoundField” project

Composing the instrument: An alternative approach to musical relationships between composer, instrument, performer and audience.

This thesis develops a new conception for ‘composed instruments’ and explores how these instruments can serve to challenge the established norms of musical relationships. It investigates the relationships between music and those who listen, make and facilitate it and, by extension, the relationship between the audience, the performer, and the composer. Music need not be bound by the constraints of traditional instruments and performances can be interactive as opposed to didactic. These notions are investigated through the lens of a series of composed instruments, particularly the Arduinome (and variants), the Large Flat Panel Speakers (LaFPanS) and the Augmented Televisions (ATVs). The building of each of these instruments contributed to the refinement of the concept and, in turn, each has proven to offer a range of artistic possibilities as a result of being developed through a compositional process. The notion of the composed instrument as defined within this text aligns somewhat with the Fluxus group of artists, fostering the transition from audience to performer through universal playability and the levelling of musical hierarchies. Such an approach can re-organise performance hierarchies and have a democratising effect on music-making

Designing and Composing for Interdependent Collaborative Performance with Physics-Based Virtual Instruments

Interdependent collaboration is a system of live musical performance in which performers can directly manipulate each other’s musical outcomes. While most collaborative musical systems implement electronic communication channels between players that allow for parameter mappings, remote transmissions of actions and intentions, or exchanges of musical fragments, they interrupt the energy continuum between gesture and sound, breaking our cognitive representation of gesture to sound dynamics. Physics-based virtual instruments allow for acoustically and physically plausible behaviors that are related to (and can be extended beyond) our experience of the physical world. They inherently maintain and respect a representation of the gesture to sound energy continuum. This research explores the design and implementation of custom physics-based virtual instruments for realtime interdependent collaborative performance. It leverages the inherently physically plausible behaviors of physics-based models to create dynamic, nuanced, and expressive interconnections between performers. Design considerations, criteria, and frameworks are distilled from the literature in order to develop three new physics-based virtual instruments and associated compositions intended for dissemination and live performance by the electronic music and instrumental music communities. Conceptual, technical, and artistic details and challenges are described, and reflections and evaluations by the composer-designer and performers are documented

Virtual Reality and Choreographic Practice:The Potential for New Creative Methods

Virtual reality (VR) is becoming an increasingly intriguing space for dancers and choreographers. Choreographers may find new possibility emerging in using virtual reality to create movement and the WhoLoDancE: Whole-Body Interaction Learning for Dance Education project is developing tools to assist in this process. The interdisciplinary team which includes dancers, choreographers, educators, artists, coders, technologists and system architects have collaborated in engaging, discussing, analysing, testing and working with end-users to help with thinking about the issues that emerge in the creation of these tools. The paper sets out to explore the creative potential of VR in the context of WhoLoDancE and how this may offer new insights for the choreographer and dancer. We pay attention to the virtual environment, the virtual performance and the virtual dancer as some of the key components for equipping the choreographer to use in the creating process and to inform the dancing body. The cyclical process of live body to virtual, back to the dancing body as a choreographic device is an innovative way to approach practice. This approach may lead to new insights and innovations in choreographic methods that may extend beyond the project and ultimately take dance performance in a new direction