Sound Spheres: A Design Study of the Articulacy of a Non-Contact Finger Tracking Virtual Musical Instrument

A key challenge in the design of Virtual Musical instruments (VMIs) is finding expressive, playable, learnable mappings from gesture to sound that progressively reward practice by performers. Designing such mappings can be particularly demanding in the case of non-contact musical instruments, where physical cues can be scarce. Unaided intuition works well for many instrument designers, but others may find design and evaluation heuristics useful when creating new VMIs. In this paper we gather existing criteria from the literature to assemble a simple set of design and evaluation heuristics that we dub articulacy. This paper presents a design case study in which an expressive non-contact finger-tracking VMI, Sound Spheres, is designed and evaluated with the support of the articulacy heuristics. The case study explores the extent to which articulacy usefully informs the design of a non-contact VMI, and we reflect on the usefulness or otherwise of heuristic approaches in this context

End-user action-sound mapping design for mid-air music performance

How to design the relationship between a performer’s actions and an instrument’s sound response has been a consistent theme in Digital Musical Instrument (DMI) research. Previously, mapping was seen purely as an activity for DMI creators, but more recent work has exposed mapping design to DMI musicians, with many in the field introducing soware to facilitate end-user mapping, democratising this aspect of the DMI design process. This end-user mapping process provides musicians with a novel avenue for creative expression, and offers a unique opportunity to examine how practising musicians approach mapping design.Most DMIs suffer from a lack of practitioners beyond their initial designer, and there are few that are used by professional musicians over extended periods. The Mi.Mu Gloves are one of the few examples of a DMI that is used by a dedicated group of practising musicians, many of whom use the instrument in their professional practice, with a significant aspect of creative practice with the gloves being end-user mapping design. The research presented in this dissertation investigates end-user mapping practice with the Mi.Mu Gloves, and what influences glove musicians’ design decisions based on the context of their music performance practice, examining the question: How do end-users of a glove-based mid-air DMI design action–sound mapping strategies for musical performance?In the first study, the mapping practice of existing members of the Mi.Mu Glove community is examined. Glove musicians performed a mapping design task, which revealed marked differences in the mapping designs of expert and novice glove musicians, with novices designing mappings that evoked conceptual metaphors of spatial relationships between movement and music, while more experienced musicians focused on designing ergonomic mappings that minimised performer error.The second study examined the initial development period of glove mapping practice. A group of novice glove musicians were tracked in a longitudinal study. The findings supported the previous observation that novices designed mappings using established conceptual metaphors, and revealed that transparency and the audience’s ability to perceive their mappings was important to novice glove musicians. However, creative mapping was hindered by system reliability and the novices’ poorly trained posture recognition.The third study examined the mapping practice of expert glove musicians, who took part in a series of interviews. Findings from this study supported earlier observations that expert glove musicians focus on error minimisation and ergonomic, simple controls, but also revealed that the expert musicians embellished these simple controls with performative ancillary gestures to communicate aesthetic meaning. The expert musicians also suffered from system reliability, and had developed a series of gestural techniques to mitigate accidental triggering.The fourth study examined the effects of system-related error in depth. A laboratory study was used to investigate how system-related errors impacted a musician’s ability to acquire skill with the gloves, finding that a 5% rate of system error had a significant effect on skill acquisition.Learning from these findings, a series of design heuristics are presented, applicable for use in the fields of DMI design, mid-air interaction design and end-user mapping design

Interactive Tango Milonga: An Interactive Dance System for Argentine Tango Social Dance

abstract: When dancers are granted agency over music, as in interactive dance systems, the actors are most often concerned with the problem of creating a staged performance for an audience. However, as is reflected by the above quote, the practice of Argentine tango social dance is most concerned with participants internal experience and their relationship to the broader tango community. In this dissertation I explore creative approaches to enrich the sense of connection, that is, the experience of oneness with a partner and complete immersion in music and dance for Argentine tango dancers by providing agency over musical activities through the use of interactive technology. Specifically, I create an interactive dance system that allows tango dancers to affect and create music via their movements in the context of social dance. The motivations for this work are multifold: 1) to intensify embodied experience of the interplay between dance and music, individual and partner, couple and community, 2) to create shared experience of the conventions of tango dance, and 3) to innovate Argentine tango social dance practice for the purposes of education and increasing musicality in dancers.Dissertation/ThesisDoctoral Dissertation Music 201

Risk and expression: Physical and material risk states in computational music practices

This research investigates qualities of physical and material risk within musical performance practices and the value that such properties may hold for less physical engagements afforded by computational instruments. The two studies designed for this research draw upon the experiences of practitioners directly, allowing them to speak about their creative processes, values, and priorities, and how risk and expressivity might factor into their practice. Through comparative studies, artifact design, in-depth discussions, and the application of Thematic Analysis I am able to share the perceptions and experiences of practitioners as they themselves describe. By identifying the value that physical and material risk, uncertainty, and the potential for failure play in the creative process we can potentially provide a compelling argument for the importance of such qualities in practices which do not naturally engage with them. Designing for risk and assessing the experiences of practitioners within the field of experimental media performance will contribute to a better understanding of the value of physical and corporeal materials within digital practices and present potential guidelines for the creation and use of new instruments for creative musical expression.Ph.D

An investigation of audio signal-driven sound synthesis with a focus on its use for bowed stringed synthesisers

This thesis proposes an alternative approach to sound synthesis. It seeks to offer traditional string players a synthesiser which will allow them to make use of their existing skills in performance. A theoretical apparatus reflecting on the constraints of formalisation is developed and used to shed light on construction-related shortcomings in the instrumental developments of related research. Historical aspects and methods of sound synthesis, and the act of musical performance, are addressed with the aim of drawing conclusions for the construction of algorithms and interfaces. The alternative approach creates an openness and responsiveness in the synthesis instrument by using implicit playing parameters without the necessity to define, specify or measure all of them. In order to investigate this approach, several synthesis algorithms are developed, sounds are designed and a selection of them empirically compared to conventionally synthesised sounds. The algorithms are used in collaborative projects with other musicians in order to examine their practical musical value. The results provide evidence that implementations using the approach presented can offer musically significant differences as compared to similarly complex conventional implementations, and that - depending on the disposition of the musician - they can form a valuable contribution to the sound repertoire of performers and composers

The Thummer Mapping Project (ThuMP)

This paper presents the Thummer Mapping Project (ThuMP), an industry partnership project between ThumMotion P/L and The University of Western Sydney (UWS). ThuMP sought to developing mapping strategies for new interfaces for musical expression (NIME), specifically the Thummer™, which provides thirteen simultaneous degrees of freedom. This research presents a new approach to the mapping problem resulting from a primary design research phase and a prototype testing and evaluation phase. In order to establish an underlying design approach for the Thummer™ mapping strategies, a number of interviews were carried out with high-level acoustic instrumental performers, the majority of whom play with the Sydney Symphony Orchestra, Sydney, Australia. Mapping strategies were developed from analysis of these interviews and then evaluated in trial usability testing