Noted: Musical affordances for collective exploratory music-making

Noted is a product system that allows for exploration of musical ideas and collaboration through an integrated tangible user interface (TUI). It combines a mobile app and physical controller accessories that attach to the phone. This interaction of digital app functionality and tangible controls highlights the role of musical haptics as the primary channel of interaction between a musician and an instrument. This design concept is structured around four main themes: Exploratory Workflows, Musical Haptics, Musical Affordances and Collective Action

Tangibility and Richness in Digital Musical Instrument Design

PhDThe sense of touch plays a fundamental role in musical performance: alongside hearing, it is the primary sensory modality used when interacting with musical instruments. Learning to play a musical instrument is one of the most developed haptic cultural practices, and within acoustic musical practice at large, the importance of touch and its close relationship to virtuosity and expression is well recognised. With digital musical instruments (DMIs) – instruments involving a combination of sensors and a digital sound engine – touch-mediated interaction remains the foremost means of control, but the interfaces of such instruments do not yet engage with the full spectrum of sensorimotor capabilities of a performer. This poses compelling questions for digital instrument design: how does the nuance and richness of physical interaction with an instrument manifest itself in the digital domain? Which design parameters are most important for haptic experience, and how do these parameters affect musical performance? Built around three practical studies which utilise DMIs as technology probes, this thesis addresses these questions from the point of view of design, of empirical musicology, and of tangible computing. In the first study musicians played a DMI with continuous pitch control and vibrotactile feedback in order to understand how dynamic tactile feedback can be implemented and how it influences musician experience and performance. The results suggest that certain vibrotactile feedback conditions can increase musicians’ tuning accuracy, but also disrupt temporal performance. The second study examines the influence of asynchronies between audio and haptic feedback. Two groups of musicians, amateurs and professional percussionists, were tasked with performing on a percussive DMI with variable action-sound latency. Differences between the two groups in terms of temporal accuracy and quality judgements illustrate the complex effects of asynchronous multimodal feedback. In the third study guitar-derivative DMIs with variable levels of control richness were observed with non-musicians and guitarists. The results from this study help clarify the relationship between tangible design factors, sensorimotor expertise and instrument behaviour. This thesis introduces a descriptive model of performer-instrument interaction, the projection model, which unites the design investigations from each study and provides a series of reflections and suggestions on the role of touch in DMI design.Doctoral Training Centre for Media and Arts Technolog

Subtlety and detail in digital musical instrument design

PhD thesisSubtlety and detail are fundamental to what makes musical instruments special, and worth dedicating a life’s practice to, for designer, maker, player and listener alike. While instruments are recognised and classified by form, it is the nuances of individual instruments that constitute their power to say what could not be said any other way. Digital musical instruments (DMI) have long been criticised as lacking expressive depth, but technology of sufficient fidelity now exists, which raises compelling questions. What can contemporary DMI designers learn from heritage practices about mastering subtlety and detail? What forms does this mastery take, and how can it be elucidated, compared and shared? Using DMI design tools, kits and activities as probes, this thesis addresses these questions from the perspectives of design, embodiment and craft. In a preliminary study, violin luthiers were asked about subtlety and detail in their practice and culture. The outcomes suggested that subtle details originate in the tacit and embodied realms, which are facilitated to develop by specific contexts, environments and materials. In the first study, attendees of a DMI research conference participated in a workshop reflecting on subtlety and detail. Attendees were divided into groups and explored the physical details of a DMI design kit, in an activity book ended by discussion. Responses focused on re-interpretations of instrumental identity, suggesting that the provided context motivated in the opposite direction to the original brief. In the second study, the same kit was deployed with single rather than co-located groups of digital luthiers, modifying instead the sound of the instrument via a Pure Data patch, and responses focused less on instrumental identity and more on gesture-sound mapping strategies. Provocatively, neither studies resulted in sustained focus on details, motivating a novel DMI probe and activity for individuals. In the third study, digital and traditional instrument makers, musicians and other creatives, were invited to handcraft the resonance models of a digital tuned percussion instrument using sculpting clay, responding to constrained briefs. Participants’ backgrounds deeply influenced their responses, and distinctive themes emerged related to aesthetics, tacit and embodied knowledge, and algorithmic pattern. This thesis introduces a scale-based ontology of DMI design, dividing detail into macro, meso and micro levels. Focusing on the micro scale, a series of reflections and suggestions are provided based on the investigations, for how DMI design practitioners, technologists and researchers can illuminate this domain, for the benefit of subtle and detailed digital musical expression