Physical Interactions with Digital Strings - A hybrid approach to a digital keyboard instrument

A new hybrid approach to digital keyboard playing is presented, where the actual acoustic sounds from a digital keyboard are captured with contact microphones and applied as excitation signals to a digital model of a prepared piano, i.e., an extended wave-guide model of strings with the possibility of stopping and muting the strings at arbitrary positions. The parameters of the string model are controlled through TouchKeys multitouch sensors on each key, combined with MIDI data and acoustic signals from the digital keyboard frame, using a novel mapping. The instrument is evaluated from a performing musician's perspective, and emerging playing techniques are discussed. Since the instrument is a hybrid acoustic-digital system with several feedback paths between the domains, it provides for expressive and dynamic playing, with qualities approaching that of an acoustic instrument, yet with new kinds of control. The contributions are two-fold. First, the use of acoustic sounds from a physical keyboard for excitations and resonances results in a novel hybrid keyboard instrument in itself. Second, the digital model of "inside piano" playing, using multitouch keyboard data, allows for performance techniques going far beyond conventional keyboard playing

PLXTRM : prediction-led eXtended-guitar tool for real-time music applications and live performance

peer reviewedThis article presents PLXTRM, a system tracking picking-hand micro-gestures for real-time music applications and live performance. PLXTRM taps into the existing gesture vocabulary of the guitar player. On the first level, PLXTRM provides a continuous controller that doesn’t require the musician to learn and integrate extrinsic gestures, avoiding additional cognitive load. Beyond the possible musical applications using this continuous control, the second aim is to harness PLXTRM’s predictive power. Using a reservoir network, string onsets are predicted within a certain time frame, based on the spatial trajectory of the guitar pick. In this time frame, manipulations to the audio signal can be introduced, prior to the string actually sounding, ’prefacing’ note onsets. Thirdly, PLXTRM facilitates the distinction of playing features such as up-strokes vs. down-strokes, string selections and the continuous velocity of gestures, and thereby explores new expressive possibilities

Designing and Composing for Interdependent Collaborative Performance with Physics-Based Virtual Instruments

Interdependent collaboration is a system of live musical performance in which performers can directly manipulate each other’s musical outcomes. While most collaborative musical systems implement electronic communication channels between players that allow for parameter mappings, remote transmissions of actions and intentions, or exchanges of musical fragments, they interrupt the energy continuum between gesture and sound, breaking our cognitive representation of gesture to sound dynamics. Physics-based virtual instruments allow for acoustically and physically plausible behaviors that are related to (and can be extended beyond) our experience of the physical world. They inherently maintain and respect a representation of the gesture to sound energy continuum. This research explores the design and implementation of custom physics-based virtual instruments for realtime interdependent collaborative performance. It leverages the inherently physically plausible behaviors of physics-based models to create dynamic, nuanced, and expressive interconnections between performers. Design considerations, criteria, and frameworks are distilled from the literature in order to develop three new physics-based virtual instruments and associated compositions intended for dissemination and live performance by the electronic music and instrumental music communities. Conceptual, technical, and artistic details and challenges are described, and reflections and evaluations by the composer-designer and performers are documented

Noted: Musical affordances for collective exploratory music-making

Noted is a product system that allows for exploration of musical ideas and collaboration through an integrated tangible user interface (TUI). It combines a mobile app and physical controller accessories that attach to the phone. This interaction of digital app functionality and tangible controls highlights the role of musical haptics as the primary channel of interaction between a musician and an instrument. This design concept is structured around four main themes: Exploratory Workflows, Musical Haptics, Musical Affordances and Collective Action

Interaction Design for Digital Musical Instruments

The thesis aims to elucidate the process of designing interactive systems for musical performance that combine software and hardware in an intuitive and elegant fashion. The original contribution to knowledge consists of: (1) a critical assessment of recent trends in digital musical instrument design, (2) a descriptive model of interaction design for the digital musician and (3) a highly customisable multi-touch performance system that was designed in accordance with the model. Digital musical instruments are composed of a separate control interface and a sound generation system that exchange information. When designing the way in which a digital musical instrument responds to the actions of a performer, we are creating a layer of interactive behaviour that is abstracted from the physical controls. Often, the structure of this layer depends heavily upon: 1. The accepted design conventions of the hardware in use 2. Established musical systems, acoustic or digital 3. The physical configuration of the hardware devices and the grouping of controls that such configuration suggests This thesis proposes an alternate way to approach the design of digital musical instrument behaviour – examining the implicit characteristics of its composite devices. When we separate the conversational ability of a particular sensor type from its hardware body, we can look in a new way at the actual communication tools at the heart of the device. We can subsequently combine these separate pieces using a series of generic interaction strategies in order to create rich interactive experiences that are not immediately obvious or directly inspired by the physical properties of the hardware. This research ultimately aims to enhance and clarify the existing toolkit of interaction design for the digital musician

The Original Beat: An Electronic Music Production System and Its Design

The barrier to entry in electronic music production is high. It requires expensive, complicated software, extensive knowledge of music theory and experience with sound generation. Digital Audio Workstations (DAWs) are the main tools used to piece together digital sounds and produce a complete song. While these DAWs are great for music professionals, they have a steep learning curve for beginners and they must run native on a user’s computer. For a novice to begin creating music takes much more time, eort, and money than it should. We believe anyone who is interested in creating electronic music deserves a simple way to digitize their ideas and hear results. With this idea in mind, we created a web based, co-creative system to allow beginners and pros alike to easily create electronic digital music. We outline the requirements for such a system and detail the design and architecture. We go through the specifics of the system we implemented covering the front-end, back-end, server, and generation algorithms. Finally, we will review our development time-line, examine the challenges and risks that arose when building our system, and present future improvements

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc

Space shuttle avionics system

The Space Shuttle avionics system, which was conceived in the early 1970's and became operational in the 1980's represents a significant advancement of avionics system technology in the areas of systems and redundacy management, digital data base technology, flight software, flight control integration, digital fly-by-wire technology, crew display interface, and operational concepts. The origins and the evolution of the system are traced; the requirements, the constraints, and other factors which led to the final configuration are outlined; and the functional operation of the system is described. An overall system block diagram is included

Audio-haptic relationships as compositional and performance strategies

As a performer of firstly acoustic and latterly electronic and electro-instrumental music, I constantly seek to improve my mode of interaction with the digital realm: that is, to achieve a high level of sensitivity and expression. This thesis illustrates reasons why making use of haptic interfaces—which offer physical feedback and resistance to the performer—may be viewed as an important approach in addressing the shortcomings of some the standard systems used to mediate the performer’s engagement with various sorts of digital musical information. By examining the links between sound and touch, and the performer-instrument relationship, various new compositional and performance strategies start to emerge. I explore these through a portfolio of original musical works, which span the continuum of composition and improvisation, largely based around performance paradigms for piano and live electronics. I implement new haptic technologies, using vibrotactile feedback and resistant interfaces, as well as exploring more metaphorical connections between sound and touch. I demonstrate the impact that the research brings to the creative musical outcomes, along with the implications that these techniques have on the wider field of live electronic musical performance

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc