Musical instrument mapping design with Echo State Networks

Echo State Networks (ESNs), a form of recurrent neural network developed in the field of Reservoir Computing, show significant potential for use as a tool in the design of mappings for digital musical instruments. They have, however, seldom been used in this area, so this paper explores their possible applications. This project contributes a new open source library, which was developed to allow ESNs to run in the Pure Data dataflow environment. Several use cases were explored, focusing on addressing current issues in mapping research. ESNs were found to work successfully in scenarios of pattern classification, multiparametric control, explorative mapping and the design of nonlinearities and uncontrol. 'Un-trained' behaviours are proposed, as augmentations to the conventional reservoir system that allow the player to introduce potentially interesting non-linearities and uncontrol into the reservoir. Interactive evolution style controls are proposed as strategies to help design these behaviours, which are otherwise dependent on arbitrary values and coarse global controls. A study on sound classification showed that ESNs could reliably differentiate between two drum sounds, and also generalise to other similar input. Following evaluation of the use cases, heuristics are proposed to aid the use of ESNs in computer music scenarios

Why musical memory can be preserved in advanced Alzheimer's disease

Musical memory is relatively preserved in Alzheimer's disease and other dementias. In a 7 Tesla functional MRI study employing multi-voxel pattern analysis, Jacobsen et al. identify brain regions encoding long-term musical memory in young healthy controls, and show that these same regions display relatively little atrophy and hypometabolism in patients with Alzheimer's disease.See Clark and Warren (doi:10.1093/brain/awv148) for a scientific commentary on this article. Musical memory is relatively preserved in Alzheimer's disease and other dementias. In a 7 Tesla functional MRI study employing multi-voxel pattern analysis, Jacobsen et al. identify brain regions encoding long-term musical memory in young healthy controls, and show that these same regions display relatively little atrophy and hypometabolism in patients with Alzheimer's disease.See Clark and Warren (doi:10.1093/awv148) for a scientific commentary on this article

Nonlinear Dynamical Systems as Enablers of Exploratory Engagement with Musical Instruments

This paper presents a small scale study that examined links between the inclusion of nonlinear dynamical processes in musical tools and particular kinds of engagement. Communication-oriented attitudes to engagement that view the tool as a medium for transmission of ideas are contrasted with material-oriented attitudes that focus on the specific sonic properties and behaviours of a given tool, and the latter are linked to the inclusion of nonlinear dynamical elements. Methodological issues are raised and discussed, particularly with regard to the holistic nature of musical instruments, the difficulties of independently testing isolated design elements, and potential methods for addressing these difficulties

Multiparametric interfaces for fine-grained control of digital music

Digital technology provides a very powerful medium for musical creativity, and the way in which we interface and interact with computers has a huge bearing on our ability to realise our artistic aims. The standard input devices available for the control of digital music tools tend to afford a low quality of embodied control; they fail to realise our innate expressiveness and dexterity of motion. This thesis looks at ways of capturing more detailed and subtle motion for the control of computer music tools; it examines how this motion can be used to control music software, and evaluates musicians’ experience of using these systems. Two new musical controllers were created, based on a multiparametric paradigm where multiple, continuous, concurrent motion data streams are mapped to the control of musical parameters. The first controller, Phalanger, is a markerless video tracking system that enables the use of hand and finger motion for musical control. EchoFoam, the second system, is a malleable controller, operated through the manipulation of conductive foam. Both systems use machine learning techniques at the core of their functionality. These controllers are front ends to RECZ, a high-level mapping tool for multiparametric data streams. The development of these systems and the evaluation of musicians’ experience of their use constructs a detailed picture of multiparametric musical control. This work contributes to the developing intersection between the fields of computer music and human-computer interaction. The principal contributions are the two new musical controllers, and a set of guidelines for the design and use of multiparametric interfaces for the control of digital music. This work also acts as a case study of the application of HCI user experience evaluation methodology to musical interfaces. The results highlight important themes concerning multiparametric musical control. These include the use of metaphor and imagery, choreography and language creation, individual differences and uncontrol. They highlight how this style of interface can fit into the creative process, and advocate a pluralistic approach to the control of digital music tools where different input devices fit different creative scenarios

A History of Audio Effects

Audio effects are an essential tool that the field of music production relies upon. The ability to intentionally manipulate and modify a piece of sound has opened up considerable opportunities for music making. The evolution of technology has often driven new audio tools and effects, from early architectural acoustics through electromechanical and electronic devices to the digitisation of music production studios. Throughout time, music has constantly borrowed ideas and technological advancements from all other fields and contributed back to the innovative technology. This is defined as transsectorial innovation and fundamentally underpins the technological developments of audio effects. The development and evolution of audio effect technology is discussed, highlighting major technical breakthroughs and the impact of available audio effects

Pocket gamelan: tuneable trajectories for flying sources in Mandala 3 and Mandala 4

This paper describes two new live performance scenarios for performing music using bluetooth-enabled mobile phones. Interaction between mobile phones via wireless link is a key feature of the performance interface for each scenario. Both scenarios are discussed in the context of two publicly performed works for an ensemble of players in which mobile phone handsets are used both as sound sources and as hand-held controllers. In both works mobile phones are mounted in a specially devised pouch attached to a cord and physically swung to produce audio chorusing. During performance some players swing phones while others operate phones as hand-held controllers. Wireless connectivity enables interaction between flying and hand-held phones. Each work features different bluetooth implementations. In one a dedicated mobile phone acts as a server that interconnects multiple clients, while in the other point to point communication takes place between clients on an ad hoc basis. The paper summarises bluetooth tools designed for live performance realisation and concludes with a comparative evaluation of both scenarios for future implementation of performance by large ensembles of nonexpert players performing microtonal music using ubiquitous technology

The Role of Nonlinear Dynamics in Musicians' Interactions with Digital and Acoustic Musical Instruments

Nonlinear dynamical processes are fundamental to the behaviour of acoustic musical instruments, as is well explored in the case of sound production. However, such processes may have profound and under-explored implications for how musicians interact with instruments. While nonlinear dynamical processes are ubiquitous in acoustic instruments, they are present in digital musical tools only if explicitly implemented. Thus, an important resource with potentially major effects on how musicians interact with acoustic instruments is typically absent in the way musicians interact with digital instruments. 24 interviews with free improvising musicians were conducted to explore the role that nonlinear dynamics play in the participants’ musical practices, and to understand how such processes can afford distinctive methods of creative exploration. Thematic analysis of the interview data is used to demonstrate the potential for nonlinear dynamical processes to provide repeatable, learnable, controllable and explorable interactions, and to establish a vocabulary for exploring nonlinear dynamical interactions. Two related approaches to engaging with nonlinear dynamical behaviours are elaborated: edge-like interaction which involves the creative use of critical thresholds; and deep exploration which involves exploring the virtually unlimited subtleties of a very small control region. The elaboration of these approaches provides an important bridge that connects the concrete descriptions of interaction in musical practices on the one hand, to the more abstract mathematical formulation of nonlinear dynamical systems on the other