Playing fast and loose with music recognition

We report lessons from iteratively developing a music recognition system to enable a wide range of musicians to embed musical codes into their typical performance practice. The musician composes fragments of music that can be played back with varying levels of embellishment, disguise and looseness to trigger digital interactions. We collaborated with twenty-three musicians, spanning professionals to amateurs and working with a variety of instruments. We chart the rapid evolution of the system to meet their needs as they strove to integrate music recognition technology into their performance practice, introducing multiple features to enable them to trade-off reliability with musical expression. Collectively, these support the idea of deliberately introducing ‘looseness’ into interactive systems by addressing the three key challenges of control, feedback and attunement, and highlight the potential role for written notations in other recognition-based systems

The gift of the algorithm: beyond autonomy and control

This piece brings together, participation, algorithmic composition and augmentation (as a mechanism by which people can work together to augment and support a composer’s workflow). The performance is about understanding the ways in which composition and performance can be understood, socially, aesthetically and scientifically. This performance becomes a piece of research and design in its own right, a more experimental manifestation of HCI, but it also demonstrates and disrupts conventional production and performance by making the multiple layers of practice and provenance obvious. *See Program notes for a fuller description of the piece for public consumption. We also aim to discuss this further and demo at the Performance workshop that we have submitted. This is part the on going research of the FAST project and aims to engage the wider interdisciplinary Audio Mostly community. • Program notes This piece expands upon Chamberlain’s work into compositional practices that explore autonomy and control, and builds upon the Numbers into Notes system as developed by De Roure. The piece (which is an evolving work) uses the symbolism of the gift to frame parts of the interactions that have occurred in the development of the piece. Individuals are given the chance to create an algorithm. This is made into a physical entity (containing a sequence), which is then gifted to the composer; these together are combined and used to compose a piece. The piece is then performed and given back to the audience (live), of which some members have created the original algorithms. The performance creates a gift, a souvenir, a memento of the experience which some of the audience members can take away. The performance also acts as a way in which we can also understand the interplay between algorithms, art, performance, provenance and participation

Interacting with robots as performers and producers of music

Is it really so strange to think about a robot as something, or perhaps someone that can produce music, as a performer or even as a composer? What happens when robots perform on stage to live audiences, and when they are perceived as intelligent? In this abstract we start to unpack and explicate some of the issues that emerge when the fields of music technology and robotics come together. The aim of this piece of writing is to prompt the Digital Music Research community to engage in debate, in order develop this emerging field of research

Interconnected alchemy: an apparatus for alchemical algorithms

The trope of the fraudulent or occult alchemist, prevalent since the mediaeval period of alchemy’s introduction into European thought, belies the endeavour of practitioners from ancient Egypt onwards. Alchemists used observation, experimentation, and drew conclusions to understand the world around them. Notions of interconnectedness, harmoniousness and codification pervade the alchemical pursuit—and alchemy interconnects literature, art, mathematics, and music....One of our tools is "numbers into notes", a web app for algorithmic composition based on early mathematics, in which the role of the human is to parameterize the algorithm and map number ranges to musical note

Are the robots coming? Designing with autonomy & control for musical creativity & performance

This paper1 expands upon our previous work, and starts to unpack notions of autonomy and control in musical composition and performance-based systems. The term autonomous has become synonymous with technologies such as “autonomous vehicles” and “drones”, while notions of control have mainly been raised in respect to the “control” of industrial systems and in respect to protocols. This position piece disrupts these notions and provides a platform, introducing a more radical proposition in respect to the representation of autonomy and control of features that can be used to design systems that support musical composition and performance. This paper supports a growing interest within the Design, HCI and Artificial Intelligence communities, leading us to think about Human Like Computing systems and the development of a Computational Creativity Continuum

Interaction, instruments and performance: HCI and the design of future music technologies

Rationale There has been little chance for researchers, performers and designers in the UK to come together in order to explore the use and design of new and evolving technologies for performance. This workshop examines the interplay between people, musical instruments, performance and technology. Now, more than ever technology is enabling us to augment the body, develop new ways to play and perform, and augment existing instruments that can span the physical and digital realms. By bringing together performers, artists, designers and researchers we aim to develop new understandings how we might design new performance technologies. Some Topics - Methods and Approaches; What are the methods and approaches that we can employ to understanding interaction and interplay in performance and what impact does technology have on this? - Sonic Augmentation; can performance and sound change the experiential attributes of places, e.g. make them more accessible, more playful? -Physical/digital augmentation; how can one augment one’s self or existing musical instruments and artifacts physically and digitally? - Meaning and Mediation; can people narrate or make sense and movement as part of performance – how does the audience understand this? - Mobility and Immobility; performance and movement, what are the dynamics of performing at rest or whilst mobile, how can technology supported co-located and distributed performance and reception? - Locating Content and Spatialisation; how is performance located, how does sound and performance become part of the spatial fabric and what software tools can support this? - Personalization and Reflection; how can people use new performance technologies to narrate and reflect upon experiences – both as performer and spectator? These are some tentative implications and questions that we expect to address in the workshop. Goals The main goal of the workshop is to bring people together to discuss the issues mentioned previously and to explore this emergent space. As part of Audio Mostly we would like to build this community and develop a network that would engender ongoing participation, debate, scholarship and collaboration. The workshop would also like to encourage early career researchers and PhD students to attend in order to grow the community

Designing the audience journey through repeated experiences

We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience ‘journey’ made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances

The design of future music technologies: ‘sounding out’ AI, immersive experiences & brain controlled interfaces

This paper outlines some of the issues that we will be discussing in the workshop “The Design of Future Music Technologies: ‘Sounding Out’ AI, Immersive Experiences & Brain Controlled Interfaces.” Musical creation, performance and consumption is at a crossroads, how will future technologies be affected by exciting and innovative new developments in artificial intelligence, immersive technologies and developing mechanisms for interfacing with music, such as Brain Controlled systems. In many respects this document acts as a mini survey, made up of supporting material, a bibliography of works and offers a series of quotes from work that has mainly emerged from the FAST Project – see: www.semanticaudio.co.uk

Contesting control: journeys through surrender, self-awareness and looseness of control in embodied interaction

As Human-Computer Interaction (HCI) engages with technologies that sense and actuate the body, there is a need to reconsider the human bodily experience. We present three case studies that each involve different forms of bodily experience: a breath-controlled amusement ride, a brain-controlled film, and an interactive musical duet with a physically actuated piano. We introduce a conceptual framework to describe how control becomes contested between human and computer in such experiences, using the three dimensions of: surrender of control, self-awareness of control, and looseness of control. We reveal how our experiences took users on journeys through control that traversed the space of these dimensions. We propose that our framework is not only relevant to playful cultural experiences, such as those charted in our case studies, but can also inform the design of embodied interaction more widely by emphasising the human experience of control when engaging with autonomous and bodily-focused systems, from future robots and vehicles to today’s gaze, speech and gestural interfaces

Designing the audience journey through repeated experiences

We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience ‘journey’ made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances