Sonifying the Scene: re-framing and manipulating meaning through audio augmentation

Digital locative music technologies are transforming the ways in which we are able to manipulate and re-frame the meaning of architecture, landscape and art. In this note we explore and outline some of the key features that are associated with this. Defining the future possibilities and challenges that we have identified through our re-search in the area. Our work is supported by examples and critically assesses the relationship between physical and aural presence, examining the manipulation and re-framing of meaning through audio augmentation. “Is Space Audible?” [7]

Sonifying the Scene: re-framing and manipulating meaning through audio augmentation

Digital locative music technologies are transforming the ways in which we are able to manipulate and re-frame the meaning of architecture, landscape and art. In this note we explore and outline some of the key features that are associated with this. Defining the future possibilities and challenges that we have identified through our re-search in the area. Our work is supported by examples and critically assesses the relationship between physical and aural presence, examining the manipulation and re-framing of meaning through audio augmentation. “Is Space Audible?” [7]

^muzicode$: composing and performing musical codes

We present muzicodes, an approach to incorporating machine-readable ‘codes’ into music that allows the performer and/or composer to flexibly define what constitutes a code, and to perform around it. These codes can then act as triggers, for example to control an accompaniment or visuals during a performance. The codes can form an integral part of the music (composition and/or performance), and may be more or less obviously present. This creates a rich space of playful interaction with a system that recognises and responds to the codes. Our proof of concept implementation works with audio or MIDI as input. Muzicodes are represented textually and regular expressions are used to flexibly define them. We present two contrasting demonstration applications and summarise the findings from two workshops with potential users which highlight opportunities and challenges, especially in relation to specifying and matching codes and playing and performing with the system

Sculpting a mobile musical soundtrack

We present an in-the-wild project to design and study a mobile musical soundtrack that enhances the experience of visiting a sculpture park. As with soundtracks for films and games, the goal was to enhance the emotional and narrative aspects of the experience while remaining in the background. We describe a compositional approach in which we first established a broad musical landscape before treating specific exhibits with detailed musical trajectories. Our study reveals how our soundtrack dramatically shaped visitors’ experiences while they remained largely unaware of its operation. We distil seven experiential factors to be addressed by mobile soundtracks alongside ten compositional guidelines

You’ll never walk alone: designing a location-based soundtrack

Musical soundtracks have great potential to enhance mobile walking experiences such as tours and guides, much as they already do for films and games. They also raise new challenges for composers as the music must fit a given landscape and respond to walkers’ trajectories. We present a design for an interactive mobile soundtrack to accompany a visit to a sculpture park. We describe the motivating factors used to structure soundtrack and drive the compositional process

You’ll never walk alone: designing a location-based soundtrack

Musical soundtracks have great potential to enhance mobile walking experiences such as tours and guides, much as they already do for films and games. They also raise new challenges for composers as the music must fit a given landscape and respond to walkers’ trajectories. We present a design for an interactive mobile soundtrack to accompany a visit to a sculpture park. We describe the motivating factors used to structure soundtrack and drive the compositional process

Guidelines for composing locative soundtracks

This thesis investigates the composition of original adaptive musical soundtracks for locative walking activities such as cultural visiting, mobile games and urban, and nature walks; those semi-formal orchestrated walking experiences. This investigation views the ‘soundtrack’ – similarly to those typically found in ‘display’ media experiences such as films and computer games – as an accompaniment rather than the principal feature of the experience. Thus its role is to support and enhance the walking ‘narrative’. In order to best achieve this the soundtrack needs to be heard as congruent and embedded into the activity. This thesis is oriented towards the field of Human Computer Interaction (HCI) and also at composers of such soundtracks; reflecting the thesis’s intention to develop guidelines for locative soundtrack composition that draws upon detailed mappings between musical structure and spatial structure that drives the creation and experience of both. An initial study explored how a group of participants interpreted and responded to different musical features which adapted to their walking routes. This study revealed that participants formed a set of connections between the physical space and the musical structures. These findings were then used to motivate an in-the-field design, composition and deployment of a large-scale adaptive soundtrack for a public cultural visiting experience, which was subsequently experienced by a group of visitors. This study revealed that the soundtrack was considered congruent with the activity and was deeply engaging, quite distinct from a typical visit to this site. These research activities are reflected upon and discussed to distil a framework of guidelines for composing locative soundtracks that is generalizable to other settings and activities

Augmenting a guitar with its digital footprint

We explore how to digitally augment musical instruments by connecting them to their social histories. We describe the use of Internet of Things technologies to connect an acoustic guitar to its digital footprint – a record of how it was designed, built and played. We introduce the approach of crafting interactive decorative inlay into the body of an instrument that can then be scanned using mobile devices to reveal its digital footprint. We describe the design and construction of an augmented acoustic guitar called Carolan alongside activities to build its digital footprint through documented encounters with twenty-seven players in a variety of settings. We reveal the design challenge of mapping the different surfaces of the instrument to various facets of its footprint so as to afford appropriate experiences to players, audiences and technicians. We articulate an agenda for further research on the topic of connecting instruments to their social histories, including capturing and performing digital footprints and creating personalized and legacy experiences

Audio in place: media, mobility & HCI: creating meaning in space

Audio-based content, location and mobile technologies can offer a multitude of interactional possibilities when combined in innovative and creative ways. It is important not to underestimate impact of the interplay between location, place and sound. Even if intangible and ephemeral, sounds impact upon the way in which we experience the built as well as the natural world. As technology offer us the opportunity to augment and access the world, mobile technologies offer us the opportunity to interact while moving though the world. They are technologies that can mediate, provide and locate experience in the world. Vision, and to some extent the tactile senses have been dominant modalities discussed in experiential terms within HCI. This workshop suggests that there is a need to better understand how sound can be used for shaping and augmenting the experiential qualities of places through mobile computing

Considering musical structure in location-based experiences

Locative music experiences are often non-linear and as such the final structure of the music heard is guided by the movements of the user. We note an absence of principles regarding how composers should approach the structuring of such locative soundtracks. For instance, how does one compose for a non-linear soundtrack using linear, pre-composed placed sounds, where fixed musical time is placed into the indeterminate time of the user’s experience? Furthermore, how does one create a soundtrack that is suitable for the location, but also functions as a coherent musical structure? We explore these questions by analyzing an existing ‘placed sound’ work from a traditional music theory perspective and in doing so reveal some structural principals from ‘fixed’ musical forms can also support the composition of contemporary locative music experiences