This paper outlines technical and aesthetic approaches to sound spatialisation for electroacoustic music composition. In particular, the paper discusses how spatialisation (sound diffusion) is used to realise specific musical objectives. Technological solutions to problems associated with adapting multichannel compositions for live spatialisation are explored, with particular reference to the open-source Resound system [2, 3]. Examples of Resound applications are provided to illustrate the potential of the system for controlling complex spatial behaviour during live performance

Mooney, J

Stefani, E

Ewan Stefani

James Mooney

University of Michigan Library Repository

Spatial Composition in the Multi-Channel Domain: Aesthetics and Techniques

White Rose Research Online

This is a repository copy of Spatial composition in the multi-channel domain: aesthetics and techniques.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/80581/Version: Published VersionProceedings Paper:Stefani, E and Mooney, J (2009) Spatial composition in the multi-channel domain: aesthetics and techniques. In: Proceedings of the International Computer Music Conference 2009. International Computer Music Conference, 16-19 Aug 2009, Montreal, Canada. International Computer Music Association . eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. SPATIAL COMPOSITION IN THE MULTI-CHANNEL DOMAIN:AESTHETICS AND TECHNIQUESDr Ewan StefaniSchool of Music / ICSRiMUniversity of Leeds, UKe.j.stefani@leeds.ac.ukDr James MooneyCulture LabNewcastle University, UKjames.mooney@ncl.ac.ukABSTRACTThis paper outlines technical and aesthetic approachesto sound spatialization for electroacoustic music composi-tion. In particular, the paper discusses how spatialization(sound diffusion) can be used to realize specific musicalobjectives. Technological solutions to problems associatedwith adapting multichannel compositions for live spatial-ization are explored, with particular reference to the open-source Resound system [3, 4]. Some preliminary examplesof Resound applications are provided to illustrate the poten-tial of the system for controlling complex spatial behaviourduring live performance.1. INTRODUCTIONThere is an emerging debate in the sonic arts community[1, 2, 5, 6, 8] concerning the expressive potential and musi-cal function of multi-channel spatialization. This paper willdiscuss the musical objectives behind novel approaches tospatialization (sound diffusion) and the techniques that havebeen developed to realize such objectives. Three areas ofpreliminary investigation are outlined here:1. Strategies to support the realization of compositionalaims by dynamic reallocation of fixed material to gr-oups of loudspeakers and the control of sophisticatedspatial behaviour during live performance using theResound system.2. Development of compositional techniques linked tosound diffusion, including using acoustic space andloudspeaker placement to remix or balance textural(‘vertical’) layers within the music, and embed a spa-tial dimension into the (temporal) composition of rhy-thmic, harmonic, melodic and spectral morphologies.3. An aesthetic approach to sound diffusion, which em-phasizes the customization of techniques for individ-ual performance venues. In particular, the consider-ation of the acoustic properties of the space and theplacement of loudspeakers in relation to the audienceare highlighted to ensure that the aesthetic objectivesof the work are realized (and enhanced) in live perfor-mance.2. APPROACHES TO SPATIALIZATION2.1. OverviewThe objectives for the investigation of spatialization in thiscontext can be summarized in the following:• Using the inherent flexibility of Behaviours in Re-sound to enable a music-led approach to spatializa-tion, as opposed to conventional hardware techniquesthat are constrained or compromized by the physicallimitations of the mixing console.• Taking a site-specific approach to the placement anddistribution of sonic material, enabling the composerto investigate the expressive potential of a particularperformance space by exploiting acoustic features andphysical dimensions of the venue.• Developing techniques for the successful transposi-tion of a composition to different performance spacesvia customization of Behaviours in Resound.• Composing musical material to investigate the poten-tial of Resound behaviours.• Consideration of the audience listening experience be-yond the ‘sweet spot’ within the venue, using simul-taneous and autonomous movement of multiple audiochannels to create shifts in perspective. This may bea particularly effective technique for varying the bal-ance of component musical parts as passages of com-plex music are reiterated, for example.• Revealing composed structures, textures and other mu-sical details to the audience via the spatial separationor autonomous spatial behaviour of individual sub-mixed ‘stems’ within a composition.3. COMPOSED SPACE, DIFFUSED SPACE ANDTHE LIMITATIONS OF THE MIXING DESKWhen composing we are dealing (amongst other things) withthe distribution of audio material among a finite number ofsource channels; two for stereo, four for quad, et cetera.Material might be distributed among the source channelsby way of panning, or via counterpoint between channels,for example. Smalley refers to this, ‘the space as com-posed on to the recorded media,’ as the ‘composed space’[7]. However, the final spatial experience depends on boththe placement of the loudspeakers and the way in which thesource channels are routed and diffused to them; the ‘dif-fused space.’ Thus the overall outcome is a combination ofboth compositional and performative elements.A site-specific approach to multi-channel compositionand sound diffusion requires the composer to gather data onthe acoustics of the space and plan where the loudspeakerswill be positioned. Practical considerations of cable man-agement and safety will also be important at this initial stage.The realistic potential of the space may then be factored intothe organization of compositional material. For example,the space may be used to enhance a phenomenological im-pression of ‘intimacy’ or ‘distance.’ This can be exploredby placing small loudspeakers close to the audience (or un-derneath seats) and larger loudspeakers off-stage or facingaway from the audience. Exploiting the dramatic effect ofhidden loudspeakers may also be possible, so that soundssuddenly emerge from a new location within the space atkey moments within the composition.In performance, we diffuse our fixed number of sourcechannels to a (usually) larger number of loudspeakers. Thus,we have a choice with respect to which source channels aresent to which loudspeakers. However, when diffusion is car-ried out using a standard mixing desk (as is often the case),the relationship between source channels and loudspeakersis limited, with each fader controlling the attenuation of onepredetermined source-channel-to-loudspeaker (i.e. input-to-output, or I/O) routing. It is only normally possible to haveone I/O routing per fader, making it difficult to affect com-plex or rapid transitions, particularly with more than twosource channels. Both of these factors limit the extent towhich multi-channel composition and diffusion techniquescan be explored. If we are to adopt a site-specific approachand embrace the process of diffusion as one that extends theprocess of composition into the performance space, then thepaucity of the mixing desk as an expressive tool is an in-hibiting factor.4. RESOUND APPLICATIONSThe following sections focus on specific multi-channel dif-fusion scenarios, their implementation in Resound, and asummary of the spatio-aesthetic explorations that the Re-sound system enables.4.1. Spatial Redistribution of Multi-channel SourcesIn Resound, each fader can simultaneously control multipleI/O routings. In Figure 1 (i) the four channels of a quadra-phonic source can be diffused to four different groups offour loudspeakers using only four faders. The same con-figuration would require sixteen faders on a normal audiomixing desk; not to mention a super-human level of dexter-ity! The Resound system permits a fuller exploration of theintersection between composed space and diffused space.Figure 1. Control of four different quadraphonic source toloudspeaker mappings using: (i) grouped mappings on fourfaders; (ii) a Multipoint Crossfade Behaviour on two faders.Resound also allows diffusion to be controlled semi-automatically via a suite of algorithmic spatialization rou-tines called Behaviours. Figure 1 (ii) demonstrates howthe same four I/O-mapping groups can be mapped on totwo faders using the ‘Multipoint Crossfade’ Behaviour. Thefader on the left crossfades between each of the four groupsof I/O routings in turn, while the fader on the right (shownhere at constant maximum level) controls overall amplitude.Of course the examples chosen here are arbitrary: in areal performance the grouping of I/O-mappings would be anaesthetic one. Some less obvious I/O mappings are shownin Figure 2. Such examples demonstrate the potential of Re-sound to enable the composer to incorporate sophisticatedand dramatic alterations during live performance. Groups ofloudspeakers may be given distinct musical identities, suchas a particular range of articulations or a family of timbres.Such mappings of musical, spectral and morphological char-acteristics to groups of loudspeakers may be fixed for a pe-riod of time during the composition, until they are likely tohave become familiar to the audience. For dramatic effect,or to emphasize a major structural event, the mappings canthen be altered at key moments within the piece.The Multipoint Crossfade can essentially map any spa-tial trajectory on to a single fader. The sequence can incor-porate any number of channels and is therefore particularlyappropriate for the manipulation of multi-channel sources.Figure 2. Some potential (arbitrary) source-channel-to-loudspeaker mappings for a quadraphonic source.4.2. RotationThe examples in Figure 3 illustrate two ways in which ro-tations of multi-channel source material could be achieved.Configuration (i) uses one fader for each stage of the rota-tion, a smooth transition being achieved by manual cross-fading between the four faders.Figure 3. Rotating a quadraphonic source using: (i)grouped mappings on four faders; (ii) a Multipoint Cross-fade Behaviour with two controls.Or, as in configuration (ii), the four I/O-mapping groupsthat constitute the rotation could be mapped on to one con-trol using the Multipoint Crossfade Behaviour. Resounddoes not have to operate with faders; any MIDI or OSC con-troller can be used, and a rotary control as pictured abovewould be intuitive in this case.As noted previously, the aesthetic outcome of these tech-niques will depend fundamentally upon the compositionalapproaches taken in building the (in this case four-channel)source material. For example, in the composition ThreeSpaces[9], the Multipoint Crossfade Behaviour is used to ro-tate four musical parts simultaneously between three groupsof loudspeakers in the space to enable gradual changes inperspective to be investigated. The rate of movement iscontrolled manually and varies according to the size of thevenue and diffusion system.4.3. Dynamic Control of Multi-channel SpreadFigure 4 illustrates how two faders – again, configured asMultipoint Crossfades – can be used to dynamically shiftthe ‘front’ and ‘rear’ planes of a four-channel source. Fader(i) controls the positioning of source channels 1 and 2 viathree pairs of loudspeakers in the front half of the hall, whilefader (ii) does the same thing for source channels 3 and 4for three further loudspeaker pairs towards the rear. Thisaffords intuitive control over the spatial separation of theconstituent channels within the venue. The end result willdepend upon how the four source channels have been usedcompositionally. In the case of a single, homogeneous four-channel image – that is, if panning has been used across allfour channels – then this configuration can be used to manip-ulate the volume of space that the auditory image occupieswhilst maintaining intact the internal spatial relationshipswithin it.1 If, on the other hand, the four channels have beentreated individually, as four discrete monophonic sources,then this setup affords us control over four quite distinctzones of sound within the venue. This kind of explorationtakes us beyond the conventional ‘sweet spot’ paradigm andcan be particularly effective when loudspeakers have beenpositioned so as to emphasize the acoustic characteristics ofthe venue.Figure 4. Using two Multipoint Crossfade Behaviours todynamically shift the front and rear planes of a quadra-phonic source.4.4. Random and Sequential Spatial DistributionThe Mexican Wave Behaviour crossfades automatically thr-ough a predefined set of I/O mappings in sequence. Wecould, for example, iterate through all of the I/O mappingsillustrated in Figure 2 at a given (fader assigned) frequencyand overall amplitude. The Random Behaviour is similar,only the iteration is random rather than sequential.As an illustration of this process, in Klangfarben Study[10], musical lines are broken-up and distributed betweenfour discrete groups of electroacoustic sounds which are eachseparated spatially around a large concert venue. By cross-fading through a sequence of I/O mappings, Resound is usedto alter the spatial position of each group of sounds inde-pendently, so that the relative positions of musical eventsvary dynamically at key moments within the piece. This Be-haviour is also effective for exposing individual lines within1Some will argue that this kind of diffusion does not maintain imageintegrity in terms of accurate phantom imaging. This is of course true,however we can at least ensure that the orientation of the spatial axes re-mains consistent.polyphonic textures and revealing the decomposition of ch-ords or complex spectra. Each component within the musi-cal texture (or spectrum) can be moved individually to pre-defined locations within the space, to maximize the expres-sive potential of a particular concert venue. The ability tomodify such Behaviours with a small number of physicalcontrols enables the composer to ‘orchestrate’ any numberof sophisticated spatial sequences and interactions duringlive performance of the work. In this way, Resound opens upnew avenues for exploration within the multi-channel elec-troacoustic work.5. CONCLUSIONSThe studio mixing desk is in many ways an inappropriatetool for live sound diffusion, and as a consequence we arelimited in what we can achieve with it within the restrictedconfines of our own dexterity. This has adversely affectedour creative ambition with regard to spatial composition anddiffusion, and as a consequence much territory remains un-or under-explored, particularly in the multi-channel domain.By allowing multiple I/O-to-fader mappings and semi-automated spatialization Behaviours, Resound enables a ful-ler exploration of the spatio-performative potential of themulti-channel electroacoustic work than could otherwise beachieved with conventional mixing hardware. In particu-lar it has been shown that Resound provides the tools withwhich multiple independent sources can be diffused simul-taneously2, and with which homogeneous multi-channel im-ages can be presented, adapted and spatially redistributed inreal time. This permits the composer to explore the uniqueexpressive potential of the performance space to maximizethe dramatic elements of the music. The composer mayalso consider the audience listening experience beyond a‘sweet spot’ within the venue, using independent movementof multiple audio channels to create dynamic shifts in per-spective or focus.Resound has become a powerful tool for the successfultransposition of a composition to different performance en-vironments. Typically, when performing a work in a newvenue (and on an unfamiliar diffusion system), the acous-matic composer must significantly reorganize multi-channelaudio material to achieve a satisfactory spatialization of themusic in the space that has been provided. The advantageoffered by Resound is that once an outline for spatializationhas been composed by organising the music into discrete‘stems’ (where material is arranged according to musicalcriteria), these parts may be dynamically re-mapped duringlive performance to fully exploit the acoustic properties ofthe space and the layout of the diffusion system.It is often the case that electroacoustic musicians com-pose their music with all of the realities of the sound dif-2For a discussion of the live spatialization of multiple live sources in animprovisational context, see [2]fusion context borne in mind. Where constraints exist –with the venue, the system, the loudspeakers, and so on –this is taken into account, in one way or another, in thecompositional decisions made. Hence, the limitations ofthe diffusion system impact upon composition. In redefin-ing the possibilities of sound diffusion the Resound sys-tem thus opens up unforeseen possibilities in the compo-sitional domain, particularly with respect to multi-channelwork. Further development of the system will benefit enor-mously from detailed input from composers, and collabora-tions are currently being sought for future work in this area.6. REFERENCES[1] D. Berezan, “In flux: A new approach to sound dif-fusion performance practice for fixed media music,”in Proceedings of the International Computer MusicConference, Belfast, UK, 2008.[2] J. Mooney, P. Bell, and A. Parkinson, “Sound spa-tialisation, improvisation and ambiguity,” in Proceed-ings of the Sound and Music Computing Conference,Berlin, Germany, 2008.[3] J. Mooney and D. Moore, “A concept-based system forthe live diffusion of sound via multiple loudspeakers,”in Proceedings of the Digital Music Research NetworkConference, Leeds, UK, 2007.[4] ——, “Resound: open-source live sound spatialisa-tion,” in Proceedings of the International ComputerMusic Conference, Belfast, UK, 2008.[5] A. Moore, D. Moore, and J. Mooney, “M2 diffusion:The live diffusion of sound in space,” in Proceedings ofthe International Computer Music Conference, Miami,FL., USA, 2004.[6] F. Ontodo, “Contemporary trends in the use of spacein electroacoustic music,” in Organized Sound, vol.13(1), 2008, pp. 77–81.[7] D. Smalley, “Spectromorphology: explaining soundshapes,” in Organized Sound, vol. 2(2), 1997, pp. 107–126.[8] N. Stavropoulos, “Multi-channel formats in electroa-coustic composition: Acoustic space as a carrier ofmusical structure,” in Proceedings of the Digital MusicResearch Network Conference, London, UK, 2006.[9] E. Stefani, Three Spaces. Available online athttp://www.leeds.ac.uk/cepra/media.html, 2008.[10] ——, Klangfarben Study. Available online athttp://www.leeds.ac.uk/cepra/media.html, 2009.

Spatial composition in the multi-channel domain: aesthetics and techniques

https://eprints.whiterose.ac.uk/80581/1/ES-JM-ICMC09.pdf

Spatial composition in the multi-channel domain: aesthetics and techniques

Abstract

Similar works

Full text

Available Versions

University of Michigan Library Repository

White Rose Research Online