Resound is an open source cross-platform software tool for real time multi-channel sound spatialisation. This paper gives a non-technical account of the key capabilities of the system, predominantly from the standpoint of live performance in electroacoustic music, with specific examples of how the various functionalities might be used

Mooney, JR

Moore, D

White Rose Research Online

This is a repository copy of Resound: open-source live sound spatialisation.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/80584/Version: Published VersionProceedings Paper:Mooney, JR and Moore, D (2008) Resound: open-source live sound spatialisation. In: Proceedings of the International Computer Music Conference 2008. International Computer Music Conference 2008, 01 Aug 2008, Belfast, UK. International Computer Music Association . eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. RESOUND: OPEN-SOURCE LIVE SOUND SPATIALISATIONJames MooneyNewcastle UniversityCulture LabDavid MooreUniversity of SheffieldDepartment of MusicABSTRACTResound is an open source cross-platform software toolfor real time multi-channel sound spatialisation. This pa-per gives a non-technical account of the key capabilities ofthe system, predominantly from the standpoint of live per-formance in electroacoustic music, with specific examplesof how the various functionalities might be used.1. BACKGROUNDThe Resound system has its roots in electroacoustic music,where the expression ‘sound diffusion’ describes the prac-tice by which music from fixed media (CD etc.) is pre-sented to an audience via loudspeakers in a performancecontext. 1 Typically the source is stereophonic but, be-cause performance venues are comparatively large, morethan two loudspeakers are needed, requiring duplicationof the audio signals. Thus arises the necessity of sounddiffusion, and with it an entire aesthetic and performancepractice [1] [3] [11] [12].Traditionally sound diffusion has been carried out us-ing standard audio mixing desks as both the means ofduplicating signals and as the performance interface, thefaders being used to distribute the source channels amongmultiple loudspeaker pairs. Although widely adopted thisapproach has its limitations, particularly in terms of in-terface ergonomics and flexibility of input-to-output map-ping. The M2 sound diffusion system [4] [7] [8] soughtto address these issues by making more ergonomic useof the familiar fader interface via a dynamic matrix-basedmix engine implemented in software. Four years of expe-rience with M2 have revealed new aspects of performancepractice, as well as highlighting areas in which further im-provements and optimisations could be made.2. SYSTEM DESCRIPTIONResound comprises two software programs: a server ap-plication and a graphical user interface (GUI). The serverdeals with all signal processing, with live audio inputsand loudspeakers being connected to a multi-channel au-dio interface. Audio playback directly from the server is1 The term ‘sound diffusion’ is used to connote the practice specif-ically in the context of electroacoustic music. The more general term‘sound spatialisation’ is used to imply that the practice might apply morewidely; in theatre, installation, virtual reality or scientific experiment forexample. See http://culturelab.ncl.ac.uk/auditoryenvironments.also possible. The GUI allows the performer to interactwith the system and control spatialisation using an OSCor MIDI controller device. 2 The server and GUI applica-tions can both run on the same computer, or on separatemachines with communication via Ethernet. The follow-ing sections summarise the functionality of the system.3. ONE FADER TO ONE LOUDSPEAKERResound can be configured to emulate the operation ofa mixing desk based system. Here, the routing of inputchannels to loudspeakers is fixed and each fader merelycontrols the output level of a single loudspeaker (hence‘one fader to one loudspeaker’). This is useful as manypractitioners are already familiar with this setup.4. GROUPED LOUDSPEAKERSA single fader can be used to simultaneously control thelevels of a group of loudspeakers. In this way a stereosource could be diffused to a specific pair of loudspeak-ers, or to a group of four, or indeed to any combinationof loudspeakers, without having to manipulate multiplefaders.5. PROPORTIONAL GROUPSFaders controlling multiple output levels can be config-ured to do so in variable relative amplitudes. A singlefader could control the diffusion of a stereo source to allthe loudspeakers, as described previously, but with a biasin loudness towards, say, the front pair of loudspeakers.6. DYNAMIC INPUT TO OUTPUT MAPPINGIn Figure 1 the ‘left’ channel of a stereo source is mappedto loudspeakers 1, 3, 5 and 7, and the ’right’ to 2, 4, 6and 8. It is common practice for the mapping of inputchannels to loudspeakers to be fixed in this way, and thisis often ultimately due to hardware constraints. AlthoughResound can emulate this if required it is in fact possibleto route any input channel to any loudspeaker. For exam-ple, one could define a routing in which the left channelof a stereo source is routed to loudspeaker 8 and the right2 For convenience we will refer to ‘faders’ as the control methodthroughout this article. Of course, any OSC or MIDI device could beused.Figure 1. Fixed source channel mapping common in tra-ditional stereo diffusion systems.channel to speaker 1, inverting the normal left-right axis.As with all of the previous examples, this mapping can becontrolled with a single fader and implemented alongsideother mappings.7. ADDITIVE SPATIALISATIONAs a simple scenario let’s say that fader a controls the levelof a stereo source mapped to the main pair of loudspeak-ers, while fader b controls the level of the same sourcemapped to all of the loudspeakers within the array. If weuse both faders together, their effects will be mixed to-gether: fader b diffuses the stereo source ‘everywhere,’and we can boost the level of the main pair of loudspeak-ers further by additionally raising fader a. This can bereferred to as additive spatialisation.8. SUBTRACTIVE SPATIALISATIONIt is also possible for a fader to have a subtractive influ-ence, such that raising the fader will progressively reducethe level of a given set of loudspeakers. If, for instance,fader c is configured to act subtractively on loudspeakers3 through 8, then raising fader c will reduce the levelsof these six loudspeakers without affecting the levels ofloudspeakers 1 and 2. We might describe this particularsubtractive routine as an ‘all to mains’ transition. It is theability to perform manipulations additively and subtrac-tively that differentiates Resound from other matrix basedsystems such as the DM-8 [10].9. SEMI-AUTOMATED SPATIALISATIONBEHAVIOURSIn all of the examples so far there has been a simple pro-portionality - either direct (additive) or inverse (subtrac-tive) - between fader position and resulting output lev-els. Resound also provides a framework for defining semi-automated spatialisation ‘behaviours’ whereby the fadersare used to control parameters other than loudness. Someexamples are as follows.9.1. Mexican WaveThe Mexican Wave behaviour cycles through a sequenceof loudspeakers, or a sequence of groups of two or moreloudspeakers. The user can control the rate at which thesequence is iterated (including the direction of iteration)and the overall amplitude level by assigning these param-eters to faders. The Mexican Wave behaviour can be used,for instance, to automate front-to-back or back-to-frontmovement within a venue; rotational movement is also apossibility.9.2. RandomThe Random behaviour also acts upon a group of loud-speakers, but iterates through the group in a random orderas opposed to sequentially. Again, the user can controlthe rate at which loudspeakers are selected, and overallamplitude level of the behaviour. This behaviour can beparticularly effective for the spatialisation of textural orgranular material where an erratic spatial effect is sought.9.3. Multi-point CrossfadeThe Multi-point Crossfade behaviour is similar to the Mex-ican Wave, however in this case the behaviour does notiterate automatically through the sequence. Instead, theactive loudspeaker(s) at any given time is governed by theposition of a fader. This behaviour might be useful, forexample, as a way of adjusting the width of stereo im-ages. Traditionally, this would be done by cross-fading thesource between ‘main’ and ‘wide’ pairs of loudspeakers,necessitating the use of four faders. With the Multi-pointCrossfade the same operation can be carried out with onefader.10. PUT IT ALL TOGETHER...All of the above techniques can be combined within asingle performance. A thirty-two fader OSC controllermight comprise, for instance, eight faders that operate onspecific loudspeakers individually, eight that operate ongroups of two, eight that operate on groups of four, and afurther eight configured with semi-automated behaviours.Because all interactions within the Resound system canbe applied additively or subtractively, there is no conflictbetween faders that act upon the same loudspeakers: theresults are simply summed (positively or negatively). Thisis particularly useful in the context of semi-automated be-haviours, which can be used to dramatic effect on theirown or subtly mixed in, as appropriate to the musical con-text.11. COLLECTIVESSystems based around mixing desk hardware tend to poseproblems with multi-channel sources, owing to the limita-tions of the ‘one fader to one loudspeaker’ paradigm. AsHarrison notes:If you’ve got an eight-channel source, and ev-ery channel of the eight has a fader, how doyou do crossfades? You haven’t got enoughhands! [5]The M2 system improved interface ergonomics by al-lowing multiple mappings to be grouped, but this intro-duced secondary problems in that such configurations takea long time to set up. Resound addresses this by introduc-ing the ‘collective’ [6] as new method of dealing with mul-tiple assignments, allowing these to be made more quicklyand efficiently.With collectives it is thus perfectly feasible to set upspatialisation routines that deal with 5.1, 8, or n channelimages, presenting markedly different performance op-portunities from those afforded by stereo. Experimentswhereby eight-channel images are spatialised among mul-tiple groups of eight loudspeakers have been carried outat BEAST. 3 Resound is of course able to cater for thiskind of configuration; however, because Resound dealswith input-to-output mappings dynamically in real time,multi-channel images can also be inverted, translated, ro-tated, and so on. Furthermore, greater-than-stereo imagesusually have a front-to-back axis (as well as left-to-right)that can be manipulated in interesting ways. For example,the ‘front three’ and ‘rear two’ of a 5.1 image could be‘Mexican Waved’ or ‘Multi-point Crossfaded’ indepen-dently between multiple groups of loudspeakers.12. PRESETSA preset is a snapshot of the complete state of the Resoundsystem. Presets can be recalled instantaneously with akey press, and can be used to swap between input sourceswithout the need for physical repatching, or in various cre-ative ways in performance. Saved banks of presets canbe used as a means of sharing spatialisation techniquesamong practitioners.13. SPATIALISATION OF MULTIPLE SOURCESConsider the following scenario.In my last piece [...] I used a six-channelhexagonal array as a ‘surround,’ and I hadtwo solo speakers [...] for close-up stuff. WhatI would like to do [...] is be able to diffusethe ‘twos’ and the ‘sixes’ completely inde-pendently. [5]Resound allows the performer to spatialise multiple in-put sources independently. To realise the scenario de-scribed by Harrison, we might assign one bank of fadersto control only the six-channel image, and another bankto control only the stereo image. Of course both banks offaders could incorporate their own groups and behaviours.A similar scenario has proved effective in improvisationwith multiple live electronic sources [2].3 These experiments are described in Chapter 4 of [4]Figure 2. Graphical representation of a mix matrix archi-tecture. (The lettering of cross-point nodes refers to anexample in the text.)14. HOW DOES IT WORK?In essence Resound allows each fader to control the levelsof one or more input to output (source-channel to loud-speaker) mappings as defined by the user. This is achievedvia the use of a matrix-based mix engine at the heart ofthe server application. Figure 2 describes a mix matrixarchitecture diagrammatically. In this case the matrix hastwo inputs and eight outputs, but in principle it can be anysize. Audio inputs are shown on the left, audio outputsalong the top. Every possible input to output mapping isreferred to as a cross-point. Each node of the matrix hasan attenuator that determines the amplitude of the signalallowed to pass through that node. In this way, any inputcan be mixed to any output in any proportion.The GUI application acts as a remote control for themix matrix, allowing the user to map multiple nodes on tofaders. Mappings can be made directly; for example wemight map matrix nodes a, b, c and d to fader 1, meaningthat fader 1 would simultaneously control the level of ‘L’to outputs 1 and 3, and ‘R’ to 2 and 4. Alternatively wecan map, say, nodes w, x, y and z to a Mexican Wave be-haviour, and then map the behaviour’s amplitude and rateparameters to faders 2 and 3, respectively. So, when wetalk about faders and behaviours acting upon loudspeak-ers, this is not strictly accurate: in effect they do, but inreality they act upon matrix nodes. The GUI applicationalso allows matrix nodes to be locked to a given value.Locked parameters, once set, are not controlled by faders.A detailed technical and conceptual description of thesystem is given elsewhere [6] and source code is availableonline [9].15. MATRIX MIXING TECHNIQUESA common stereo diffusion architecture can be emulatedby configuring the matrix as illustrated in Figure 3(a). (Thenumbering of the output nodes corresponds with that ofthe loudspeakers in Figure 1.) Here, the input nodes arelocked at maximum amplitude, as are alternating ‘left’ and‘right’ cross-point nodes. We then assign each of the out-put nodes to an individual fader. The same configurationcan also be realised as shown in Figure 3(b). In this caseall of the input and output nodes are locked, and alternat-ing cross-point nodes are assigned to faders. In these sim-ple scenarios there will be no difference to the performer:a ‘one fader to one loudspeaker’ model with fixed left andright stereo mappings is emulated in both cases. Scenario(b) is, however, more flexible. Because we are remotelycontrolling the cross-point nodes as opposed to outputswe have complete dynamic control over the mapping ofsource channels to loudspeakers.Although relatively conservative examples have beengiven for ease of explanation, the same principles can beapplied to achieve more unorthodox mappings across largernumbers of input and output channels. Through com-bining cross-point spatialisation with the ability to mapmultiple nodes simultaneously to faders, performers cancircumnavigate the potentially cumbersome nature of themix matrix architecture and exploit its latent flexibility.16. FUTURE DIRECTIONSAn embedded scripting language for defining behaviourswill increase accessibility and increase the creative poten-tial of the system. High level languages such as Pythonand Lua are being considered.Augmenting the DSP functionality of each matrix node,perhaps by providing filtering and time-delay algorithms,is a relatively straightforward extension of the current ca-pabilities of the system.Massively multi-channel systems (with two-hundred ormore loudspeaker outputs) require exponentially increas-ing processor power and I/O capability. Systems such asResound adopt a monolithic approach. A clustered com-puting solution would be more scalable and is currentlybeing designed.In the absence of permanent installations, rehearsal,software development and performance documentation islogistically difficult. Virtualisation of large arrays via Am-bisonic and binaural implementations is planned.17. CONCLUSIONResound provides a simple but powerful user-configurableframework for multi-loudspeaker sound spatialisation andis particularly well suited to real-time interaction and im-provisation in a performance context. The ability to spec-ify and control mappings in groups, proportionally, addi-tively, subtractively and semi-automatically via behavioursallows performers to realise dynamic, flexible, and easyto control spatialisation routines that would be difficult orimpossible to achieve in real time with analogue mixinghardware, particularly with multi-channel input sources.Because the system can operate additively and subtrac-tively, unorthodox or experimental methods can be usedalongside more traditional fixed routings, allowing per-formers to adapt gradually, supplementing and extendingtheir existing repertoire of spatialisation techniques. Ef-fectively, performers are at liberty to design their spatiali-sation instrument as appropriate to the music and accord-ing to their personal preference. Above all, however, Re-sound is a work in progress, and an open source projectthat actively encourages collective participation and on-going development.Figure 3. Two ways of emulating fixed stereo routing.Locked nodes are square; nodes assigned to fader controlare black circles.18. REFERENCES[1] Barrière, F. and Bennett, G. [eds]. Composi-tion / Diffusion in Electroacoustic Music, Edi-tions Mnemosyne, Bourges, 1998.[2] Dead Dialogues: Sound Spatialisation inFree Improvisation. http://video.google.com/videoplay?docid= 47006225079839391[3] Harrison, J. “Sound, Space, Sculpture - SomeThoughts on the ’What,’ ’How’ and ’Why’ ofSound Diffusion, Organised Sound 3(2), 1998.[4] Mooney, J. Sound Diffusion Systems for theLive Performance of Electroacoustic Music,Ph.D. Thesis, University of Sheffield, 2005.http://www.james-mooney.co.uk/publications.[5] Mooney, J. [ed]. “An Interview with ProfessorJonty Harrison”, Appendix 2 in [4].[6] Mooney, J. and Moore, D. “A Concept-BasedModel for the Diffusion of Sound via Multi-ple Loudspeakers”, Proceedings of the DigitalMusic Research Network Conference, Leeds,UK, 2007.[7] Moore, A., Moore, D. and Mooney, J. “M2Diffusion: The Live Diffusion of Soundin Space”, Proceedings of the InternationalComputer Music Conference, Miami, 2004.[8] Moore, D. Real-time Sound Spatializa-tion, Software Design and Implementation,Ph.D. Thesis, University of Sheffield, 2004.http://music.dept.shef.ac.uk/phd thesis/rtss.pdf.[9] Resound. http://resound.sourceforge.net.[10] Rolfe, C. “A Practical Guide to Diffusion”,EContact! 2(4), 1999. http://cec.concordia.ca/econtact/Diffusion/pracdiff.htm.[11] Smalley, D. “Space-form and the AcousmaticImage”, Organised Sound 12(1), 2007.[12] Wishart, T. “Spatial Motion”, Ch. 10 in OnSonic Art. Routledge, London, 1996.

Resound: open-source live sound spatialisation

James Mooney

David Moore

University of Michigan Library Repository

RESOUND: OPEN-SOURCE LIVE SOUND SPATIALISATION

http://hdl.handle.net/2027/spo.bbp2372.2008.081

Resound: open-source live sound spatialisation

Abstract

Similar works

Full text

Available Versions

White Rose Research Online

University of Michigan Library Repository