This paper presents an implementation for dynamic two or three dimensional spatial distribution of granulated sound (or granular synthesis) over an arbitrary loudspeaker system

Wilson, Scott

English

University of Birmingham Research Portal

  Spatial swarm granulationWilson, ScottCitation for published version (Harvard):Wilson, S 2008, 'Spatial swarm granulation' Paper presented at International Computer Music Conference,1/01/08, .Link to publication on Research at Birmingham portalGeneral rightsUnless a licence is specified above, all rights (including copyright and moral rights) in this document are retained by the authors and/or thecopyright holders. The express permission of the copyright holder must be obtained for any use of this material other than for purposespermitted by law.•	Users may freely distribute the URL that is used to identify this publication.•	Users may download and/or print one copy of the publication from the University of Birmingham research portal for the purpose of privatestudy or non-commercial research.•	User may use extracts from the document in line with the concept of ‘fair dealing’ under the Copyright, Designs and Patents Act 1988 (?)•	Users may not further distribute the material nor use it for the purposes of commercial gain.Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.When citing, please reference the published version.Take down policyWhile the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has beenuploaded in error or has been deemed to be commercially or otherwise sensitive.If you believe that this is the case for this document, please contact UBIRA@lists.bham.ac.uk providing details and we will remove access tothe work immediately and investigate.Download date: 01. Feb. 2019SPATIAL SWARM GRANULATIONScott WilsonUniversity of BirminghamMusic DepartmentABSTRACTThis paper presents an implementation for dynamic two orthree dimensional spatial distribution of granulated sound(or granular synthesis) over an arbitrary loudspeaker sys-tem.1. INTRODUCTIONThe proliferation of large scale heterogeneous multichan-nel loudspeaker systems, such as the Gmebaphone / Cy-berne´phone of the Institut International de Musique Elec-troacoustique de Bourges (IMEB)[2], the Acousmoniumof the Groupe de Recherches Musicales (GRM) [8], andBirmingham ElectroAcoustic Sound Theatre (BEAST) [8],combined with the increasing potential to present ‘mas-sively’ multichannel works has presented us with a uniqueproblem: How to make use of the new multichannel pos-sibilities of such systems, while retaining their traditionaladvantages (e.g. adaptability, flexibility, robustness in avariety of spaces and acoustics). This is the focus of BEAST-Mulch, an AHRC funded research project at the Univer-sity of Birmingham led by the author, along with JontyHarrison and Sergio Luque Ancona.[9]A central problem explored in the project is how to si-multaneously and dynamically control location (and dif-fuseness) of sound, especially given that such systems typ-ically involve large numbers of speakers distributed in threedimensions at various distances from the centre of the lis-tening area. This paper presents one approach to this prob-lem, an implementation of a large-scale multichannel gran-ulator which makes use of swarming algorithms to deter-mine grain location.2. PRE-EXISTINGWORK2.1. BoidsMost work making use of flocking or swarming simula-tions is based upon Craig Reynolds’ 1986 Boids algorithm[6]. Since this approach is well known, I will not discussit in great detail here. Simply put, Reynolds demonstratedthat complex emergent behaviour, such as that observedin flocks of birds, schools of fish, swarms of insects, etc.,could be simulated using three simple rules: Separation,or the tendency of individuals (‘boids’ using Reynolds ter-minology) in a flock to avoid their close neighbours; align-ment, or the tendency of individuals to match the directionof their close neighbours; and cohesion, or the tendency ofindividuals to steer towards the average position of theirclose neighbours.This model has been extended with other arbitrary rules,for instance goal setting (i.e. attractors), and boundary set-ting (i.e. keeping the flock within a delimited space).2.2. Swarm GranulatorTim Blackwell and Michael Young of Goldsmiths Collegedeveloped Swarm Granulator as part of a larger project ex-ploring swarming and flocking approaches to parametergeneration for digital synthesis and DSP, particularly foruse in interactive systems involving live improvisers.[1]Grossly simplified, Swarm Granulator defines a six di-mensional parameter space in which swarm particles move.Input from human improvisers is interpreted according tovarious complicated schemes and represented within theparameter space in the form of attractors. Granulation pa-rameters are derived from the position of swarm particleswithin the space, the movement of which is determinedby a modified version of Reynold’s classic ‘Boids’ algo-rithm. The parameters derived are grain pitch, amplitude,duration, time between successive grains, and grain attackand decay times. Particle movement is not used to controlspatial location of the resulting sound.2.3. BATBoidsAndrew Lewis developed BATBoids as part of the BangorAudio Toolkit [4], at the University of Bangor. BATBoidsis used for spatialising multiple sound file copies. It islimited to two dimensions and a maximum of eight outputchannels, and is not designed for granulation.2.4. BEASTMulchAs part of the BEASTMulch (BEAST multichannel) project[9], the author has been exploring granulation techniquesover a large number of channels since 2004. One generalprinciple being explored as part of the project is that ofproducing works with a variety of streams or tracks (or‘stems’ to adapt a term from audio mastering), which canbe spatialised separately on any arbitrarily laid out speakersystem according to a pre-defined classification scheme(e.g. distance, relative diffuseness, sub-array membership,etc.). Adapting traditional stereo ‘diffusion’ techniques tothis approach has proved both effective in terms of creat-ing vivid multichannel effects, and retaining the flexibilityand adaptability that a live diffusion approach allows.This approach also proved effective with realtime gran-ulation; for instance assigning granulation to sub-arraysof loudspeakers resulted in ‘diffuse but localised’ effects.The ability to control the relative location and extent ofdiffuse effects proved subjectively satisfactory, particu-larly on large systems and within large spaces. 1 The issueof how to effectively interpolate between locations and de-grees of diffusion, however, remained unsolved.3. IMPLEMENTATION3.1. Class TaxonomyThe BEASTMulch project is implemented largely in theSuperCollider open source DSP and synthesis environ-ment. 2 Our implementation is based mainly around threeSC classes: BMSwarmGranulator, BMGrainBoid, and BM-GrainBoidSpace.BMSwarmGranulator is a sampled sound granulatorroughly in the style of Barry Truax’s GSAMX programme,implemented as part of his PODX system [7]. It allowscontrol of grain pitch, time stretch factor, duration, a ran-dom amount of variation in duration, delay between grainsin a ‘stream’, a random amount of variation in delay, a ran-dom offset from the current pointer position in the buffer(time smearing), and any extra parameters required for acustomised grain. 3 Each of these parameters can be de-termined on a per-grain basis by a function or equivalentobject (i.e. any SC object or class which returns an appro-priate result when sent the message ‘value’.) In additionto these the number of simultaneous grain streams can becontrolled (statistically rather than literally in this case).BMGrainBoid and BMGrainBoidSpace model the stateof individual boids and of a multi-dimensional space withinwhich boids can move, respectively. The number of di-mensions in a BMGrainBoidSpace is not limited in prin-ciple, but in practice (since we are modelling spatial lo-cation) we have to date worked with two and three di-mensional spaces. Speaker location is modelled by theBMSpeakerArray and BMSpeaker classes, which providelocation along three axes, with {0, 0, 0} generally corre-sponding to approximately head height of a person seatedat the control desk.3.2. Determining Grain LocationEach Boid represents a stream of grains. At grain spawntime the position of the corresponding boid is sampled and1 At the current time BEAST routinely mounts upwards of 90 dis-cretely addressable loudspeakers across 4 or more vertical strata withinour ‘home base’, the CBSO Centre in Birmingham.2 See http://supercollider.sourceforge.net3 In SuperCollider terms, grains generated can be based upon a cus-tom SynthDef rather than the default one, providing it includes the re-quired controls.BMGrainBoidBMGrainBoidBMGrainBoidBMGrainBoidSpace...BMSwarmGranulatorSpeaker Arraykd-treeRequest the next boidSend updated boid for proximity testChannel outputSpeaker 1Speaker 2Speaker 3...Figure 1. Block diagram of the localisation process.used to determine grain location. An efficient kd-tree al-gorithm 4 is used to determine the nearest speaker to theboid. The generated grain is then hard assigned to thisspeaker. Figure 1 presents a block diagram of this pro-cess.This hard assigning approach was chosen for a numberof reasons. Although panning between adjacent membersof an equidistant array (e.g. a ring or dome) is relativelysimple to do convincingly under normal circumstances,simulating positions between arbitrarily spaced speakersin three dimensions is not straightforward. In subjectivetesting, it was determined that the perceivable differencebetween hard panned and other approaches was minimalwith anything more than a few grains, and that with areasonably large number of grains of short duration thedifference was perceptually indistinguishable. This is notsurprising, given that in typical cases (i.e. with 16 simulta-neous grain streams or more) it is very likely that multiplegrains will be assigned to each active speaker at any givenmoment. Individual channels consisting of short grainswith small variations in offset, delays due to distance, etc.serve adequately to create decorrelation between activechannels (and thus the desired impression of diffusenessand/or increased physical volume [3]).Selective use of other approaches (e.g. ambisonics) tosimulate location (or even movement over the course of agrain’s life) might prove useful in special cases (e.g. asmall number of simultaneous grains of relatively longduration). However such cases fall outside the intendedrange of uses of the current system’s design, and presum-ably would result in the segregation of grains into individ-ually perceivable sonic events (i.e. literally perceivable asa swarm of sources), rather than in a ‘diffuse but localised’single sound.4 Implemented in SuperCollider as a ‘Quark’ (i.e. as part of SC’sexternal package managing system) by Dan Stowell3.3. Boid MovementBMGrainBoid implements the standard ‘boids’ rules ofseparation, cohesion, and alignment. In addition it im-plements a rule for attractors, in order to allow the boidsto converge on a location (and thus control movement), aboundary rule, to prevent the boids from leaving the per-formance space, and an object avoidance rule, which dis-courages the boids from entering a specified area. Thelatter rule can be useful in cases where a large area (forinstance the audience seating) contains no loudspeakers;by applying this rule the boids will travel around this areaeven if the shortest path to an attractor is directly throughit. If there is no attractor present the swarm will wanderfreely around the space.In addition to these rules a user is able to both scaleand limit the velocity of the boids, and control the maxi-mum distance at which the avoidance rule comes into ef-fect. Through careful variation of these parameters onecan control both the physical size of the swarm, and theextent to which that size varies as individuals move awayin response to the avoidance rule. The latter can be effec-tive in giving the resulting sound a subjective ‘liveliness’(somewhat akin to the effect of rustling leaves) since inpractice quick outward movements mean greater variationin the assignment of grains to speakers.Multiple swarms can either be independent, using mul-tiple instances of BMGrainBoidSpace, or interact using asingle instance, along with careful application of the rulesin order to encourage and control segregation.3.4. Visualisation and ControlWhile SuperCollider provides useful 2D drawing possi-bilities through its Pen class, its native 3D drawing ca-pabilities are somewhat rudimentary. Because of this theauthor implemented SCQuartzComposerView as an SCGUI widget. This allows one to embed and exchange datawith Quartz Compositions; visual rendering and process-ing specifications created with Apple’s highly optimisedQuartz Composer graphical rendering and processing lan-guage.Our implementation makes use of this for creating 3Drepresentations of speaker and boid locations. Figures 2and 3 present examples of this involving 40 grain streamsspatialised over a 52 loudspeaker system.The speakers and boids are rendered using an iterativealgorithm from the data supplied by the SC classes. Avariety of 3D transformations (e.g. rotation and zooming)are possible.Parameters can be controlled programatically from code,both ‘on the fly’ or in a predetermined fashion (SuperCol-lider is an interpreted language); or through the use of anexternal controller such as a graphics tablet. Capture of in-put is of course possible, and by providing a random seedGrainBoid behaviour and granulation results can be madeprecisely reproducible. 55 In fact the only randomised element in terms of GrainBoid be-haviour is the initial boid position.Figure 2. Detail of swarm visualisation. The spheres rep-resent boids, the boxes loudspeakers.4. SYSTEM BALANCINGIn order for this approach to be effective it is importantthat the system be balanced so that the levels of the in-dividual speakers are perceptually equal. Although thiscan be difficult to do in a non-homogenous system 6 , withBEAST we have developed an automated process whichmeasures a series of band-limited noise pulses sent to eachspeaker in turn through an omnidirectional microphone.These levels are normalised to the highest captured level,and all outputs are scaled down to match the lowest. Theselevels can then be further refined by ear if needed.It is worth noting that this automated approach elimi-nates the natural falloff in level which occurs due to dis-tance (i.e. one would expect that as a sound moved ontomore distant speakers, its amplitude would decrease). Oncethe system is balanced however, and thus in a known state,it is possible to reintroduce distance amplitude falloff basedon speaker position.Automatic speaker localisation is also possible, and hasproven effective in tests. This can simplify the tediousprocess of precisely measuring and/or placing each indi-vidual loudspeaker in a large system. With hard assignedgrains, however, small inaccuracies in speaker locationdata are not problematic.5. ASSESSMENTAlthough rigourous perceptual testing has not been car-ried out to date, in subjective testing by BEAST users the6 Specialised speakers such as tweeters and subwoofers are excludedfrom this process, as well as from the granulation output.Figure 3. Overhead view of swarm visualisation with a 52 loudspeaker array.method was deemed successful in both creating the effectof ‘localised but diffuse’ granular sources, and in convinc-ingly moving those sources through a performance space.Presumably due to the combination of decorrelated chan-nels, the effect was considered quite robust regardless oflistening position, suffering from being ‘out of the sweetspot’ only when one was in close proximity to an individ-ual loudspeaker. Naturally, movement across large gapsin loudspeaker spacing can prove objectionable, but thiseffect can be reduced somewhat by increasing the size ofthe swarm, and/or applying an avoidance rule.6. FUTUREWORKA number of possible directions for future work suggestthemselves. One obvious possibility would be the exten-sion of the boids approach to include other parameters /dimensions in a manner analogous to that used in Black-well and Young’s Swarm Granulator, discussed above.Due to the need to adapt to varying work formats, aes-thetics, and approaches to spatialisation, systems such asBEAST have become increasingly hybridised, and able toadapt to a variety of demands within a single concert. Aspart of the BEASTMulch project, for instance, the authorhas ported Ville Pullki’s Vector Base Amplitude Panning(VBAP)[5] to SuperCollider, and BEAST setups in recentyears have often included a partial dome of loudspeakers,which it is possible to address as a 3D VBAP Array. Al-though not specifically designed for 2D or 3D equidistantspeaker arrays, the swarm granulation system discussedherein would work in straightforward way with them, andcould easily be adapted to specify position based on az-imuth and elevation angles (as VBAP requires). As well,it could be extended to incorporate distance simulation us-ing any of the common approaches. An ambisonic versionwould also be a possibility. This would make the systemmore generally useful with smaller setups, many of whichconsist mostly or entirely of a single 2D ring.Although the current implementation makes use of gran-ulated sampled sound, the approach could also be used forspatialising granular synthesis.7. REFERENCES[1] T. Blackwell and M. Young. Swarm granulator. InThe European Workshop on Evolutionary Music andArt, 2004.[2] C. Clozier. The gmebaphone concept and the cy-berne´phone instrument. Computer Music Journal, 25(4):81–90, 2001.[3] G. S. Kendall. The decorrelation of audio signals andits impact on spatial imagery. Computer Music Jour-nal, 19(4):71–87, Winter 1995.[4] A. Lewis. Bangor audio toolkit. URL http://www.bangor.ac.uk/music/studios/BAT/.[5] V. Pulkki. Spatial sound generation and perceptionby amplitude panning techniques. Technical Re-port 62, Helsinki University of Technology, Labora-tory of Acoustics and Audio Signal Processing, 2001.[6] C. Reynolds. Herds, and schools: A distributedbehavioral model. Computer Graphics, 21(4 (SIG-GRAPH ’87 Conference Proceedings)):25–34, 1987.[7] B. Truax. Time shifting of sampled sound with areal-time granulation technique. In Proceedings ofthe 1990 International Computer Music Conference.Computer Music Association, San Francisco, 1990.[8] H. Tutschku. On the interpretation of multi-channelelectroacoustic works on loudspeaker-orchestras:Some thoughts on the grm- acousmonium andbeast. URL http://huseac.fas.harvard.edu/hydra/Interpretation-eng.pdf.[9] S. Wilson, J. Harrison, and S. L. Ancona. Beast-mulch. URL http://www.beast.bham.ac.uk/research/mulch.shtml.

Spatial swarm granulation

  University of BirminghamSpatial swarm granulationWilson, ScottCitation for published version (Harvard):Wilson, S 2008, 'Spatial swarm granulation', Paper presented at International Computer Music Conference,1/01/08.Link to publication on Research at Birmingham portalGeneral rightsUnless a licence is specified above, all rights (including copyright and moral rights) in this document are retained by the authors and/or thecopyright holders. The express permission of the copyright holder must be obtained for any use of this material other than for purposespermitted by law.•	Users may freely distribute the URL that is used to identify this publication.•	Users may download and/or print one copy of the publication from the University of Birmingham research portal for the purpose of privatestudy or non-commercial research.•	User may use extracts from the document in line with the concept of ‘fair dealing’ under the Copyright, Designs and Patents Act 1988 (?)•	Users may not further distribute the material nor use it for the purposes of commercial gain.Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.When citing, please reference the published version.Take down policyWhile the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has beenuploaded in error or has been deemed to be commercially or otherwise sensitive.If you believe that this is the case for this document, please contact UBIRA@lists.bham.ac.uk providing details and we will remove access tothe work immediately and investigate.Download date: 01. Mar. 2020SPATIAL SWARM GRANULATIONScott WilsonUniversity of BirminghamMusic DepartmentABSTRACTThis paper presents an implementation for dynamic two orthree dimensional spatial distribution of granulated sound(or granular synthesis) over an arbitrary loudspeaker sys-tem.1. INTRODUCTIONThe proliferation of large scale heterogeneous multichan-nel loudspeaker systems, such as the Gmebaphone / Cy-berne´phone of the Institut International de Musique Elec-troacoustique de Bourges (IMEB)[2], the Acousmoniumof the Groupe de Recherches Musicales (GRM) [8], andBirmingham ElectroAcoustic Sound Theatre (BEAST) [8],combined with the increasing potential to present ‘mas-sively’ multichannel works has presented us with a uniqueproblem: How to make use of the new multichannel pos-sibilities of such systems, while retaining their traditionaladvantages (e.g. adaptability, flexibility, robustness in avariety of spaces and acoustics). This is the focus of BEAST-Mulch, an AHRC funded research project at the Univer-sity of Birmingham led by the author, along with JontyHarrison and Sergio Luque Ancona.[9]A central problem explored in the project is how to si-multaneously and dynamically control location (and dif-fuseness) of sound, especially given that such systems typ-ically involve large numbers of speakers distributed in threedimensions at various distances from the centre of the lis-tening area. This paper presents one approach to this prob-lem, an implementation of a large-scale multichannel gran-ulator which makes use of swarming algorithms to deter-mine grain location.2. PRE-EXISTINGWORK2.1. BoidsMost work making use of flocking or swarming simula-tions is based upon Craig Reynolds’ 1986 Boids algorithm[6]. Since this approach is well known, I will not discussit in great detail here. Simply put, Reynolds demonstratedthat complex emergent behaviour, such as that observedin flocks of birds, schools of fish, swarms of insects, etc.,could be simulated using three simple rules: Separation,or the tendency of individuals (‘boids’ using Reynolds ter-minology) in a flock to avoid their close neighbours; align-ment, or the tendency of individuals to match the directionof their close neighbours; and cohesion, or the tendency ofindividuals to steer towards the average position of theirclose neighbours.This model has been extended with other arbitrary rules,for instance goal setting (i.e. attractors), and boundary set-ting (i.e. keeping the flock within a delimited space).2.2. Swarm GranulatorTim Blackwell and Michael Young of Goldsmiths Collegedeveloped Swarm Granulator as part of a larger project ex-ploring swarming and flocking approaches to parametergeneration for digital synthesis and DSP, particularly foruse in interactive systems involving live improvisers.[1]Grossly simplified, Swarm Granulator defines a six di-mensional parameter space in which swarm particles move.Input from human improvisers is interpreted according tovarious complicated schemes and represented within theparameter space in the form of attractors. Granulation pa-rameters are derived from the position of swarm particleswithin the space, the movement of which is determinedby a modified version of Reynold’s classic ‘Boids’ algo-rithm. The parameters derived are grain pitch, amplitude,duration, time between successive grains, and grain attackand decay times. Particle movement is not used to controlspatial location of the resulting sound.2.3. BATBoidsAndrew Lewis developed BATBoids as part of the BangorAudio Toolkit [4], at the University of Bangor. BATBoidsis used for spatialising multiple sound file copies. It islimited to two dimensions and a maximum of eight outputchannels, and is not designed for granulation.2.4. BEASTMulchAs part of the BEASTMulch (BEAST multichannel) project[9], the author has been exploring granulation techniquesover a large number of channels since 2004. One generalprinciple being explored as part of the project is that ofproducing works with a variety of streams or tracks (or‘stems’ to adapt a term from audio mastering), which canbe spatialised separately on any arbitrarily laid out speakersystem according to a pre-defined classification scheme(e.g. distance, relative diffuseness, sub-array membership,etc.). Adapting traditional stereo ‘diffusion’ techniques tothis approach has proved both effective in terms of creat-ing vivid multichannel effects, and retaining the flexibilityand adaptability that a live diffusion approach allows.This approach also proved effective with realtime gran-ulation; for instance assigning granulation to sub-arraysof loudspeakers resulted in ‘diffuse but localised’ effects.The ability to control the relative location and extent ofdiffuse effects proved subjectively satisfactory, particu-larly on large systems and within large spaces. 1 The issueof how to effectively interpolate between locations and de-grees of diffusion, however, remained unsolved.3. IMPLEMENTATION3.1. Class TaxonomyThe BEASTMulch project is implemented largely in theSuperCollider open source DSP and synthesis environ-ment. 2 Our implementation is based mainly around threeSC classes: BMSwarmGranulator, BMGrainBoid, and BM-GrainBoidSpace.BMSwarmGranulator is a sampled sound granulatorroughly in the style of Barry Truax’s GSAMX programme,implemented as part of his PODX system [7]. It allowscontrol of grain pitch, time stretch factor, duration, a ran-dom amount of variation in duration, delay between grainsin a ‘stream’, a random amount of variation in delay, a ran-dom offset from the current pointer position in the buffer(time smearing), and any extra parameters required for acustomised grain. 3 Each of these parameters can be de-termined on a per-grain basis by a function or equivalentobject (i.e. any SC object or class which returns an appro-priate result when sent the message ‘value’.) In additionto these the number of simultaneous grain streams can becontrolled (statistically rather than literally in this case).BMGrainBoid and BMGrainBoidSpace model the stateof individual boids and of a multi-dimensional space withinwhich boids can move, respectively. The number of di-mensions in a BMGrainBoidSpace is not limited in prin-ciple, but in practice (since we are modelling spatial lo-cation) we have to date worked with two and three di-mensional spaces. Speaker location is modelled by theBMSpeakerArray and BMSpeaker classes, which providelocation along three axes, with {0, 0, 0} generally corre-sponding to approximately head height of a person seatedat the control desk.3.2. Determining Grain LocationEach Boid represents a stream of grains. At grain spawntime the position of the corresponding boid is sampled and1 At the current time BEAST routinely mounts upwards of 90 dis-cretely addressable loudspeakers across 4 or more vertical strata withinour ‘home base’, the CBSO Centre in Birmingham.2 See http://supercollider.sourceforge.net3 In SuperCollider terms, grains generated can be based upon a cus-tom SynthDef rather than the default one, providing it includes the re-quired controls.BMGrainBoidBMGrainBoidBMGrainBoidBMGrainBoidSpace...BMSwarmGranulatorSpeaker Arraykd-treeRequest the next boidSend updated boid for proximity testChannel outputSpeaker 1Speaker 2Speaker 3...Figure 1. Block diagram of the localisation process.used to determine grain location. An efficient kd-tree al-gorithm 4 is used to determine the nearest speaker to theboid. The generated grain is then hard assigned to thisspeaker. Figure 1 presents a block diagram of this pro-cess.This hard assigning approach was chosen for a numberof reasons. Although panning between adjacent membersof an equidistant array (e.g. a ring or dome) is relativelysimple to do convincingly under normal circumstances,simulating positions between arbitrarily spaced speakersin three dimensions is not straightforward. In subjectivetesting, it was determined that the perceivable differencebetween hard panned and other approaches was minimalwith anything more than a few grains, and that with areasonably large number of grains of short duration thedifference was perceptually indistinguishable. This is notsurprising, given that in typical cases (i.e. with 16 simulta-neous grain streams or more) it is very likely that multiplegrains will be assigned to each active speaker at any givenmoment. Individual channels consisting of short grainswith small variations in offset, delays due to distance, etc.serve adequately to create decorrelation between activechannels (and thus the desired impression of diffusenessand/or increased physical volume [3]).Selective use of other approaches (e.g. ambisonics) tosimulate location (or even movement over the course of agrain’s life) might prove useful in special cases (e.g. asmall number of simultaneous grains of relatively longduration). However such cases fall outside the intendedrange of uses of the current system’s design, and presum-ably would result in the segregation of grains into individ-ually perceivable sonic events (i.e. literally perceivable asa swarm of sources), rather than in a ‘diffuse but localised’single sound.4 Implemented in SuperCollider as a ‘Quark’ (i.e. as part of SC’sexternal package managing system) by Dan Stowell3.3. Boid MovementBMGrainBoid implements the standard ‘boids’ rules ofseparation, cohesion, and alignment. In addition it im-plements a rule for attractors, in order to allow the boidsto converge on a location (and thus control movement), aboundary rule, to prevent the boids from leaving the per-formance space, and an object avoidance rule, which dis-courages the boids from entering a specified area. Thelatter rule can be useful in cases where a large area (forinstance the audience seating) contains no loudspeakers;by applying this rule the boids will travel around this areaeven if the shortest path to an attractor is directly throughit. If there is no attractor present the swarm will wanderfreely around the space.In addition to these rules a user is able to both scaleand limit the velocity of the boids, and control the maxi-mum distance at which the avoidance rule comes into ef-fect. Through careful variation of these parameters onecan control both the physical size of the swarm, and theextent to which that size varies as individuals move awayin response to the avoidance rule. The latter can be effec-tive in giving the resulting sound a subjective ‘liveliness’(somewhat akin to the effect of rustling leaves) since inpractice quick outward movements mean greater variationin the assignment of grains to speakers.Multiple swarms can either be independent, using mul-tiple instances of BMGrainBoidSpace, or interact using asingle instance, along with careful application of the rulesin order to encourage and control segregation.3.4. Visualisation and ControlWhile SuperCollider provides useful 2D drawing possi-bilities through its Pen class, its native 3D drawing ca-pabilities are somewhat rudimentary. Because of this theauthor implemented SCQuartzComposerView as an SCGUI widget. This allows one to embed and exchange datawith Quartz Compositions; visual rendering and process-ing specifications created with Apple’s highly optimisedQuartz Composer graphical rendering and processing lan-guage.Our implementation makes use of this for creating 3Drepresentations of speaker and boid locations. Figures 2and 3 present examples of this involving 40 grain streamsspatialised over a 52 loudspeaker system.The speakers and boids are rendered using an iterativealgorithm from the data supplied by the SC classes. Avariety of 3D transformations (e.g. rotation and zooming)are possible.Parameters can be controlled programatically from code,both ‘on the fly’ or in a predetermined fashion (SuperCol-lider is an interpreted language); or through the use of anexternal controller such as a graphics tablet. Capture of in-put is of course possible, and by providing a random seedGrainBoid behaviour and granulation results can be madeprecisely reproducible. 55 In fact the only randomised element in terms of GrainBoid be-haviour is the initial boid position.Figure 2. Detail of swarm visualisation. The spheres rep-resent boids, the boxes loudspeakers.4. SYSTEM BALANCINGIn order for this approach to be effective it is importantthat the system be balanced so that the levels of the in-dividual speakers are perceptually equal. Although thiscan be difficult to do in a non-homogenous system 6 , withBEAST we have developed an automated process whichmeasures a series of band-limited noise pulses sent to eachspeaker in turn through an omnidirectional microphone.These levels are normalised to the highest captured level,and all outputs are scaled down to match the lowest. Theselevels can then be further refined by ear if needed.It is worth noting that this automated approach elimi-nates the natural falloff in level which occurs due to dis-tance (i.e. one would expect that as a sound moved ontomore distant speakers, its amplitude would decrease). Oncethe system is balanced however, and thus in a known state,it is possible to reintroduce distance amplitude falloff basedon speaker position.Automatic speaker localisation is also possible, and hasproven effective in tests. This can simplify the tediousprocess of precisely measuring and/or placing each indi-vidual loudspeaker in a large system. With hard assignedgrains, however, small inaccuracies in speaker locationdata are not problematic.5. ASSESSMENTAlthough rigourous perceptual testing has not been car-ried out to date, in subjective testing by BEAST users the6 Specialised speakers such as tweeters and subwoofers are excludedfrom this process, as well as from the granulation output.Figure 3. Overhead view of swarm visualisation with a 52 loudspeaker array.method was deemed successful in both creating the effectof ‘localised but diffuse’ granular sources, and in convinc-ingly moving those sources through a performance space.Presumably due to the combination of decorrelated chan-nels, the effect was considered quite robust regardless oflistening position, suffering from being ‘out of the sweetspot’ only when one was in close proximity to an individ-ual loudspeaker. Naturally, movement across large gapsin loudspeaker spacing can prove objectionable, but thiseffect can be reduced somewhat by increasing the size ofthe swarm, and/or applying an avoidance rule.6. FUTUREWORKA number of possible directions for future work suggestthemselves. One obvious possibility would be the exten-sion of the boids approach to include other parameters /dimensions in a manner analogous to that used in Black-well and Young’s Swarm Granulator, discussed above.Due to the need to adapt to varying work formats, aes-thetics, and approaches to spatialisation, systems such asBEAST have become increasingly hybridised, and able toadapt to a variety of demands within a single concert. Aspart of the BEASTMulch project, for instance, the authorhas ported Ville Pullki’s Vector Base Amplitude Panning(VBAP)[5] to SuperCollider, and BEAST setups in recentyears have often included a partial dome of loudspeakers,which it is possible to address as a 3D VBAP Array. Al-though not specifically designed for 2D or 3D equidistantspeaker arrays, the swarm granulation system discussedherein would work in straightforward way with them, andcould easily be adapted to specify position based on az-imuth and elevation angles (as VBAP requires). As well,it could be extended to incorporate distance simulation us-ing any of the common approaches. An ambisonic versionwould also be a possibility. This would make the systemmore generally useful with smaller setups, many of whichconsist mostly or entirely of a single 2D ring.Although the current implementation makes use of gran-ulated sampled sound, the approach could also be used forspatialising granular synthesis.7. REFERENCES[1] T. Blackwell and M. Young. Swarm granulator. InThe European Workshop on Evolutionary Music andArt, 2004.[2] C. Clozier. The gmebaphone concept and the cy-berne´phone instrument. Computer Music Journal, 25(4):81–90, 2001.[3] G. S. Kendall. The decorrelation of audio signals andits impact on spatial imagery. Computer Music Jour-nal, 19(4):71–87, Winter 1995.[4] A. Lewis. Bangor audio toolkit. URL http://www.bangor.ac.uk/music/studios/BAT/.[5] V. Pulkki. Spatial sound generation and perceptionby amplitude panning techniques. Technical Re-port 62, Helsinki University of Technology, Labora-tory of Acoustics and Audio Signal Processing, 2001.[6] C. Reynolds. Herds, and schools: A distributedbehavioral model. Computer Graphics, 21(4 (SIG-GRAPH ’87 Conference Proceedings)):25–34, 1987.[7] B. Truax. Time shifting of sampled sound with areal-time granulation technique. In Proceedings ofthe 1990 International Computer Music Conference.Computer Music Association, San Francisco, 1990.[8] H. Tutschku. On the interpretation of multi-channelelectroacoustic works on loudspeaker-orchestras:Some thoughts on the grm- acousmonium andbeast. URL http://huseac.fas.harvard.edu/hydra/Interpretation-eng.pdf.[9] S. Wilson, J. Harrison, and S. L. Ancona. Beast-mulch. URL http://www.beast.bham.ac.uk/research/mulch.shtml.

Scott Wilson

University of Michigan Library Repository

SPATIAL SWARM GRANULATION

University of Birmingham Research Archive, E-prints Repository

SPATIAL SWARM GRANULATION
Scott Wilson
University of Birmingham
Music Department
ABSTRACT
This paper presents an implementation for dynamic two or
three dimensional spatial distribution of granulated sound
(or granular synthesis) over an arbitrary loudspeaker sys-
tem.
1. INTRODUCTION
The proliferation of large scale heterogeneous multichan-
nel loudspeaker systems, such as the Gmebaphone / Cy-
berne´phone of the Institut International de Musique Elec-
troacoustique de Bourges (IMEB)[2], the Acousmonium
of the Groupe de Recherches Musicales (GRM) [8], and
Birmingham ElectroAcoustic Sound Theatre (BEAST) [8],
combined with the increasing potential to present ‘mas-
sively’ multichannel works has presented us with a unique
problem: How to make use of the new multichannel pos-
sibilities of such systems, while retaining their traditional
advantages (e.g. adaptability, flexibility, robustness in a
variety of spaces and acoustics). This is the focus of BEAST-
Mulch, an AHRC funded research project at the Univer-
sity of Birmingham led by the author, along with Jonty
Harrison and Sergio Luque Ancona.[9]
A central problem explored in the project is how to si-
multaneously and dynamically control location (and dif-
fuseness) of sound, especially given that such systems typ-
ically involve large numbers of speakers distributed in three
dimensions at various distances from the centre of the lis-
tening area. This paper presents one approach to this prob-
lem, an implementation of a large-scale multichannel gran-
ulator which makes use of swarming algorithms to deter-
mine grain location.
2. PRE-EXISTINGWORK
2.1. Boids
Most work making use of flocking or swarming simula-
tions is based upon Craig Reynolds’ 1986 Boids algorithm
[6]. Since this approach is well known, I will not discuss
it in great detail here. Simply put, Reynolds demonstrated
that complex emergent behaviour, such as that observed
in flocks of birds, schools of fish, swarms of insects, etc.,
could be simulated using three simple rules: Separation,
or the tendency of individuals (‘boids’ using Reynolds ter-
minology) in a flock to avoid their close neighbours; align-
ment, or the tendency of individuals to match the direction
of their close neighbours; and cohesion, or the tendency of
individuals to steer towards the average position of their
close neighbours.
This model has been extended with other arbitrary rules,
for instance goal setting (i.e. attractors), and boundary set-
ting (i.e. keeping the flock within a delimited space).
2.2. Swarm Granulator
Tim Blackwell and Michael Young of Goldsmiths College
developed Swarm Granulator as part of a larger project ex-
ploring swarming and flocking approaches to parameter
generation for digital synthesis and DSP, particularly for
use in interactive systems involving live improvisers.[1]
Grossly simplified, Swarm Granulator defines a six di-
mensional parameter space in which swarm particles move.
Input from human improvisers is interpreted according to
various complicated schemes and represented within the
parameter space in the form of attractors. Granulation pa-
rameters are derived from the position of swarm particles
within the space, the movement of which is determined
by a modified version of Reynold’s classic ‘Boids’ algo-
rithm. The parameters derived are grain pitch, amplitude,
duration, time between successive grains, and grain attack
and decay times. Particle movement is not used to control
spatial location of the resulting sound.
2.3. BATBoids
Andrew Lewis developed BATBoids as part of the Bangor
Audio Toolkit [4], at the University of Bangor. BATBoids
is used for spatialising multiple sound file copies. It is
limited to two dimensions and a maximum of eight output
channels, and is not designed for granulation.
2.4. BEASTMulch
As part of the BEASTMulch (BEAST multichannel) project
[9], the author has been exploring granulation techniques
over a large number of channels since 2004. One general
principle being explored as part of the project is that of
producing works with a variety of streams or tracks (or
‘stems’ to adapt a term from audio mastering), which can
be spatialised separately on any arbitrarily laid out speaker
system according to a pre-defined classification scheme
(e.g. distance, relative diffuseness, sub-array membership,
etc.). Adapting traditional stereo ‘diffusion’ techniques to
this approach has proved both effective in terms of creat-
ing vivid multichannel effects, and retaining the flexibility
and adaptability that a live diffusion approach allows.
This approach also proved effective with realtime gran-
ulation; for instance assigning granulation to sub-arrays
of loudspeakers resulted in ‘diffuse but localised’ effects.
The ability to control the relative location and extent of
diffuse effects proved subjectively satisfactory, particu-
larly on large systems and within large spaces. 1 The issue
of how to effectively interpolate between locations and de-
grees of diffusion, however, remained unsolved.
3. IMPLEMENTATION
3.1. Class Taxonomy
The BEASTMulch project is implemented largely in the
SuperCollider open source DSP and synthesis environ-
ment. 2 Our implementation is based mainly around three
SC classes: BMSwarmGranulator, BMGrainBoid, and BM-
GrainBoidSpace.
BMSwarmGranulator is a sampled sound granulator
roughly in the style of Barry Truax’s GSAMX programme,
implemented as part of his PODX system [7]. It allows
control of grain pitch, time stretch factor, duration, a ran-
dom amount of variation in duration, delay between grains
in a ‘stream’, a random amount of variation in delay, a ran-
dom offset from the current pointer position in the buffer
(time smearing), and any extra parameters required for a
customised grain. 3 Each of these parameters can be de-
termined on a per-grain basis by a function or equivalent
object (i.e. any SC object or class which returns an appro-
priate result when sent the message ‘value’.) In addition
to these the number of simultaneous grain streams can be
controlled (statistically rather than literally in this case).
BMGrainBoid and BMGrainBoidSpace model the state
of individual boids and of a multi-dimensional space within
which boids can move, respectively. The number of di-
mensions in a BMGrainBoidSpace is not limited in prin-
ciple, but in practice (since we are modelling spatial lo-
cation) we have to date worked with two and three di-
mensional spaces. Speaker location is modelled by the
BMSpeakerArray and BMSpeaker classes, which provide
location along three axes, with {0, 0, 0} generally corre-
sponding to approximately head height of a person seated
at the control desk.
3.2. Determining Grain Location
Each Boid represents a stream of grains. At grain spawn
time the position of the corresponding boid is sampled and
1 At the current time BEAST routinely mounts upwards of 90 dis-
cretely addressable loudspeakers across 4 or more vertical strata within
our ‘home base’, the CBSO Centre in Birmingham.
2 See http://supercollider.sourceforge.net
3 In SuperCollider terms, grains generated can be based upon a cus-
tom SynthDef rather than the default one, providing it includes the re-
quired controls.
BMGrainBoid
BMGrainBoid
BMGrainBoid
BMGrainBoidSpace
...
BMSwarmGranulator
Speaker Array
kd-tree
Request the 
next boid
Send updated boid 
for proximity test
Channel output
Speaker 1
Speaker 2
Speaker 3
...
Figure 1. Block diagram of the localisation process.
used to determine grain location. An efficient kd-tree al-
gorithm 4 is used to determine the nearest speaker to the
boid. The generated grain is then hard assigned to this
speaker. Figure 1 presents a block diagram of this pro-
cess.
This hard assigning approach was chosen for a number
of reasons. Although panning between adjacent members
of an equidistant array (e.g. a ring or dome) is relatively
simple to do convincingly under normal circumstances,
simulating positions between arbitrarily spaced speakers
in three dimensions is not straightforward. In subjective
testing, it was determined that the perceivable difference
between hard panned and other approaches was minimal
with anything more than a few grains, and that with a
reasonably large number of grains of short duration the
difference was perceptually indistinguishable. This is not
surprising, given that in typical cases (i.e. with 16 simulta-
neous grain streams or more) it is very likely that multiple
grains will be assigned to each active speaker at any given
moment. Individual channels consisting of short grains
with small variations in offset, delays due to distance, etc.
serve adequately to create decorrelation between active
channels (and thus the desired impression of diffuseness
and/or increased physical volume [3]).
Selective use of other approaches (e.g. ambisonics) to
simulate location (or even movement over the course of a
grain’s life) might prove useful in special cases (e.g. a
small number of simultaneous grains of relatively long
duration). However such cases fall outside the intended
range of uses of the current system’s design, and presum-
ably would result in the segregation of grains into individ-
ually perceivable sonic events (i.e. literally perceivable as
a swarm of sources), rather than in a ‘diffuse but localised’
single sound.
4 Implemented in SuperCollider as a ‘Quark’ (i.e. as part of SC’s
external package managing system) by Dan Stowell
3.3. Boid Movement
BMGrainBoid implements the standard ‘boids’ rules of
separation, cohesion, and alignment. In addition it im-
plements a rule for attractors, in order to allow the boids
to converge on a location (and thus control movement), a
boundary rule, to prevent the boids from leaving the per-
formance space, and an object avoidance rule, which dis-
courages the boids from entering a specified area. The
latter rule can be useful in cases where a large area (for
instance the audience seating) contains no loudspeakers;
by applying this rule the boids will travel around this area
even if the shortest path to an attractor is directly through
it. If there is no attractor present the swarm will wander
freely around the space.
In addition to these rules a user is able to both scale
and limit the velocity of the boids, and control the maxi-
mum distance at which the avoidance rule comes into ef-
fect. Through careful variation of these parameters one
can control both the physical size of the swarm, and the
extent to which that size varies as individuals move away
in response to the avoidance rule. The latter can be effec-
tive in giving the resulting sound a subjective ‘liveliness’
(somewhat akin to the effect of rustling leaves) since in
practice quick outward movements mean greater variation
in the assignment of grains to speakers.
Multiple swarms can either be independent, using mul-
tiple instances of BMGrainBoidSpace, or interact using a
single instance, along with careful application of the rules
in order to encourage and control segregation.
3.4. Visualisation and Control
While SuperCollider provides useful 2D drawing possi-
bilities through its Pen class, its native 3D drawing ca-
pabilities are somewhat rudimentary. Because of this the
author implemented SCQuartzComposerView as an SC
GUI widget. This allows one to embed and exchange data
with Quartz Compositions; visual rendering and process-
ing specifications created with Apple’s highly optimised
Quartz Composer graphical rendering and processing lan-
guage.
Our implementation makes use of this for creating 3D
representations of speaker and boid locations. Figures 2
and 3 present examples of this involving 40 grain streams
spatialised over a 52 loudspeaker system.
The speakers and boids are rendered using an iterative
algorithm from the data supplied by the SC classes. A
variety of 3D transformations (e.g. rotation and zooming)
are possible.
Parameters can be controlled programatically from code,
both ‘on the fly’ or in a predetermined fashion (SuperCol-
lider is an interpreted language); or through the use of an
external controller such as a graphics tablet. Capture of in-
put is of course possible, and by providing a random seed
GrainBoid behaviour and granulation results can be made
precisely reproducible. 5
5 In fact the only randomised element in terms of GrainBoid be-
haviour is the initial boid position.
Figure 2. Detail of swarm visualisation. The spheres rep-
resent boids, the boxes loudspeakers.
4. SYSTEM BALANCING
In order for this approach to be effective it is important
that the system be balanced so that the levels of the in-
dividual speakers are perceptually equal. Although this
can be difficult to do in a non-homogenous system 6 , with
BEAST we have developed an automated process which
measures a series of band-limited noise pulses sent to each
speaker in turn through an omnidirectional microphone.
These levels are normalised to the highest captured level,
and all outputs are scaled down to match the lowest. These
levels can then be further refined by ear if needed.
It is worth noting that this automated approach elimi-
nates the natural falloff in level which occurs due to dis-
tance (i.e. one would expect that as a sound moved onto
more distant speakers, its amplitude would decrease). Once
the system is balanced however, and thus in a known state,
it is possible to reintroduce distance amplitude falloff based
on speaker position.
Automatic speaker localisation is also possible, and has
proven effective in tests. This can simplify the tedious
process of precisely measuring and/or placing each indi-
vidual loudspeaker in a large system. With hard assigned
grains, however, small inaccuracies in speaker location
data are not problematic.
5. ASSESSMENT
Although rigourous perceptual testing has not been car-
ried out to date, in subjective testing by BEAST users the
6 Specialised speakers such as tweeters and subwoofers are excluded
from this process, as well as from the granulation output.
Figure 3. Overhead view of swarm visualisation with a 52 loudspeaker array.
method was deemed successful in both creating the effect
of ‘localised but diffuse’ granular sources, and in convinc-
ingly moving those sources through a performance space.
Presumably due to the combination of decorrelated chan-
nels, the effect was considered quite robust regardless of
listening position, suffering from being ‘out of the sweet
spot’ only when one was in close proximity to an individ-
ual loudspeaker. Naturally, movement across large gaps
in loudspeaker spacing can prove objectionable, but this
effect can be reduced somewhat by increasing the size of
the swarm, and/or applying an avoidance rule.
6. FUTUREWORK
A number of possible directions for future work suggest
themselves. One obvious possibility would be the exten-
sion of the boids approach to include other parameters /
dimensions in a manner analogous to that used in Black-
well and Young’s Swarm Granulator, discussed above.
Due to the need to adapt to varying work formats, aes-
thetics, and approaches to spatialisation, systems such as
BEAST have become increasingly hybridised, and able to
adapt to a variety of demands within a single concert. As
part of the BEASTMulch project, for instance, the author
has ported Ville Pullki’s Vector Base Amplitude Panning
(VBAP)[5] to SuperCollider, and BEAST setups in recent
years have often included a partial dome of loudspeakers,
which it is possible to address as a 3D VBAP Array. Al-
though not specifically designed for 2D or 3D equidistant
speaker arrays, the swarm granulation system discussed
herein would work in straightforward way with them, and
could easily be adapted to specify position based on az-
imuth and elevation angles (as VBAP requires). As well,
it could be extended to incorporate distance simulation us-
ing any of the common approaches. An ambisonic version
would also be a possibility. This would make the system
more generally useful with smaller setups, many of which
consist mostly or entirely of a single 2D ring.
Although the current implementation makes use of gran-
ulated sampled sound, the approach could also be used for
spatialising granular synthesis.
7. REFERENCES
[1] T. Blackwell and M. Young. Swarm granulator. In
The European Workshop on Evolutionary Music and
Art, 2004.
[2] C. Clozier. The gmebaphone concept and the cy-
berne´phone instrument. Computer Music Journal, 25
(4):81–90, 2001.
[3] G. S. Kendall. The decorrelation of audio signals and
its impact on spatial imagery. Computer Music Jour-
nal, 19(4):71–87, Winter 1995.
[4] A. Lewis. Bangor audio toolkit. URL http://
www.bangor.ac.uk/music/studios/BAT/.
[5] V. Pulkki. Spatial sound generation and perception
by amplitude panning techniques. Technical Re-
port 62, Helsinki University of Technology, Labora-
tory of Acoustics and Audio Signal Processing, 2001.
[6] C. Reynolds. Herds, and schools: A distributed
behavioral model. Computer Graphics, 21(4 (SIG-
GRAPH ’87 Conference Proceedings)):25–34, 1987.
[7] B. Truax. Time shifting of sampled sound with a
real-time granulation technique. In Proceedings of
the 1990 International Computer Music Conference.
Computer Music Association, San Francisco, 1990.
[8] H. Tutschku. On the interpretation of multi-channel
electroacoustic works on loudspeaker-orchestras:
Some thoughts on the grm- acousmonium and
beast. URL http://huseac.fas.harvard.
edu/hydra/Interpretation-eng.pdf.
[9] S. Wilson, J. Harrison, and S. L. Ancona. Beast-
mulch. URL http://www.beast.bham.ac.
uk/research/mulch.shtml.


http://hdl.handle.net/2027/spo.bbp2372.2008.127

SPATIAL SWARM GRANULATION

Abstract

Similar works

Full text

Available Versions

University of Birmingham Research Portal

University of Birmingham Research Portal

University of Michigan Library Repository

University of Birmingham Research Archive, E-prints Repository