As part of an investigation into the potential of historical theatre sound effects as a resource for Sonic Interaction Design (SID), an acoustic theatre wind machine was constructed and analysed as an interactive sounding object. Using the Sound Designer’s Toolkit (SDT), a digital, physical modelling- based version of the wind machine was programmed, and the acoustic device fitted with a sensor system to control the digital model. This paper presents an initial comparison between the sound output of the acoustic theatre wind machine and its digital counterpart. Three simple and distinct rotational gestures are chosen to explore the main acoustic parameters of the output of the wind machine in operation: a single rotation; a short series of five rotations to create a sustained sound; and a longer series of ten rotations that start at speed and diminish in energy. These gestures are performed, and the resulting acoustic and digital sounds recorded simultaneously, facilitating an analysis of the temporal and spectral domain in Matlab of the same real-time performance of both sources. The results are reported, and a discussion of how they inform further calibration of the real-time synthesis system is presented

Keenan, Fiona

Pauletto, Sandra

White Rose Research Online

An Acoustic Wind Machine and its Digital Counterpart : Initial Audio Analysis and Comparison

This is a repository copy of An Acoustic Wind Machine and its Digital Counterpart : Initial Audio Analysis and Comparison.White Rose Research Online URL for this paper:https://eprints.whiterose.ac.uk/113697/Version: Published VersionConference or Workshop Item:Keenan, Fiona orcid.org/0000-0003-2046-9036 and Pauletto, Sandra orcid.org/0000-0002-9404-851X (2016) An Acoustic Wind Machine and its Digital Counterpart : Initial Audio Analysis and Comparison. In: Interactive Audio Systems Symposium, 23 Sep 2016. eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. An Acoustic Wind Machine and its Digital Counterpart: Initial Audio Analysis and Comparison FIONA KEENAN1 AND SANDRA PAULETTO2 1 Department of Theatre, Film, Television and Interactive Media, University of York  e-mail: fk632@york.ac.uk  2 Department of Theatre, Film, Television and Interactive Media, University of York  e-mail: sandra.pauletto@york.ac.uk   September 23rd 2016  Abstract As part of an investigation into the potential of historical theatre sound effects as a resource for Sonic Interaction Design (SID), an acoustic theatre wind machine was constructed and analysed as an interactive sounding object. Using the Sound Designer’s Toolkit (SDT), a digital, physical modelling-based version of the wind machine was programmed, and the acoustic device fitted with a sensor system to control the digital model. This paper presents an initial comparison between the sound output of the acoustic theatre wind machine and its digital counterpart. Three simple and distinct rotational gestures are chosen to explore the main acoustic parameters of the output of the wind machine in operation: a single rotation; a short series of five rotations to create a sustained sound; and a longer series of ten rotations that start at speed and diminish in energy. These gestures are performed, and the resulting acoustic and digital sounds recorded simultaneously, facilitating an analysis of the temporal and spectral domain in Matlab of the same real-time performance of both sources. The results are reported, and a discussion of how they inform further calibration of the real-time synthesis system is presented.   1 Introduction and Related Work  Sonic Interaction Design (SID) researches tactile, performative and multisensory aspect of sonic experience with the aim of designing new sonic interactions [1]. Theatre has a long history of the live performance of sound with mechanical devices and acoustic materials, and as such we propose that it represents a potentially rich resource for the design of new sonic interactions and action-sound couplings.   This paper presents the first stage of an investigation into one such device, a theatre wind machine. This device originates in the nineteenth century, and consists of a wooden slatted cylinder mounted on an axle and rotated with a crank handle against a cloth [2]. The friction between the moving wood and cloth creates a wind-like sound. The wind machine was chosen because it offers the possibility of exploring a continuous action-sound coupling in an experimental setting.   Relevant research has already been undertaken in this area by Serafin and de Götzen [3], who analysed the sound production of Luigi Russolo’s intonarumori family of mechanical noise intoners created for early twentieth century Futurist musical performances. This analysis then informed the creation of a user interface controlling a synthesis engine, replicating the original enactive workings of the historical devices.   We aim to extend this research to the area of historical theatre sound effects, and use the methodology of replicating an acoustic device as a user interface and synthesis engine in order to fully examine the action-sound coupling of the original machine in an experimental setting.   2 Methodology  This work began with the construction of an acoustic wind machine according to historical design instructions (see Figure 1) in order to have a specific example to model in software [4]. As part of the initial evaluation, its stages of sound production were deconstructed using an entity-action model [5]:   1. The cylinder of our wind machine example consists of twelve individual slats.  2. During rotation, each slat comes into contact with the cloth, produces sound through friction, and then falls silent as it moves out of range.  An Acoustic Wind Machine and its Digital Counterpart Keenan and Pauletto Interactive Audio Systems Symposium, September 23rd 2016, University of York, United Kingdom  23. Only seven slats are in contact with the cloth at any one time.     Figure 1: A historical wind machine design (left) [6], and our reconstructed version (right).  This mechanical process formed the basis for the creation of a synthesis engine in MaxMSP [7] using the Sound Designer’s Toolkit (SDT) suite of objects [8].The current version of the synthesis engine consists of twelve instances of a physical model of nonlinear friction, using the sdt.scraping~, sdt.friction~, sdt.inertial~ and sdt.modal~ objects. This combines a simulation of a probe sliding on a surface with a nonlinear friction model between one inertial and one modal object. These components are the most recent iterations of the SDT’s dynamic friction model [9] [10]. The SDT is designed to allow a full exploration of the parameters of its models in real-time, and as such arguments to the objects used were chosen through a period of rehearsal with the digital system and the original acoustic machine to approximate the best settings before this initial evaluation. These are outlined in the tables below.   Surface profile (a signal) noise~ Grain (density of micro-impacts) 0.004 Velocity (m/s) Real-time data: Orientation angle Force (N) Real-time data: accelerometer to torque equation Table 1: Parameters to sdt.scraping~   Strike velocity 0.0 Mass of inertial object (kg) 0.01 Fragment size (to simulate crumpling) 1 Table 2: Parameters to sdt.inertial~    External rubbing force signal from sdt.scraping~ Bristle stiffness (evolution of mode lock-in) 100000 Bristle dissipation (sound bandwidth) 1 Linear viscosity (speed of timbre evolution and pitch) 0.3 Amount of sliding noise (perceived surface roughness) 0.0 Dynamic friction coefficient (high values reduce sound bandwidth) 0.1 Static friction coefficient (smoothness of sound attack) 0.5 (for Hemp and Wood) Breakaway coefficient (transients of elasto-plastic state) 0.1 Stribeck velocity (smoothness of sound attacks) 0.1 Force (N) 0.5 Pickup index of object 1 (contact point) 0 Pickup index of object 2 (contact point) 0 Table 3: Parameters to sdt.friction~  Frequency of each mode (Hz) 400, 720, 1200 Decay of each mode (s) 0.04, 0.02, 0.03 Weight of each mode (1/kg) 64 64 64 Modal gain masks for each pickup Pickup0: 0.1, 0.15 and 0.25 Pickup1: 0.1, 0.15, 0.2 Fraction of whole object (for crumpling algorithm) 1 Active modes 3 Table 4: Parameters to sdt.modal~ The synthesis engine is activated with real-time data from an Inertial Measurement Unit (IMU) sensor, Arduino and XBee mounted on the acoustic wind machine, facilitating wireless data transmission to the software. As the crank handle is rotated, the IMU transmits orientation (polar angle about the wind machine’s axle) and acceleration data to the patch in real time. This data stream is currently configured to drive parameters to sdt.scraping~, modelling the sound produced by each slat (see Table 1). The acceleration data informs the mapping of the force parameter to sdt.scraping~, and the orientation data is translated to a linear movement controlling its velocity parameter.   To enable a single stream of data to control twelve digital slats, the orientation data is placed degrees out of phase to correspond with the position of each slat on the original acoustic device (see Figure 2). This method digitally preserves their irregular placement on the original wind machine. As a slat moves into the range of the cloth, its digital counterpart also produces sound. In this way, no audio enveloping is required to control the sound produced, and the trajectory of each slat is controlled entirely with data.  An Acoustic Wind Machine and its Digital Counterpart Keenan and Pauletto Interactive Audio Systems Symposium, September 23rd 2016, University of York, United Kingdom  3  Figure 2: Side view of acoustic wind machine showing slat positions in a 360° rotation from their position of origin (slat F).  In order to evaluate the effectiveness of the real-time synthesis engine in its current form, three distinct rotational gestures were chosen to be performed while the acoustic and digital wind machines were simultaneously recorded. The aim was to examine the spectral behaviour of both sources, and in particular determine whether the digital wind machine’s amplitude envelope accurately matched that of the acoustic version, as this would reveal whether the rotational behaviour of the acoustic device had transferred correctly to the digital version.    Gestures were chosen to give as complete a picture as possible of the of both systems in performance to facilitate their comparison:   1. A single rotation.  2. Five rotations to produce a continuous sound.  3. Ten rotations that start at speed, but then diminish in energy.    These gestures were repeatedly recorded, giving a total of 30 examples for each. Here we present the analysis of representative audio clips using Matlab and the MIR Toolbox [11].   3 Results  Amplitude Envelope  Our initial analysis of the audio has revealed a 20ms delay between the acoustic machine and the onset of the digital version. We believe this to be due the efficiency of the MaxMSP patch itself, and while a small amount of latency is to be expected, we are taking steps to reduce this for the future. In order to facilitate an accurate comparison of the two sound sources, this delay between the onsets has been removed from the files.   For a gesture of one rotation (Figure 3), the analysis shows that the digital sound has a faster attack at the start of the sound and a faster decay at the end. The overall duration of the two sounds and the shape of the sustained part of the envelope are similar. The envelopes of the digital and acoustic sound for the gesture of five continuous rotations (Figure 4) show similar start and decay times. The peaks of the 5 rotations are recognizable in both signals, although the digital rotations tend to lag a few milliseconds behind the acoustic rotations.   Figure 3: Amplitude Envelope for 1 Rotation - Acoustic (blue) and Digital (red)  Figure 4: Amplitude Envelope for 5 Rotations - Acoustic (blue) and An Acoustic Wind Machine and its Digital Counterpart Keenan and Pauletto Interactive Audio Systems Symposium, September 23rd 2016, University of York, United Kingdom  4Digital (red) For a gesture of ten rotations that start at speed and then diminish in energy, the digital wind machine does not diminish in energy as its acoustic counterpart does. The peaks of the digital wind machine corresponding to the ten rotations are also more irregular than the acoustic machine (Figure 5).    Figure 5: Amplitude Envelope for 10 Rotations - Acoustic (blue) and Digital (red)  Spectra  The FFT spectrum of the acoustic and digital wind machines remain consistent over time, and as such only an analysis of a single rotation for each (acoustic and digital sound) is shown (see Figures 6 and 7).    Figure 6: Spectrum for 1 rotation with acoustic wind machine.   Figure 7: Spectrum for 1 rotation with digital wind machine.  The spectrum of the acoustic sound shows small peaks at around 350Hz, 600Hz and 800Hz. The spectrum of the digital sound shows pronounced peaks at around 650Hz, 1200Hz and 2000Hz.   4 Discussion  The rotational mechanism of the acoustic wind machine has transferred relatively well to the digital version. This is the case in particular when the gesture is simple and sustained in time. The gesture of ten rotations diminishing in energy has revealed a bigger discrepancy between the behaviour of the synthesis engine and the behaviour of the acoustic machine. The energy of the digital wind machine does not diminish as the acoustic machine does, and the amplitude peaks characterizing the rotations are less distinct. This gesture requires more variability in performance over time, and a higher amount of energy at the start of the gesture. It is possible that the issue is due to the polling of the data from the sensor. This will be looked at in future work, as the digital machine needs to be able to withstand variability in performance.  The spectra of the acoustic and digital wind machines are quite different, with the digital version containing more high frequency information. In particular the three distinct peaks in the digital spectrum need reducing, suggesting that the frequency parameters to sdt.modal~ need to be adjusted to reduce the brightness of the sound and bring the synthesis engine closer to the acoustic machine.   We propose to revisit the machine model within MaxMSP to improve the response of the digital system to variations in rotational speed. The parameters of the sound engine will also be examined to further develop the quality of the friction through further real-time manipulation.    An Acoustic Wind Machine and its Digital Counterpart Keenan and Pauletto Interactive Audio Systems Symposium, September 23rd 2016, University of York, United Kingdom  55 Conclusion  We have described the initial stages of design of a digital model of a theatre wind machine, and its comparison with the acoustic version on which it is based. Suggestions for the improvement of the digital model were outlined.    6 Acknowledgement  This research is supported by the Arts and Humanities Research Council (AHRC), through the White Rose College of the Arts and Humanities (WRoCAH).   References  [1] K. Franinović and S. Serafin, Sonic Interaction Design: MIT Press, 2013. [2] J. Moynet, A. S. Jackson, and M. G. Wilson, French Theatrical Production in the Nineteenth Century: "L'Envers du Théatre" by M.J. Moynet, 1873 vol. 10. State University of New York, USA: Max Reinhardt Foundation with the Center for Modern Theatre Research, 1976. [3] S. Serafin and A. De Götzen, "An Enactive Approach to the Preservation of Musical Instruments Reconstructing Russolo's Intonarumori," Journal of New Music Research, vol. 38, pp. 231-239, 2009. [4] F. Keenan, "A Theatre Wind Machine as Interactive Sounding Object," presented at the International Conference on Live Interfaces (ICLI), Doctoral Colloquium, University of Sussex, 2016. [5] A. Farnell, Designing Sound: MIT Press Cambridge, 2010. [6] F. Napier, Noises Off: A Handbook of Sound Effects: London: JG Miller, 1936. [7] MaxMSP Available: https://cycling74.com/  [8] S. D. Monache, P. Polotti, and D. Rocchesso, "A Toolkit for Explorations in Sonic Interaction Design," in Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound, 2010, p. 1. [9] F. Avanzini, S. Serafin, and D. Rocchesso, "Modeling interactions between rubbed dry surfaces using an elasto-plastic friction model," in International Conference on Digital Audio Effects (DAFX), 2002. [10] S. Delle Monache, D. D. C. Drioli, F. Fontana, S. Papetti, P. Polotti, and D. Rocchesso, "Closing the loop of sound evaluation and design (CLOSED): Deliverable 2.1, algorithms for ecologically-founded sound synthesis: Library and documentation," Technical report, UNIVERONA (Verona)2007. [11] O. Lartillot and P. Toiviainen, "A Matlab toolbox for musical feature extraction from audio," presented at the International Conference on Digital Audio Effects (DAFX), Bordeaux, France, 2007.  