This paper focuses on the use of GPGPU (General- Purpose computing on Graphics Processing Units) for audio processing. This is a promising approach to problems where a high parallelization of tasks is desirable. Within the context of binaural spatialization we will develop a convolution engine having in mind both offine and real-time scenarios, and the support for multiple sound sources. Details on implementations and strategies used with both dominant technologies, namely CUDA and OpenCL, will be presented highlighting both advantages and issues. Comparisons between this approach and typical CPU implementations will be presented as well as between frequency (FFT) and time-domain approaches. Results will show that benefits exist in terms of execution time for a number of situations

Mauro, Davide Andrea

English

Marshall University

Marshall UniversityMarshall Digital ScholarWeisberg Division of Computer Science FacultyResearch Weisberg Division of Computer Science3-2013Audio convolution on GPUs: a follow-upDavide Andrea MauroMarshall University, maurod@marshall.eduFollow this and additional works at: https://mds.marshall.edu/wdcs_facultyPart of the Computational Engineering CommonsThis Conference Proceeding is brought to you for free and open access by the Weisberg Division of Computer Science at Marshall Digital Scholar. Ithas been accepted for inclusion in Weisberg Division of Computer Science Faculty Research by an authorized administrator of Marshall Digital Scholar.For more information, please contact zhangj@marshall.edu, beachgr@marshall.edu.Recommended CitationMauro DA. Audio convolution on GPUs: a follow-up. Paper presented at the AIA-DAGA Conference on Acoustics, March, 2013,Merano (Italy)Audio convolution on GPUs: a follow-upDavide A. Mauro1,21 Laboratorio di Informatica Musicale (LIM), Universita´ degli Studi di Milano, Milan, Italy2 Institut Mines-Te´le´com, TE´LE´COM ParisTech, CNRS-LTCI, Paris, Francedavide-andrea.mauro@telecom-paristech.frIntroductionThis paper focuses on the use of GPGPU (General-Purpose computing on Graphics Processing Units) foraudio processing. This is a promising approach toproblems where a high parallelization of tasks is de-sirable. Within the context of binaural spatializationwe will develop a convolution engine having in mindboth offline and real-time scenarios, and the support formultiple sound sources. Details on implementations andstrategies used with both dominant technologies, namelyCUDA and OpenCL, will be presented highlightingboth advantages and issues. Comparisons between thisapproach and typical CPU implementations will bepresented as well as between frequency (FFT) and time-domain approaches. Results will show that benefits existin terms of execution time for a number of situations.Convolution EnginesEven if the process is well known and understood interms of mathematics, the realization of implementationsthat work in real-life scenarios is not trivial. One ofthe greatest obstacle is the computational complexitythat convolution requires both in the time and frequencydomain approaches. This means that the problem couldbe theoretically solved but the computer architecturedoes not allow it to be solved in a reasonable time forsome practical cases of interest. As shown in Figure 1 thesystem requires as input an anechoic signal (monophonic)and a impulse response (stereo) and the overall outputwill be two channel spatialized sound that can feed bothheadphones or loudspeakers (with crosstalk cancelationalgorithms [3]). For a review of the state of the art inconvolution engines please refer to [7].Overlap-add algorithmSince the size of the filter kernel can become very high, itis not convenient to use a single window to transform theentire signal so a number of methods can be implementedto overcome this. We choose to use a method calledOverlap-add (OA, OLA). It is an efficient way to evaluatethe discrete convolution of a very long signal x[n] with afinite impulse response (FIR) filter h[n]. The concept isto divide the problem into multiple convolutions of h[n]with short segments of x[n]:y[n] = x[n]∗h[n] :=∞∑m=−∞h[m]x[n−m] =M∑m=1h[m]x[n−m](1)where h[m] = 0 for m outside the region [1,M ].Figure 1: The workflow diagram of the system.Figure 2: Schematic view of the overlap-add convolutionmethod.xk[n] :={x[n + kL] n = 1, 2, ···, L0 otherwise(2)where L is an arbitrary segment length.x[n] =∑kxk[n− kL] (3)So y[n] can be written as a sum of convolutions:y[n] =(∑kxk[n− kL])∗ h[n] =∑k(xk[n− kL] ∗ h[n])(4)The method is depicted in Figure 2It is particularly interesting for our tasks since it workson independent pieces of input and thus is well suited fora parallelized approach such as one that employs a GPU.ImplementationsIn order to make comparisons with the GPU imple-mentations that we will present we need a referenceimplementation that can serve as a basis in terms ofexecution time and bitwise precision. For this reasonthree different prototypes have been developed that usedifferent algorithms.The first two prototypes are Matlab scripts that useboth a Time Domain and a Frequency Domain approach.Since the computational complexity for the Time Domainapproach is O(n2) this can not be used when the filterkernels are big. In our experiments, according to aMax/MSP implementation that will be introduced inthe following section, we choose to limit the size to 256samples.The frequency domain implementation (presented in [6])will be used to validate the results in terms of bitwiseprecision. Since Matlab is mainly intended as a pro-totyping environment there is no focus on performanceand every other implementation can outperform ourMatlab testbase by orders of magnitude. Moreover, thisimplementation works only in “direct mode”; this impliesthat a single FFT is performed for the entire signal andtherefore the algorithm may not be applicable for longsequences due to memory constraints or implementationlimits.One of the CPU implementation is written in C++ and isbased on the FFTW3 library (see [5]). It is based on thearchitecture previously presented and implements bothmodalities (Direct and OLA) previously discussed.The FFTW library itself is based on Cooley-Tukey algo-rithm [4]. As presented by the authors, the interactionof the user with FFTW occurs in two stages: planning,in which FFTW adapts to the hardware, and execution,in which FFTW performs useful work for the user.To compute a DFT, the user first invokes the FFTWplanner, specifying the problem to be solved. Theproblem is a data structure that describes the “shape”of the input data - array sizes and memory layouts -but does not contain the data itself. In return, theplanner yields a plan, an executable data structure thataccepts the input data and computes the desired DFT.Afterwards, the user can execute the plan as many timesas desired.A CUDA convolution engineFor the CPU implementation with CUDA we were ableto implement both Direct and OLA algorithm. Weconsider the benefits of both approaches in the followingsection while presenting performance comparisons. ForFFT we use a library called CUFFT which is actuallybased on FFTW3 library with some other optimizationsspecifically designed for GPUs. One of the current issueis the CUFFT limit of 64 millions of points.An OpenCL convolution engineThe FFT used is based on Apple implementation (http://developer.apple.com/library/mac/#samplecode/OpenCL_FFT/Introduction/Intro.html). One of thecurrent limitations is that the factorization algorithmsworks only for powers of 2 (radix-2). So the payloadshould be adapted to make the sum with the length ofthe filter kernel to be the closest greater power of 2.The cGPUconv prototypeFrom a number of the previously cited prototypes wederived a single application that allows the user to choosebetween a CPU- or a GPU-based algorithm and betweena direct mode (a single window for the entire signal) andan Overlap-add mode. It is structured as a “wrapper”around the single module that has the capability ofopening audio files and writing them back to disk thanksto libsndfile (see [2]). There is also an early-stage supportfor realtime convolution acquiring sound from audioinput and playing it back through audio output. It isa command line tool that compiles and executes both onMicrosoft Windows, Apple OSX, and Linux applicationsas long as they have, or there exists a version of:• Boost Library;• PortAudio;• Libsndfile for I/O;• FFTW3 library for CPU implementation;• CUDA (Versions 4 or 5) Framework;• OpenCL (1.1) driver.The program can be adapted removing functionalitiesprovided by subsets of the previous requirements byremoving the components that make use of those pre-requisites. The source code is available from the authorathttps://code.google.com/p/cgpuconv/.Performance ComparisonsPerformances of these algorithms depends on the sizeof input. Therefore, to characterize the “trade-off”, wetested them with different input sizes. We then computethe time spent on the convolution procedure, excludingthe load procedure that reads from audio files and thewrite to disk procedure for the results, which are collat-eral to our primary goal. A special case is representedby the first execution for both the CUDA and OpenCLimplementation where for the former there exists someextra time devoted to the load of the environment whilefor the latter, apart from the aforementioned setup, wehave to take into account the time that the driver allocateto compile kernel functions.The algorithms were tested on a number of differentplatform:• A1: Apple Macbook Pro 13.3” (MacBookPro5,5),OSX 10.6.8. Intel Core 2 Duo processor @2,53 GHz,8 GB Ram, NVIDIA GeForce 9400GM 256 MBvRAM shared memory. OpenCL 1.1, CUDA 4.0.• A2: Asus M50S, Ubuntu 12.10 32bit. Intel Core2 Duo processor @2,50 GHz, 4 GB Ram NVIDIA 0 50 100 150 200 250 300 350 400 45027 28 29 210 211 212 213 214 215 216Execution time (ms)N. of samplesComparation of execution time for Direct ModeA1 CPUA1 CUDAA1 OpenCLA2 CPUA2 CUDAA3 CPUA3 CUDAA3 OpenCLFigure 3: Execution time for Direct mode depending oninput size. Part 1GeForce 9500M G, 512 MB vRAM dedicated mem-ory. CUDA 5.0.• A3: ASUS CG8250 Windows 8 64bit. Intel Corei7-2600 processor @3,40 GHz, 8 GB Ram NVIDIAGeForce GTX 560 Ti, 1024 MB vRAM dedicatedmemory. OpenCL 1.1, CUDA 5.0.All the audio files are high quality PCM uncompressedfiles and have a sample rate of 96 kHz and a quantizationword of 24 bit. With this bit depth the theoreticaldynamic range is ∼ 144 dB.For each algorithm we measured the difference computedbetween the signal under test and the reference (comingfrom the Matlab implementation) with a phase inversion.So the difference on a sample by sample basis gives us anew signal that can be used as a degree of similaritybetween the two original signals. For each and everyproposed approach this signal is below -122 dB FS (dBon the full scale) meaning there is no practical difference,and the result is in the order of magnitude of the noisefloor.Coming to the execution time of the algorithms wepropose a summary of the results presented in Figures3 - 4, and 5 - 6. Results are depicted as a function of thenumber of input samples, averaged over 100 runs, resultsare split in two parts for better clarity.The exploit of CUDA 5 framework, as well as the newarchitectures supported by PCIxpress 3.0 standard showthe continuous improvements that can be obtained usingGPGPU. We also present in Table 1 results for a “real-case scenario”. We have a violin sound that is threeminutes long and a reverberant impulse response of 1second (sample rate 96kHz):• Input: 17703123 samples (∼3’10”)• Kernel: 96000 (∼1”)Please note that “-” occurs when there is not enoughfree RAM / vRAM to handle the data. The ideabehind this work is to have a system that can runon most home computer so the relatively old and low 0 5000 10000 15000 20000 25000217 218 219 220 221 222 223Execution time (ms)N. of samplesComparation of execution time for Direct ModeA1 CPUA1 CUDAA1 OpenCLA2 CPUA2 CUDAA3 CPUA3 CUDAA3 OpenCLFigure 4: Execution time for Direct mode depending oninput size. Part 2 0 50 100 150 200 250 300 350 400 45027 28 29 210 211 212 213 214 215 216Execution time (ms)N. of samplesComparation of execution time for Overlap-addA1 CPUA1 CUDAA1 OpenCLA2 CPUA2 CUDAA3 CPUA3 CUDAA3 OpenCLFigure 5: Execution time for Overlap-add depending oninput size. Part 2 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000217 218 219 220 221 222 223Execution time (ms)N. of samplesComparation of execution time for Overlap-addA1 CPUA1 CUDAA1 OpenCLA2 CPUA2 CUDAA3 CPUA3 CUDAA3 OpenCLFigure 6: Execution time for Overlap-add depending oninput size. Part 2(A1/A2/A3) (ms) Direct OLACPU -/-/- 9699/10160/3543CUDA -/-/- 6181/ 5930/1381OpenCL 7486/-/658 6699/-/1526Table 1: Performance comparisons. Time in ms.powerful graphic card is a good example of what can beachieved with standard equipment. There are differencebetween implementations and this can be explained bythe different way of encoding real and complex numbers.Also note that there does not exist a concept of “paging”for video RAM so if a structure is too big to fit in memorythere is no automatic way to handle the situation.Conclusions and Future WorksIn this paper we presented a number of prototypes thatare suitable for spatialization of sounds exploiting thepotentialities of GPUs. Some issues are still presentbut we want to point out that the basic concepts hereexpressed are valid and mark a profitable direction.Performance results suggest that for a number of realcase applications there are benefits that can be at leastof 1/3 of the execution time, compared to the referenceCPU implementation, using old framework and outdatedhardware. and can be further improved with otherGPU-specific, but not hardware specific, optimizations.Benefits are increasingly evident as the size of thefilter kernel grows and this is particularly useful forconvolution with long reverberant impulse responses (e.g.BRIRs) that can be employed in order to render realenvironments.Some of the limitations previously presents, in CUDA4.0 framework, or like bandwidth with PCIxpress 2.0standard are overcame by new architecture that allowsto stream data back and forth from the GPU in realtimeand solve a number of the previous issues.Further improvements will consider the application ofpartitioned convolution, a technique first introduced byArmelloni, Giottoli, and Farina [1]. It is also possible topartition in a non-uniform manner for latency reductionpurposes [9]. [8] also used this algorithm for an efficientFFT implementation on GPU.AcknowledgmentsThe author would like to thank Lorenzo Picinali andBrian F.G. Katz for extensive conversations concerningthese researches, and Andrea Trentini for the remotehost for testing purposes. This work has been partiallyfunded by the Enhanced Music Interactive Platform forInternet User (EMIPIU) project and by the EuropeanCommission under contract “FP7-287723 REVERIE”.References[1] E. Armelloni, C. Giottoli, and A. Farina. Implemen-tation of real-time partitioned convolution on a DSPboard. In Applications of Signal Processing to Audioand Acoustics, 2003 IEEE Workshop on., pages 71–74. IEEE, 2003.[2] E. Castro Lopo. Libsndfile [computer software].Retrieved December, 28:2005, 2005.[3] E. Y. Choueiri. Optimal crosstalk cancellation forbinaural audio with two loudspeakers. 2011.[4] J. Cooley and J. Tukey. An algorithm for the machinecalculation of complex Fourier series. Math. Comput,19(90):297–301, 1965.[5] M. Frigo and S. Johnson. The design andimplementation of FFTW3. Proc. IEEE (SpecialIssue on Program Generation, Optimization, andPlatform Adaptation), 93:216–231, 2005.[6] D. A. Mauro. Effetti della distanza nella spazial-izzazione e localizzazione binaurale. B.A. Thesis,Universita` degli Studi di Milano, July 2006.[7] D. A. Mauro. On Binaural Spatialization and the useof GPUs for audio processing. PhD thesis, Universita`degli Studi di Milano, March 2012.[8] M. Rush. Modeling a GPU-based Convolution EngineEEC 289Q.[9] F. Wefers and M. Vorla¨nder. Potential ofnon-uniformly partitioned convolution with freelyadaptable fft sizes. In Audio Engineering SocietyConvention 133, 2012.

Audio convolution on GPUs: a follow-up

https://mds.marshall.edu/cgi/viewcontent.cgi?article=1057&context=wdcs_faculty

Audio convolution on GPUs: a follow-up

Abstract

Similar works

Full text

Available Versions

Marshall University