Nielsen, Chinton Møller

Pedersen, Michael Syskind

Experiments have shown that the performance of instantaneous gradient flow beamforming by Cauwenberghs et al. is reduced significantly in reverberant conditions. By expanding the gradient flow principle to convolutive mixtures, separation in a reverberant environment is possible. By use of a circular four microphone array with a radius of 5 mm, and applying convolutive gradient flow instead of just applying instantaneous gradient flow, experimental results show an improvement of up to around 14 dB can be achieved for simulated impulse responses and up to around 10 dB for a hearing aid application with real impulse responses

Online Research Database In Technology

GRADIENT FLOW CONVOLUTIVE BLINDSOURCE SEPARATIONMichael Syskind Pedersen∗Informatics and Mathematical Modelling, Building 321Technical University of Denmark, DK-2800 Kgs. Lyngby, DenmarkPhone: +45 4525 3904E-mail: msp@imm.dtu.dkWeb: imm.dtu.dk/~mspChlinton Møller NielsenTechnology & Innovation, Bang & OlufsenPeter Bangs Vej 15, DK-7600 Struer, DenmarkPhone: +45 9684 4058.Email: chn@bang-olufsen.dkAbstract. Experiments have shown that the performance of in-stantaneous gradient flow beamforming by Cauwenberghs et al.is reduced significantly in reverberant conditions. By expandingthe gradient flow principle to convolutive mixtures, separation ina reverberant environment is possible. By use of a circular four-microphone array with a radius of 5 mm, and applying convolutivegradient flow instead of just applying instantaneous gradient flow,experimental results show an improvement of up to around 14 dBcan be achieved for simulated impulse responses and up to around10 dB for a hearing aid application with real impulse responses.INTRODUCTIONThe gradient flow blind source separation technique proposed by Cauwen-berghs et al. [5] uses a four microphone array to separate 3 sound signals.The gradient flow can be regarded as a preprocessing step in order to enhancethe difference between the signals before a blind separation algorithm is ap-plied. The gradient flow technique requires small array sizes. Small arraysizes occur in some source separation applications such as hearing aids. Herethe physical dimensions of the microphone array may limit the separationperformance due to the very small difference between the recorded signals.In the literature, some attempts exist to separate sound signals by use of amicrophone arrays with a dimension of about 1 cm [2, 6, 7]. These techniques∗This work was supported by the Oticon Foundationare either based on beamforming, blind source separation [3], or a combina-tion of these techniques. The gradient flow method is able to estimate delayedversions of the source signals, as well as the source arrival angles. As shownin the simulations, the model may fail in reverberant environments, i.e. wheneach of the source signals is convolved in time. Here, a model is proposedthat extends the instantaneous gradient flow model to a convolutive gradientflow model. Simulations show that the convolutive model is able to cope withreverberant situations, in which the instantaneous model fails.INSTANTANEOUS GRADIENT FLOW MODELThe gradient flow model is described into details in [5, 8, 10, 11]. Eachsignal xpq is received by a sensor placed at location (p, q), which is shownin Figure 1. At a point in the coordinate system r, there is a delay, τ(r),between an incoming wavefront and the origin. The delay with respect to then’th source signal, sn is denoted as τn(r). It is assumed that the sources arelocated in the far-field. Hence the wavefront of the incoming waves is linear.Using that assumption the delay can be described the following way [5]:τ(r) ≈1cr · u, (1)where u is a unit vector pointing in the direction of the source and c is thevelocity of the wave.Now consider a sensor placed at the coordinates (p, q) as in Figure 1. Thetime delay from the source can be expressed asτnpq = pτn1 + qτn2 , (2)where τn1 = r1 ·un/c and τn2 = r2 ·un/c. τn1 and τn2 are the time differences inthe directions of the two orthogonal vectors r1 and r2 as shown in Figure 1.The point rpq can be described as rpq = pr1 + qr2.Description of fieldThe field is described by the incoming waves. At the center of the coordinatesystem, the contribution to the field from the n’th source is given by sn(t).By using the Taylor series expansion, the field from the n’th source at thepoint r in the coordinate system is given by sn(t + τn(r)), where [11]sn(t + τn(r)) = sn(t) +11!τn(r)s˙n(t) +12!(τn(r))2s¨n(t) + . . . . (3)Here, ˙ and¨denote the 1’st and 2’nd order derivative, respectively. Hence,the received signal at rpq can be written asxpq(t) =N∑n=1sn(t+ τn(r)) =N∑n=1sn(t)+11!τn(r)s˙n(t)+12!(τn(r))2s¨n(t)+ . . . .(4)Additionally, a noise term εpq(t) ∝ N(0, σ) can be added [5]. The receivedsignal can be approximated by using only the first two terms of (4):xpq(t) =N∑n=1sn(t + τn(r)) ≈N∑n=1sn(t) + τn(r)s˙n(t). (5)Notice, the Taylor approximation only holds, if the dimension of the array isnot too large (see [5] for details).pqsnrpqr1r2unFigure 1: Sensor placed at the point r with the position coordinates (p, q) so thatthe point is described the following way: rpq = pr1 + qr2, where r1 and r2 areorthogonal vectors. The time delay between (p, q) and the origin with respect tothe n’th source signal is denoted as τnpq.Gradient FlowThe spatial derivatives along the position coordinates (p, q) around the originin the coordinate system are found of various orders (i, j) [11].ξij(t) ≡∂i+j∂ip∂jqxpq(t) |p=q=0 (6)=N∑n=1(τn1 )(τn2 )di+jdi+jtsn(t) (7)Additionally, the derivative of the sensor noise νij(t) may be added.Corresponding to (5), the 0’th and 1’st order terms yield:ξ00(t) =∑nsn(t) (8)ξ10(t) =∑nτn1dsn(t)dt=∑nτn1 s˙n(t) (9)ξ01(t) =∑nτn2dsn(t)dt=∑nτn2 s˙n(t) (10)The estimates of the 0th order term ξ00(t), i.e. the estimate of the field inthe origin, can be obtained from the sensors as the average of the signalssince the sensors are symmetrically distributed around the origin at the fourcoordinates (0,1), (1,0), (0,-1) and (-1,0):ξ00(t) ≈14(x−1,0 + x1,0 + x0,−1 + x0,1). (11)The estimates of the two 1’st order derivatives can as well be estimated fromthe sensors:ξ10(t) =∂x∂p≈∆x∆p=x1,0 − x−1,01− (−1)=12(x1,0 − x−1,0) (12)ξ01(t) =∂x∂q≈∆x∆q=x0,1 − x0,−11− (−1)=12(x0,1 − x0,−1) (13)By taking the time derivative of ξ00(t), the following equation can be ob-tained.ddtξ00(t) =N∑n=1ddtsn(t) (14)Thus, the following instantaneous linear mixture can be obtained.ξ˙00(t)ξ10(t)ξ01(t) ≈1 · · · 1τ11 · · · τN1τ12 · · · τN2s˙1(t)...s˙N (t) (15)This equation is of the type x = As, where only x is is known. Assumingthat the source signals s are independent, (15) can be solved by independentcomponent analysis (see e.g. [3]).EXTENSION TO CONVOLUTIVE MIXTURESAs mentioned in [5], the instantaneous model (15), may be extended to con-volutive mixtures. In Figure 2, a situation is shown in which each sourcesignal does not only arrive from a single direction. Here, reflections of eachsn(t)an(l)sn(t-l)Figure 2: At the time t, a signal sn(t) originating from source n is arriving at thesensor array. At the same time, reflections from the same source arrive from otherdirections. These reflections are attenuated by the factor an and delayed by thetime lag l. Each signal received at the sensor array are therefore convolved mixturesof the original source signals. For simplification, only a single source and a singlereflection is shown.source signal may be present too. Each reflection is delayed by a factor land attenuated by an attenuated by a factor an(l). Now, similarly to (4) thereceived signal xpq at the sensor at position (p, q) is described asxpq(t) =N∑n=1L∑l=0an(l)sn(t + τn(r, l)− l), (16)where L is the assumed maximum time delay. Using the Taylor expansion,each received mixture can be written asxpq(t) =N∑n=1L∑l=0an(l)[sn(t−l)+τn(r, l)s˙n(t−l)+τn(r, l)2s¨n(t−l)+. . .](17)Using only the two first terms of the Taylor expansion and inserting τn(rpq, l) =pτn1 (l) + qτn2 (l), (17) can be written asxpq(t) ≈N∑n=1L∑l=0an(l)[sn(t− l) + (pτn1 + qτn2 )s˙n(t− l)]. (18)Similar to the instantaneous mixture case, the spatial derivatives of the con-volutive mixture can be found from (6). The 0’th order and the 1’st orderderivatives are then similarly to (8)–(10).ξ00(t) =∑n∑lan(l)sn(t− l) (19)ξ10(t) =∑n∑lan(l)τn1 (l)dsn(t− l)dt=∑n∑lan(l)τn1 (l)s˙n(t− l)(20)ξ01(t) =∑n∑lan(l)τn2 (l)dsn(t− l)dt=∑n∑lan(l)τn2 (l)s˙n(t− l)(21)The time derivative of ξ00(t) is expressed asξ˙00(t) =N∑n=1L∑l=0an(l)s˙n(t− l). (22)By expressing (22), (20) and (21) with matrix notation, the following expres-sion can be obtained:ξ˙00(t)ξ10(t)ξ01(t) ≈L∑l=0a1(l) · · · aN (l)a1(l)τ11 (l) · · · aN (l)τN1 (l)a1(l)τ12 (l) · · · aN (l)τN2 (l)s˙1(t− l)...s˙N (t− l) (23)=a1(l) · · · aN (l)a1(l)τ11 (l) · · · aN (l)τN1 (l)a1(l)τ12 (l) · · · aN (l)τN2 (l) ∗s˙1(t)...s˙N (t) , (24)where ∗ is the convolution operator. This is a convolutive mixture problem ofthe well-known type x = A ∗ s, where only an estimate of x is known. Theseestimates are found similarly to the instantaneous case from (11)–(13).FREQUENCY DOMAIN SEPARATIONIn [8], the Jade algorithm [4] was successfully applied to solve the instan-taneous mixing ICA problem (15). The Jade algorithm is based on jointdiagonalization of 4’th order cumulants. In order to solve the convolutivemixing problem (23), the problem is transformed into the frequency domain[9]. Hereby, the convolution in the time domain can be approximated bymultiplications in the frequency domain, i.e. for each frequency bin,ξ(f,m) ≈ A(f)s˙(f,m), (25)where m denotes the index of the frame of which the short-time Fourier trans-form STFT is calculated. f denotes the frequency. When solving the ICAproblem in the frequency domain, different permutations for each frequencyband may occur. In order to solve the frequency permutations, the methodsuggested in [1] has been used. It is assumed that the mixing matrices in thefrequency domain will be smooth. Therefore, the mixing matrix at frequencyband k, A(fk) is compared to the mixing matrix at band k − 1, A(fk−1).This is done by calculating the distance between any possible permutationsof A(fk) and A(fk−1), i.e.D(p) =∑i,j|a(p)ij (fk)− aij(fk−1)|, (26)Where p represents the p’th permutation. The permutation which yields thesmallest distance is assumed to be the correct permutation. Notice, for anN × N mixing matrix, there are N ! different permutations. Therefore thismethod becomes slow for large N . For a 3× 3 mixing matrix there are onlysix possible permutations.EXPERIMENTSSignals with synthetic impulse responsesThree speech sentences have been artificially mixed – two female speakersand one male speaker. The duration of each speech signal is 10 seconds, andthe speech signals have a sampling frequency of 20 kHz. A demonstrationof separated sounds is available at wwm.imm.dtu.dk/~msp. The microphonearray consists of four microphones. These are placed in a horizontal plane.An application for such a microphone array is shown in Figure 3, wherethe four microphones are placed in a single hearing aid. Here, the distancebetween the microphones and the center of the array is 5 mm. By use ofthe gradient flow method, it is possible to separate up to three sources [8].If there are more than three sources, an enhancement of the signals maybe achieved even though full separation of all sources isn’t possible. In thefirst experiment, a convolutive mixture of the three sources is simulated. Thearrival angles as well as the attenuation factor of the reverberations have beenchosen randomly. The maximum delay in this experiment has been chosen to25 samples. No sensor noise is added. The differentiator has been chosen to bea 1000 order FIR differentiator estimated with a least squares approach (eventhough a smaller order could be sufficient). The integrator is implemented asa first order Alaoui IIR filter as in [8]. Here, all 200000 samples have been usedto estimate the separated sounds. In order to achieve on-line separation, theseparated sounds may be estimated using blocks of shorter duration [8]. Theinstantaneous Jade performs well if only the direct sounds are present, butif reverberations are present too, the separation performance is significantlyreduced. The signal to interference ratio improvement is calculated as∆SIR(i) = 10 log( 〈(yi,si)2〉〈(∑j 6=i yi,sj )2〉)− 10 log( 〈(x10,si)2〉〈(∑j 6=i x10,sj )2〉), (27)Here, yi,sj is the i’th separated signal, where only the the j’th of the originalsignals has been sent through the mixing and unmixing system. x10,si is therecorded signal at the microphone at position (1,0) with only the i’th sourcesignal active. 〈·〉 denotes the expectation over all samples.The ∆SIR has been found for different DFT lengths as well as the case, wherethe instantaneous Jade has been been applied to the convolutive mixture.Hamming windows of the same length as the DFT has been used. An STFToverlap of 75% has been used. Table 5.1 shows the separation results of theconvolutive mixture. As it can be seen, the length of the DFT should be at10 mm10 mmNoseθϕFigure 3: Four microphones are placed in a hearing aid. The distance between themicrophones and the center of the array is 5 mm. By using such a configuration, itis possible to separate up to three independent sound sources. The azimuth angle,θ is defined according to the figure so that 0◦ is the direction of the nose. Likewise,the elevation angle ϕ is defined according to the figure so that 0◦ corresponds tothe horizontal plane. Both angles increase in the counterclockwise direction.least 256, in order to separate all three sources. It can as well be seen that,the FIR separating filters have to be significantly longer than the mixingfilters in order to ensure separation.Table 1: Three synthetic, artificially mixed speech signals have beenseparated. The maximum delay of each convolutive mixture is 25 sam-ples. The arrival elevation angles (ϕ) and the azimuth (θ) angles ofthe direct sounds are given. The ∆SIR have been found for the instan-taneous case and for different DFT lengths. The best separation isachieved with a DFT length of 256 or 512.UK Male UK female DK femaleθ 0◦ −112.5◦ −157.5◦ϕ 0◦ −21◦ 14◦Instantaneous JADE 9.5 dB 2.4 dB 2.5 dBDFT length=64 10.2 dB 2.4 dB 14.2 dBDFT length=128 11.0 dB 0.5 dB 11.5 dBDFT length=256 9.0 dB 9.2 dB 14.6 dBDFT length=512 8.9 dB 8.5 dB 16.5 dBDFT length=1024 6.5 dB 8.7 dB 16.2 dBTable 2: Signals generated from real impulse responses recorded bya four-microphone array placed in the right ear of a head and torsosimulator inside an anechoic room. No noise has been added. Here, the”UK female” is the hardest sound to separate. When listening to thesounds, all of them seems to be separated. When the DFT becomestoo long, the separation decreases. One explanation could be that theattempt to solve the permutation ambiguity fails.UK Male UK female DK femaleθ 0◦ −112.5◦ −157.5◦ϕ 0◦ −21◦ 14◦Instantaneous JADE 2.6 dB 2.2 dB 9.6 dBDFT length=64 10.6 dB 1.6 dB 8.3 dBDFT length=128 11.7 dB -0.4 dB 5.8 dBDFT length=256 13.1 dB 0.5 dB 6.1 dBDFT length=512 13.9 dB -0.2 dB 3.6 dBDFT length=1024 9.8 dB 0.0 dB 2.6 dBReal impulse responsesA four-microphone array has been placed in a dummy ear on the right sideof a head and torso simulator. In an anechoic room, impulse responses havebeen estimated from different directions. No sensor noise has been added.Due to the recordings in an anechoic room, the only reflections existing arethose from the head and torso simulator. The separation results are shownin Table 5.1. The performance is not as good as in the case of the syntheticimpulse responses. In contrast to the synthetic impulse responses, the micro-phones may have different amplitude and phase responses. This may reducethe performance. The ”UK female” seems to be the hardest sound to sepa-rate, but from the listening tests, it is easy to determine the separated soundfrom the two other speech signals.CONCLUSION AND FUTURE WORKThe performance by the instantaneous gradient flow beamforming is reducedsignificantly in reverberant mixtures. By expanding the gradient flow prin-ciple to convolutive mixtures, it is possible to separate convolutive mixturesin cases where the instantaneous gradient flow beamforming fails. It hasbeen shown that the extension to convolutive mixtures can be achieved bysolving a convolutive ICA problem (23) instead of solving an instantaneousICA problem (15). A frequency domain Jade algorithm has been used tosolve the convolutive mixing problem. In order to cope with a more difficultreverberant environment, other convolutive separation algorithms should beinvestigated. The mixing coefficients (23) are expected to have certain val-ues. E.g. the first row in the mixing matrices is significantly larger than thetwo other rows. Prior information on the coefficients of the mixing filterscould as well be used in order to improve the separation. The knowledge ofthe delays in the mixing filters may as well be used in order to determine thearrival angles of the mixed sounds.ACKNOWLEDGEMENTThe authors would like to thank Jan Larsen and Ulrik Kjems for useful com-ments and valuable discussions. We also acknowledge the financial supportby the Oticon Foundation.REFERENCES[1] W. Baumann, B.-U. Ko¨hler, D. Kolossa and R. Orglmeister, “Real Time Sep-aration of Convolutive Mixtures,” in ICA2001, San Diego, California, USA,December 9–12 2001, pp. 65–69.[2] W. Baumann, D. Kolossa and R. Orglmeister, “Beamforming-Based Convo-lutive Source Separation,” in ICASSP2003, Hong Kong, April 2003, vol. V,pp. 357–360.[3] J.-F. Cardoso, “Blind Signal Separation: Statistical Principles,” Proceedingsof the IEEE, vol. 9, no. 10, pp. 2009–2025, October 1998.[4] J.-F. Cardoso and A. Souloumiac, “Blind beamforming for non Gaussian sig-nals,” IEE Proceedings-F, vol. 140, no. 6, pp. 362–370, December 1993.[5] G. Cauwenberghs, M. Stanacevic and G. Zweig, “Blind Broadband Source Lo-calization and Separation in Miniature Sensor Arrays,” in IEEE Int. Symp.Circuits and Systems (ISCAS’2001), May 6–9 2001, vol. 3, pp. 193–196.[6] G. W. Elko and A.-T. N. Pong, “A simple adaptive first-order differentialmicrophone,” in Proceedings of 1995 Workshop on Applications ofSingle Processing to Audio and Accoustics, October 15–18 1995, pp.169–172.[7] T. J. Ngo and N. Bhadkamkar, “Adaptive blind separation of audio sourcesby a physically compact device using second order statistics,” in First Inter-national Workshop on ICA and BSS, Aussois, France, January 1999, pp.257–260.[8] C. M. Nielsen, Gradient Flow Beamforming utilizing IndependentComponent Analysis, Master’s thesis, Aalborg University, Institute of Elec-tronic Systems, January 5 2004.[9] V. C. Soon, L. Tong, Y. F. Huang and R. Liu, “A wideband blind identifi-cation approach to speech acquisition using a microphone array,” in IEEEInternational Conference on Acoustics, Speech, and Signal Process-ing (ICASSP-92), San Francisco, California USA: IEEE, March 23–26 1992,vol. 1, pp. 293–296.[10] M. Stanacevic, G. Cauwenberghs and G. Zweig, “Gradient Flow BroadbandBeamforming and Source Separation,” in ICA’2001, December 2001.[11] M. Stanacevic, G. Cauwenberghs and G. Zweig, “Gradient Flow AdaptiveBeamforming and Signal Separation in a Miniature Microphone Array,” inICASSP2002, Florida, USA, May 13–17 2002, vol. IV, pp. 4016–4019.

Gradient Flow Convolutive Blind Source Separation

http://orbit.dtu.dk/ws/files/4195910/imm3174.pdf

Gradient Flow Convolutive Blind Source Separation

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology