In this paper, a method for multi-channel pitch estimation is proposed. The method is a maximum likelihood estimator and is based on a parametric model where the signals in the various channels share the same fundamental frequency but can have different amplitudes, phases, and noise characteris-tics. This essentially means that the model allows for differ-ent conditions in the various channels, like different signal-to-noise ratios, microphone characteristics and reverberation. Moreover, the method does not assume that a certain array structure is used but rather relies on a more general model and is hence suited for a large class of problems. Simulations with real signals shows that the method outperforms a state-of-the-art multi-channel method in terms of gross error rate. Index Terms — Pitch estimation, microphone arrays, multi-channel audi

Christensen, Mads Græsbøll

English

Mads Græsbøll Christensen

CiteSeerX

Multi-channel maximum likelihood pitch estimation

In this paper, a method for multi-channel pitch estimation is proposed. The method is a maximum likelihood estimator and is based on a parametric model where the signals in the various channels share the same fundamental frequency but can have different amplitudes, phases, and noise characteristics. This essentially means that the model allows for different conditions in the various channels, like different signal-to-noise ratios, microphone characteristics and reverberation. Moreover, the method does not assume that a certain array structure is used but rather relies on a more general model and is hence suited for a large class of problems. Simulations with real signals shows that the method outperforms a state-of-the-art multi-channel method in terms of gross error rate

    Aalborg UniversitetMulti-Channel Maximum Likelihood Pitch EstimationChristensen, Mads GræsbøllPublished in:2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)DOI (link to publication from Publisher):10.1109/ICASSP.2012.6287903Publication date:2012Document VersionAccepted author manuscript, peer reviewed versionLink to publication from Aalborg UniversityCitation for published version (APA):Christensen, M. G. (2012). Multi-Channel Maximum Likelihood Pitch Estimation. In 2012 IEEE InternationalConference on Acoustics, Speech and Signal Processing (ICASSP) (Vol. 2012, pp. 409-412). IEEE. I E E EInternational Conference on Acoustics, Speech and Signal Processing. Proceedingshttps://doi.org/10.1109/ICASSP.2012.6287903General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: December 26, 2020MULTI-CHANNEL MAXIMUM LIKELIHOOD PITCH ESTIMATIONMads Græsbøll ChristensenDept. of Architecture, Design & Media TechnologyAalborg University, Denmarkmgc@create.aau.dkABSTRACTIn this paper, a method for multi-channel pitch estimation isproposed. The method is a maximum likelihood estimatorand is based on a parametric model where the signals in thevarious channels share the same fundamental frequency butcan have different amplitudes, phases, and noise characteris-tics. This essentially means that the model allows for differ-ent conditions in the various channels, like different signal-to-noise ratios, microphone characteristics and reverberation.Moreover, the method does not assume that a certain arraystructure is used but rather relies on a more general modeland is hence suited for a large class of problems. Simulationswith real signals shows that the method outperforms a state-of-the-art multi-channel method in terms of gross error rate.Index Terms— Pitch estimation, microphone arrays,multi-channel audio1. INTRODUCTIONAn important property of audio and speech signals is thepitch, and the pitch is one of the most frequently used fea-tures in the processing and analysis of acoustic signals. Inmany cases, the pitch of a signal is simply related to thefundamental frequency, which describes the number of timesa periodic signal repeats per time interval. Bandlimited pe-riodic signals can be expressed as a finite weighted sum ofharmonically related sinusoids having frequencies that areinteger multiples of this fundamental frequency. The prob-lem of finding the fundamental frequency from such periodicsignals buried in noise is called fundamental frequency orpitch estimation. Over the years, many different methods forpitch estimation have been devised from classical approachessuch as harmonic summation and product methods [1] tomore recent methods such as harmonic fitting [2], maximumlikelihood [3], optimal filtering [4], subspace methods [5] andBayesian methods [6,7]. For an overview, we refer the readerto [5] and the references therein. Despite the host of methodsdevoted to pitch estimation, it appears that very few methodshave been devised for estimating the pitch when multiplechannels are available, as would be the case in microphonearray processing or in studio recordings of music. Somemethods do exist, however, including those of [8, 9] and thejoint pitch and localization methods of [10, 11]. There areseveral reasons why multi-channel pitch estimation shouldbe pursued when multiple channels are available. Firstly,the presence of more data is always beneficial in frequencyestimation problems. Secondly, the conditions under whichthe signals have been recorded may differ from channel tochannel and it may be difficult to pick one channel a priorias having the best conditions. Hence, a method using allchannels is preferable. Also, spatial localization may makeit fairly easy to attenuate noise from certain angles, althoughwe shall not seek to exploit this here.In this paper, a novel method for multi-channel pitch es-timation is presented. It is a maximum likelihood estimatorbased on a Gaussian assumption and a parametric model ofsignal of interest. In this model, the fundamental frequencyis shared across channels while amplitudes, phases and noisecharacteristics are allowed to be different for each channel.The model thus takes into account that, e.g., the signal in eachchannel may have been filtered and that the noise level maybe different. Hence, the presented method is based on a quitegeneral model that can be assumed to work in many differentsituations. An important aspect of this work is that the inte-gration across channels is done in a mathematically tractablemanner.The rest of the paper is organized as follows: In Section2, the underlying parametric model and statistical assump-tions are presented after which the proposed method is de-rived in Section 3. Then, in Section 4 some experimental re-sults demonstrating the advantages of the proposed methodare presented. Finally, we conclude on the work in Section 5.2. FUNDAMENTALSWe will now present the signal model and associated as-sumptions. The proposed method operates on a signal vectorxk(n) ∈ CM at time n (termed a snapshot) for the kthchannel, defined as xk(n) = [ xk(n) · · · xk(n+M − 1) ]Twhich is constructed from the observed signal from the kthchannel xk(n), for n = 0, . . . , N − 1. We model this vectoras a sum of L harmonically related complex sinusoids inGaussian noise ek having covariance matrix Qk, i.e.,xk(n) = Z(n)ak + ek(n), (1)with ak = [Ak,1ejφk,1 · · · Ak,Lejφk,L ]T being a vector con-taining the complex amplitudes of the signal in the kth chan-nel. Moreover, the matrix Z(n) is a Vandermonde matrix attime n, defined as Z(n) =[z1(n) · · · zL(n)], wherethe mth entry of the column vector zl(n) ∈ CM is definedas [zl(n)]m = ejω0l(n+m−1) with ω0 ∈ Ω0 being the funda-mental frequency, i.e., the parameter we seek to find in the setΩ0 = (0, 2π/L). We assume that G vectors xk(n) have beenobserved for each channel. We define the signal and noiseparameter vector θk for the kth channel containing the fun-damental frequency ω0, the complex amplitudes {Ak,lejφk,l}and the noise covariance matrix Qk. Regarding the model or-der L, we remark that it is possible to extend the proposedmethod to joint fundamental frequency and order estimationusing the MAP principle [5]. However, for simplicity, we donot describe that here. Assuming that Qk is invertible, thelikelihood function (for complex signals) of xk(n) can thenbe written asp(xk(n);θk) =1πMdet(Qk)e−eHk (n)Q−1k ek(n), (2)with det(·) denoting the matrix determinant. Now, assum-ing that the deterministic part is stationary and ek(n) is in-dependent and identically distributed over n as well as in-dependent over k, the likelihood of the observed set of vec-tors {{xk(n)}G−1n=0 }Kk=1 (or {xk(n)} for short) across chan-nels can be written asp({xk(n)}; {θk}) =K∏k=1G−1∏n=0p(xk(n);θk)=K∏k=11πMGdet(Qk)Ge−∑G−1n=0 eHk (n)Q−1k ek(n).(3)There are several ways in which the noise covariance matrixcan be estimated, but they are, however, all fairly involvedand they are hence best avoided. Moreover, it may be difficultto say anything about the noise covariance matrix Qk a pri-ori. In that case, the best solution is to assume that the noiseis white in each channel1 but that the noise has different vari-ance σ2k, i.e.,Qk = σ2kI. Similar arguments hold regardingthe assumption of the noise being independent across chan-nels. With the above assumptions, the likelihood function fora single snapshot for channel k reduces top(xk(n);θk) =1(πσ2k)GMe− 1σ2k‖ek(n)‖2, (4)and the log-likelihood function is then ln p(xk(n);θk) =−M ln (πσ2k) − 1σ2k ‖ek(n)‖2, which across all channels and1The white Gaussian distribution can be shown to be the one maximizesthe entropy of the noise [12].snapshots under the aforementioned conditions yieldsln p({xk(n)}; {θk}) =−GMK∑k=1ln (πσ2k)−K∑k=1G−1∑n=0‖ek(n)‖2σ2k. (5)3. PROPOSED METHODWe will now proceed to derive the proposed estimator. Todo this, we first observe that the noise variance σ2k and thecomplex amplitude vector ak are specific to channel k whilethe fundamental frequency in Z is shared among all channels.Hence, the two former parameters can be estimated directlyfrom the individual channels (for a particular fundamental fre-quency candidate). The maximum likelihood estimate of theamplitudes for channel k can readily be shown to beâk =(G−1∑n=0ZH(n)Z(n))−1 G−1∑n=0ZH(n)xk(n). (6)This, in turn, can be used to form a noise estimate for n =0, . . . , G − 1 as êk(n) = xk(n) − Z(n)âk and, from this, amaximum likelihood noise variance estimate for channel k asσ̂2k =1GMG−1∑n=0‖êk(n)‖2. (7)Inserting these quantities into (5) then yields the concentratedlog-likelihood for channel k at time n ln p(xk(n);ω0) =−M lnπ−M ln σ̂2k, which depends only on the fundamentalfrequency ω0, and the maximization of this function over thefundamental frequency would then leads to the maximumlikelihood estimate for channel k. For all n and k, this yieldsln p({xk(n)};ω0) = −GMK lnπ −GMK∑k=1ln σ̂2k. (8)The maximum likelihood estimator (MLE) can finally bestated asω̂0 = arg minω0∈Ω0K∑k=1ln σ̂2k. (9)To summarize how the estimator works for each candidatefundamental frequency ω0 ∈ Ω0, the amplitudes are firstfound using (6) where after the noise variance is estimated foreach channel k using (7). Then, the variances are integratedacross channels as in (8) and the fundamental frequency canthen be determined using (9). An interesting special casecan be obtained as follows. For M = N only one signalvector will be available for each channel. We denote this asxk = xk(0) and similarly for the other quantities. Then thechannel k estimators reduce toâk =(ZHZ)−1ZHxk and σ̂2k =xHk Π⊥ZxkN(10)Parameter Value Parameter ValueSound vel. 340 m/s Room Dim. [5 4 6 ] mSource pos. [2 3.5 2] m Samples 4096Reverb time 0.4 s Mic. Hypercard.Ref. order -1 Number of Mic. 4Mic. pos. Random Mic. orient. RandomTable 1. Experimental settings for the mirror method.where ΠZ = Z(ZHZ)−1ZH and Π⊥Z = I − ΠZ . Not-ing then that the columns of Z are asymptotically orthogonal,i.e., limM→∞MΠZ = ZZH , the resulting estimator can bewritten as follows:ω̂0 = arg minω0∈Ω0K∑k=1ln(‖xk‖2 −1N‖ZHxk‖2). (11)Finally, we observe that ‖ZHxk‖2 is just the sum over thesquared magnitude of the Fourier transform of xk(n), denotedXk(ω) evaluated in a set of frequencies (in this case those ofthe candidate harmonics), i.e., it can be evaluated efficientlyusing an FFT as ‖ZHxk‖2 =∑Ll=1 |Xk(ω0l)|2. This can beseen as a simple extension of the classical harmonic summa-tion method [1]. It should be noted that the amplitude esti-mate in (6) also can be computed efficiently for large M thisway, although a phase shift must also be introduced to com-pensate for Z(n) being time-varying. It should also be notedthat had we assumed that the noise variance was known, theresult would have been different. In that case, the estimatorwould reduce to the maximizer of the weighted sum over thespectra across all channels, i.e.,∑Kk=1∑Ll=1 |Xk(ω0l)|2/σ2k.Moreover, if we also assume that the noise variance is thesame for all channels, the resulting estimator would simplybe the maximizer of∑Kk=1∑Ll=1 |Xk(ω0l)|2. Both are tobe contrasted with the sum over the logarithm in (11), whichshows that the way in which the cost function should be inte-grated across channels depends on whether the noise varianceis known and the same.4. EXPERIMENTAL RESULTSTo investigate the performance of the proposed method weproceed as follows. We will follow a procedure similar to thetest methodology of [9]. A set of single-pitch audio mono sig-nals from the EBU SQAM discs is used, namely the trumpet,violin and horn signals. The signals are down-sampled by afactor of four from the original 44.1 kHz sampling frequencyand converted to complex signals using the Hilbert transform,and the signals are processed (by all algorithms) in segmentsof 40 ms with 50 % overlap and an FFT size of 8192. Fromthese signals, four different channels are generated using E.Habets’ implementation2 of the mirror method [13] with set-2http://home.tiscali.nl/ehabets/rir_generator.htmltings as shown in Table 1. Note that microphone positions andorientations were picked randomly. To demonstrate the meritsof the proposed method, we add white Gaussian noise to eachchannel and test two different scenarios: one in which thenoise level is the same in all channels, a scenario we will referto as symmetrical, and one where the noise level is different ineach channel, which we will refer to as asymmetrical. For theasymmetrical scenario, the SNRs in the individual channelswere -15, -10, -5, and 0 dB, respectively, offset by an overallSNR value. A ground truth pitch is estimated from the cleanmulti-channel signal using the MPF method [9]. We willcompare the proposed method to the MPF method, which,in [9], was demonstrated to outperform a multi-channel ver-sion of YIN [14] and the method of [8]. We will evaluatethe fast, approximate version in (11) of the proposed method(denoted MLE), which relies on FFTs to compute the log-likelihood. Finally, we will, for reference, also compare tothe performance obtained with the simple multi-channel ex-tension of the classical harmonic summation method (denotedHS) mentioned in Section 3, which is also an approximatemaximum likelihood estimator when the noise variance is thesame in all channels. For the methods that require that themodel order is estimated, we used the MAP criterion [5]. Asin [9], we will measure the gross error rate (GER), definedas a relative error of more than 20 % relative to the ground-truth, under different conditions. It should be noted that thismethodology favors the MPF method as any consistent errorin estimates obtained from the clean and noisy multi-channelsignals will not be punished for this method. The results areshown in Figures 1(a) and 1(b) for the two scenarios, respec-tively. A number of observations can be made from the fig-ures. The proposed method performs the best for both sce-narios having the lowest GER. The HS method can be seento perform well for the symmetrical case, as predicted by thetheory. However, it can also be seen to to break down whenthe noise level differs between channels. The MPF methodis capable of handling this scenario due to the normalizationprocedure in [9]. However, it can generally be seen to be moresensitive to low SNRs than the proposed method.5. CONCLUSIONIn this paper, a novel multi-channel pitch estimator has beenproposed. The method is based on a maximum likelihood ap-proach, and it is based on a parametric model where the signalin each channel is modeled as a sum of harmonically relatedsinusoids in noise. The amplitudes and phases are allowed tovary across channels to account for different acoustic propa-gation paths and the signal-to-noise level is allowed to varyas well. The model is hence quite general and can be usedin many different scenarios. Simulations demonstrate that themethod generally performs well and outperforms a state-of-the-art method, especially under adverse conditions.−20 −15 −10 −5 0 5 10 15 2000.10.20.30.40.50.60.70.80.91SNR [dB]Gross Error Rate  MLEHSMPF(a)−20 −15 −10 −5 0 5 10 15 2000.10.20.30.40.50.60.70.80.91SNR [dB]Gross Error Rate  MLEHSMPF(b)Fig. 1. Performance measured in terms of gross error rate as a function of the SNR in dB for (a) symmetrical noise level and(b) asymmetrical noise level. The SNR for the latter case is the overall SNR.6. REFERENCES[1] M. Noll, “Pitch determination of human speech by har-monic product spectrum, the harmonic sum, and a maxi-mum likelihood estimate,” in Proc. Symposium on Com-puter Processing Communications, 1969, pp. 779–797.[2] H. Li, P. Stoica, and J. Li, “Computationally efficientparameter estimation for harmonic sinusoidal signals,”Signal Processing, vol. 80, pp. 1937–1944, 2000.[3] B. G. Quinn and P. J. Thomson, “Estimating the fre-quency of a periodic function,” Biometrika, vol. 78(1),pp. 65–74, 1991.[4] M. G. Christensen, J. L. Højvang, A. Jakobsson, andS. H. Jensen, “Joint fundamental frequency and orderestimation using optimal filtering,” EURASIP J. on Ad-vances in Signal Processing, vol. 2011(1), pp. 13, 2011.[5] M. G. Christensen and A. Jakobsson, Multi-Pitch Esti-mation, vol. 5 of Synthesis Lectures on Speech & AudioProcessing, Morgan & Claypool Publishers, 2009.[6] S. Godsill and M. Davy, “Bayesian harmonic models formusical pitch estimation and analysis,” in Proc. IEEEInt. Conf. Acoust., Speech, Signal Processing, 2002,vol. 2, pp. 1769–1772.[7] A. T. Cemgil, H. J. Kappen, and D. Barber, “A genera-tive model for music transcription,” IEEE Transactionson Audio, Speech, and Language Processing, vol. 14,no. 2, pp. 679–694, Mar. 2006.[8] L. Armani and M. Omologo, “Weighted auto-correlation-based f0 estimation for distant-talking in-teraction with a distributed microphone network,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process-ing, 2004, vol. 1, pp. 113–116.[9] F. Flego and M. Omologo, “Fobust f0 estimation basedon a multi-microphone periodicity function for distant-talking speech,” in Proc. European Signal ProcessingConf., 2006.[10] M. Kepesi, F. Pernkopf, and M. Wohlmayr, “Jointposition-pitch tracking for 2-channel audio,” inProc. Int. Workshop Content-Based Multimedia Index-ing, 2007, pp. 303–306.[11] J. R. Jensen, M. G. Christensen, and S. H. Jensen, “JointDOA and Fundamental Frequency Estimation Methodsbased on 2-D Filtering,” in Proc. European Signal Pro-cessing Conference, 2010.[12] G. L. Bretthorst, “An introduction to parameter esti-mation using Bayesian probability theory,” in Max. En-tropy and Bayesian Methods, P. Fougere, Ed., pp. 53–79.1990.[13] J. B. Allen and D. A. Berkley, “Image method for ef-ficiently simulating small-room acoustics,” J. Acoust.Soc. Am., vol. 65(4), pp. 943–950, 1979.[14] A. de Cheveigné and H. Kawahara, “YIN, a fundamen-tal frequency estimator for speech and music,” J. Acoust.Soc. Am., vol. 111(4), pp. 1917–1930, Apr. 2002.

Multi-Channel Maximum Likelihood Pitch Estimation

https://vbn.aau.dk/ws/files/74625881/paper.pdf

Multi-Channel Maximum Likelihood Pitch Estimation

Abstract

Similar works

Full text

Available Versions

CiteSeerX

VBN