In this paper, we propose a method to integrate the results of different cover song identification algorithms into one single measure which, on the average, gives better results than initial algorithms. The fusion of the different distance measures is made by projecting all the measures in a multi-dimensional space, where the dimensionality of this space is the number of the considered distances. In our experiments, we test two distance measures, namely the Dynamic Time Warping and the Qmax measure when applied in different combinations to two features, namely a Salience feature and a Harmonic Pitch Class Profile (HPCP). While the HPCP is meant to extract purely harmonic descriptions, in fact, the Salience allows to better discern melodic differences. It is shown that the combination of two or more distance measure improves the overall performance

Alessio Degani

Marco Dalai

Pierangelo Migliorati

Riccardo Leonardi

DEGANI, Alessio

DALAI, Marco

LEONARDI, Riccardo

MIGLIORATI, Pierangelo

Archivio istituzionale della ricerca - Università di Brescia

A HEURISTIC FOR DISTANCE FUSION IN COVER SONG IDENTIFICATIONAlessio Degani, Marco Dalai, Riccardo Leonardi and Pierangelo MiglioratiUniversity of BresciaDII, Signals and Communication LabVia Branze, 38, 25123 Brescia, ITALYABSTRACTIn this paper, we propose a method to integrate the results ofdifferent cover song identification algorithms into one singlemeasure which, on the average, gives better results than ini-tial algorithms. The fusion of the different distance measuresis made by projecting all the measures in a multi-dimensionalspace, where the dimensionality of this space is the numberof the considered distances. In our experiments, we test twodistance measures, namely the Dynamic Time Warping andthe Qmax measure when applied in different combinations totwo features, namely a Salience feature and a Harmonic PitchClass Profile (HPCP). While the HPCP is meant to extractpurely harmonic descriptions, in fact, the Salience allows tobetter discern melodic differences. It is shown that the combi-nation of two or more distance measure improves the overallperformance.Keywords: cover song identification, distance fusion1. INTRODUCTIONCover song identification aims at finding different versions ofthe same musical piece within a large database of songs. Inthe last 10 years, a lot of work has been made to try to success-fully accomplish this task. Thanks to the MIREX evaluationcampaign for Music Information Retrieval (MIR) algorithms,this research topic has gained attention and methods have im-proved in accuracy. Different algorithms have been developedin the literature and the standard approach to measuring sim-ilarity between cover songs is to exploit music facets sharedbetween them. This similarity measure is computed using dif-ferent descriptors (or features) extracted from the raw audiofile. In order for these descriptors to be effective, they haveto be relatively insensitive to the majority of musical changesamong covers, like tempo or key. Once the descriptors areextracted, a measure of distance between song descriptionsis evaluated and a similarity score between songs is thus ob-tained. Hence, a cover song identification algorithm usuallytakes a query song as an input and, after a processing stepfor the extraction of the descriptors, performs a comparisonbetween this song and all songs in a database, using the ex-tracted feature. The result of this run is a ordered list of songsranked with a distance criteria, where the most similar songmust ideally rank first in this list.Of course, different features and different distances oversuch features can be used. The first largely used descriptor,the so called Pitch Class Profile (PCP), was introduced in1999 by Fujishima [1]. Over the years, PCP (sometimes alsocalled chromagram) was extended and modified in differentvariants, some of which are still successfully used in coversong identification (see HPCP [2]).Among the possible techniques to compute distances be-tween sequences of features, we can list two which are ofparticular importance for the cover song identification prob-lem. One is the Dynamic Time Warping (DTW [3, Chapter4]), that aims to find an optimal alignment path between twogiven time-dependent sequences. It gives us a warping costvalue between a song u and a song v. Another technique isintroduced in [4], which uses the Cross-Recurrence Plot [5]and recurrence quantification analyses like Qmax [6].Different performances are obtained when using a spe-cific feature with a specific distance measure, and it is not al-ways easy to understand which feature-distance combinationbehaves better, since this obviously depends on the datasetat hand, on the query etc. Moreover, even a single feature,when extracted with different settings, can give different per-formance. Based on this fact, for example, the system Hydra[7] combines features and distances extracted with differentparameters which are fed to a Support Vector Machine whichoutput, for each pair of songs, a single bit decision of the typecover/non-cover. A similar approach is used in [8], wherea distance is calculated over three different audio descriptorand a classifier is trained with a subset of known cover ornon-cover songs pairs. In our work instead, we do not applyany classification and no training is needed.In this paper we discuss an approach to cover song identi-fication that involves a blind combination of different featuresand different distance measures without making any assump-tion on the audio descriptor used. For the evaluation, we con-sider two type of features, the Salience function and the HPCPcombined with two distance measure, namely Dynamic TimeWarp and a Qmax. Each feature, when used with a givendistance, allows sorting the songs in the database in order ofdecreasing similarity with a given query song. In this paper,Features ExtractionDistances CalculationandDistances FusionFinal Dist. ScoreforNDistanceQuery songSongs DBFig. 1. General scheme of the methodwe propose a technique to combine the lists obtained by Nfeature-distance combinations into one single N -dimensionalspace in order to assess a “globally informed” distance mea-sure. As we will see in Section 4, the so obtained combinedlist leads to a relative improvement in accuracy with respectto the results obtained by the single measures separately. Inthe Figure 1 we can see the general outline of the algorithm.In the Section 2 we explain what features and what dis-tances are involved in our test and in the Section 3 we showhow we combine them. In Section 4 and 5 we present anddiscuss the accuracy results.2. AUDIO FEATURES AND DISTANCE METRICS2.1. Audio FeaturesHere we give a brief introduction of the audio descriptors usedin our test. The described features are computed for eachframe for a total of Nf frames. This leads to a huge amountof data which would make the complexity of any distancemeasure evaluation impractical. Hence, a temporal down-sampling is applied to the sequence of features to obtain ashorter sequence of length Nt < Nf . In our case, we use anadaptive decimation factor that is dependent on the beat dura-tion in order to obtain a beat-synchronous time average. Thispart is based on the algorithm presented in [9]. Where not ex-plicitly mentioned all of the feature and distance calculationalgorithms are re-implemented by the authors of this paper.2.1.1. Pitch Salience FunctionAs presented in [10], a salience function for a given frequencyfi is calculated as a weighted sum of the energy at the first 8harmonics of the fundamental frequency fi like fi, 2fi, 3fi, . . . . The pitch salience function is calculated at each frameusing the amplitude spectrum and covers a frequency rangingfrom 55 Hz to 1.76 kHz (5 octaves range from A1 to A6)using a resolution of resolution of 1 bin/semitone.2.1.2. HPCPBriefly, the computation of the HPCP descriptor begins withsome pre-processing step like spectral peak detection [11].Next, the energy of each spectral peak from about 50 Hzto 5 kHz is represented as a vector of Pitch Classes energy.Each Pitch Class represents a semitone in a twelve-tone equal-tempered chromatic scale. The energy of each Pitch Classare calculated from the correspondent spectral peak and theweighted summation of its harmonic frequencies peak energyup to 8 terms. The reader is referred to [2] for a completedescription.2.2. Distance MeasuresIn this section we give a brief description of the distance mea-sures used for the evaluation of our method.2.2.1. CRP/QmaxBasically, theQmax distance calculates the length of the longesttime segment in which two song u and v exhibit similar fea-ture patterns. This is done by using a cross-recurrence plot.A cross-recurrence plot (CRP) is a binary similarity matrix Cwhose elements ci,j are set to 1 when there is a recurrencebetween the i-th feature vector of song u and the j-th fea-ture vector of song v, and zero otherwise. Here, a recurrencemeans that the euclidean distance between this two vectorsis below a specified threshold. For more details such as thethreshold value and the CRP algorithm see [5, 4]. When con-secutive feature vectors are similar for a certain amount offrames, a diagonal patterns of ones become visible in CRP.What the Qmax algorithm does is to quantify the presenceand the length of this diagonal patterns in the CRP using anefficient recurrence quantification analysis [4]. In a nutshell,a cumulative matrix Q is computed over the elements of Cstarting form the element c1,1 and counting the elements withvalue equal to 1 that are aligned in a diagonal way. Finally,the Qmax value is calculated as the maximum amplitude ofthe elements Qi,j of the matrix Q asQmax = max (Qi,j) . (1)The Qmax measure gives a similarity quantification oftwo input songs. In our case we need a distance measure thatcan be calculated asdu,v =√NvtQmax, (2)where Nvt is the length of the salience function of song v andplays the role of a normalization factor [4].2.2.2. Dynamic Time WarpingDynamic Time Warping [3, Chapter 4] is a technique to findan optimal path to align two time sequences. Ideally, the twosequences are warped in a non-linear way to reach the maxi-mum matching between each other. DTW gives itself a mea-sure of distance between two sequences, and it thus be used toassess similarity between two songs [12, 13]. With DTW, we0.4 0.5 0.6 0.7 0.8 0.9 10.20.30.40.50.60.70.80.91d1q,sd2q,sFig. 2. Cloud for a given q = q0, d1 calculated using DTW onSalience feature and d2 with Qmax over HPCP. The triangleidentifies the correct cover song. Note that in this case thecorrect cover does not minimize d2.obtain the total alignment costDTWu,v between two featuressequences u and v. For more details, see [3, Chapter 4]. Weused the DTW implementation freely available at [14], withonly one minor modification, namely that the a normalizationsimilar to that of (2) is applied to obtain du,v as followsdu,v =DTWu,v√Nvt. (3)In our tests this normalization leads to a performance im-provement.3. DISTANCE SELECTIONIn this Section we describe the proposed technique for themerging of two or more feature and distance measure com-binations in order to create a single ranking with improvedperformance. Independently from the used features and met-rics, we assume that N different distance metrics dq,s =[d1q,s, · · · , dNq,s] are computed between the query song q andeach song s ∈ [1 . . . S] in the database. In a nutshell, the pro-posed method mixes N distances by projecting them in a N -dimensional space in order to refine the ranked list in a morereliable way. The process is now described in detail. Assum-ing the cover song identification algorithm returns a squareS × S cross-distances matrix D where each element dq,s ofthis matrix represents a distance between the song q and thesong s in the database, and we can calculate more than onedistance matrix using different combination of features andmetrics, we obtain N distances matrix D1, · · · ,DN . For afixed query song q = q0 and a fixed distance metric n = n0,we make a normalization of the distance vector asd¯n0q0,s =dn0q0,smaxs∈[1...S][dn0q0,s],∀s ∈ [1 . . . S] (4)in order to ensure that d¯n0q0,s ∈ [0, 1]. Now, for a fixed queryq = q0 and a fixed song s = s0 in the database, we definea point in a N -dimensional space that uniquely identifies theposition of the pair (q0, s0) in the distances spaced¯q0,s0 = [d1q0,s0 , · · · , dNq0,s0 ] ∈ [0, 1]N . (5)At this point we are able to compute d¯q,s ∈ [0, 1]N foreach q, s ∈ [1 . . . S]. The points d¯q,s form an N -dimensionalcloud. An example of one such cloud with N = 2 for a givenquery q is shown in Fig. 2. We now compute a new S × Srefined distance matrixR whose elements rq,s are defined asrq,s = ||1|| − ||d¯q,s − 1||, (6)where || · || is the l2 norm and 1 = [1, · · · , 1] is the N -dimensional “one” vector. Since the vector 1 represent thepoint in our space where all the distances are maximum, theterms ||d¯q,s − 1|| in (6) expresses how far the pair (q, s) isfrom to be a non-cover pair. It follows that (6) can be seen asa measure of how likely the pair is a cover pair. The origin ofthe N -dimensional space is the ideal place where a cover pair(q, s) should be placed, so intuitively, one may think that eachelement of R can be calculated as rq,s = ||d¯q,s||. Thoughthis approach does work in practice, however, in our tests thisstrategy leads to worse results compared to (6).4. RESULTSThe evaluation task is performed using the well known coversong dataset named covers80 [15]. We perform a comparisonof the performances between the basic algorithms that use onedistance metric over a single feature type, and a number ofcombinations of choices of features and distance metrics. Theused performance indicators are some of the commonly usedindicators in Music Information Retrieval: Precision, MeanRank of First Correctly Identified Cover (MR1st) and MeanAverage Precision (MAP). The total virtual score T is calcu-lated by counting the total unique correct identified cover forall of the method in the combination and normalizing by S.As we stated in Section 2, for our evaluation we use two fea-tures type: the HPCP (H) and the Salience function (S). Forthe distance metrics we use the Qmax measure (Q) and thedynamic Time Warping (D). Table 1 reports the accuracy in-dicator for different combination of feature/distance. For theHPCP with Qmax approach, the accuracy indicators may bedifferent from the original implementation of the algorithmby J. Serra` [4] since it has been completely rewritten by theauthors. As we can see in Table 1, all the accuracy results withN > 1 bring an improvement with respect to the simple dis-tance measure. Although the best results are obtained usingall (N = 4) of the possible combination of basic distances, wecan see that using (S,D)+(H,Q) we obtain a comparable resultbut with a lower dimensionality N = 2. Furthermore, we cansee that the indicator T is proportional to the MAP. Higher Tmeans higher MAP and consequently better precision.Combination N Prec. MR1st MAP T(S,D) 1 0.47 12 0.55 -(S,Q) 1 0.52 15 0.60 -(H,D) 1 0.35 17 0.43 -(H,Q) 1 0.59 9 0.65 -(S,D)+(S,Q) 2 0.61 12 0.66 0.61(S,D)+(H,D) 2 0.46 12 0.53 0.55(S,D)+(H,Q) 2 0.65 7 0.71 0.66(S,Q)+(H,D) 2 0.56 13 0.63 0.60(S,Q)+(H,Q) 2 0.64 10 0.69 0.69(H,D)+(H,Q) 2 0.60 9 0.66 0.60(S,D)+(S,Q)+(H,D) 3 0.60 11 0.66 0.66(S,D)+(S,Q)+(H,Q) 3 0.65 9 0.71 0.74(S,Q)+(H,D)+(H,Q) 3 0.66 9 0.70 0.70(S,D)+(H,D)+(H,Q) 3 0.60 7 0.67 0.68(ALL) 4 0.66 8 0.72 0.75Table 1. Accuracy results.5. CONCLUSIONSIn this paper we presented a method to merge N combinationof feature and distance measures to increase the accuracy re-sults of a cover song identification algorithm. This method isbased uniquely on a geometric N -dimensional distance mea-sure that has a very low computational cost of the overall dis-tance refinement. A particularly useful combination has beenobtained by using a Salience Feature with a Dynamic TimeWarp similarity measure and a HPCP with a Qmax measure.This combination, in our tests, proved to give excellent per-formance with a low dimensionality N . The percentage T ofthe virtual total unique correct identified cover plays a fun-damental role for the accuracy performances of the distancefusion process. The most important property of the method,however, is that it can be used to combine any set of differentdistance metrics, regardless of what they measure and with-out making any assumption on the specific audio features in-volved.6. REFERENCES[1] Takuya Fujishima, “Realtime chord recognition of mu-sical sound: a system using common lisp music,” Proc.of the Int. Computer Music Conference (ICMC), pp.464–467, 1999.[2] Emilia Go´mez, Tonal Description of Music AudioSignals, Ph.D. thesis, Universitat Pompeu Fabra,Barcelona, Spain, 2006.[3] Meinard Mu¨ller, Information Retrieval for Music andMotion, Springer, 2007.[4] Joan Serra`, Identification of Versions of the Same Mu-sical Composition by Processing Audio Descriptions,Ph.D. thesis, Universitat Pompeu Fabra, Barcelona,Spain, 2011.[5] Norbert Marwan, M. Carmen Romano, Marco Thiel,and Ju¨rgen Kurths, “Recurrence plots for the analysisof complex systems,” Physics Reports, vol. 438, no. 5,pp. 237–329, 2007.[6] Joan Serra`, Xavier Serra, and Ralph G. Andrzejak,“Cross recurrence quantification for cover song identi-fication,” New Journal of Physics, vol. 11, 2011.[7] Suman Ravuri and Daniel P.W. Ellis, “The hydra systemof unstructured cover song detection,” MIREX 2009 ex-tended abstract, 2009.[8] Justin Salamon, Joan Serra`, and Emilia Go´mez,“Melody, bass line, and harmony representations formusic version identification,” Proceedings of the 21stinternational conference companion on World Wide Web(WWW 2012), 2012.[9] Daniel P. W. Ellis, “Beat tracking by dynamic program-ming,” Journal of New Music Research, vol. 36, no. 1,pp. 51–60, 2007.[10] Justin Salamon, Emilia Go´mez, and Jordi Bonada, “Si-nusoid extraction and salience function design for pre-dominant melody estimation,” Proc. of the 14th Int.Conference on Digital Audio Effects (DAFx-11), 2011.[11] Xavier Amatriain, Jordi Bonada, Alex Loscos, andXavier Serra, in Udo Zo¨lzer DAFX-Digital Audio Ef-fects, chapter Spectral processing, pp. 373–438, JohnWiley & Sons, 2002.[12] Joan Serra`, Emilia Go´mez, Perfecto Herrera, and XavierSerra, “Chroma binary similarity and local alignmentapplied to cover song identification,” IEEE Trans. onAudio, Speech and Language Processing, vol. 16, no. 6,pp. 1138–1152, 2008.[13] Wei-Ho Tsai, Hung-Ming Yu, and Hsin-Min Wang,“Using the similarity of main melodies to identify coverversions of popular songs for music document retrieval,”Journal of Information Science and Engineering, vol.24, pp. 1669–1687, 2008.[14] Daniel P. W. Ellis, “Dynamic time warp (dtw) in mat-lab,” URL: http://labrosa.ee.columbia.edu/matlab/dtw/,2008.[15] Daniel P. W. Ellis, “The ’covers80’ cover songdata set,” URL: http://labrosa.ee.columbia.edu/projects/coversongs/covers80/, 2007.

A Heuristic for Distance Fusion in Cover Song Identification

Crossref

A heuristic for distance fusion in cover song identification

https://iris.unibs.it/bitstream/11379/346107/1/DDLM_WIAMIS-2013_camera_ready.pdf

A Heuristic for Distance Fusion in Cover Song Identification

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Brescia

Crossref