ANTONOPOULOS, I

Cornelis, Olmo

Leman, Marc

Moelants, Dirk

PIKRAKIS, A

THEODORIDIS, S

Ghent University Academic Bibliography

MUSIC RETRIEVAL BY RHYTHMIC SIMILARITY APPLIED ONGREEK AND AFRICAN TRADITIONAL MUSICIasonas Antonopoulos, Aggelos Pikrakisand Sergios TheodoridisDept. of Informatics & TelecommunicationsUniversity Of Athens, GreeceOlmo Cornelis, Dirk Moelantsand Marc LemanIPEM - Dept. of MusicologyGhent University, BelgiumABSTRACTThis paper presents a method for retrieving music record-ings by means of rhythmic similarity in the context of tra-ditional Greek and African music. To this end, Self Sim-ilarity Analysis is applied either on the whole recordingor on instances of a music thumbnail that can be extractedfrom the recording with an optional thumbnailing scheme.This type of analysis permits the extraction of a rhythmicsignature per music recording. Similarity between signa-tures is measured with a standard Dynamic Time Warpingtechnique. The proposed method was evaluated on cor-pora of Greek and African traditional music where humanimprovisation plays a key role and music recordings ex-hibit a variety of music meters, tempi and instrumentation.1 INTRODUCTIONIn the context of Music Information Retrieval (MIR), find-ing music recordings with similar rhythmic characteristicsis a highly desired task both for the untrained listener andthe musicologist.Over the years, several methods have been proposedin the context of Western music for retrieving music withsimilar rhythmic characteristics, e.g. tempo, meter andrhythmic patterns. Here a short overview of relevant pa-pers is given: The work in [1] measures the similaritybetween rhythmic patterns extracted from music record-ings and artificially generated percussive sounds. The ap-proach in [2] extracts temporal patterns from the energyenvelop of the signal in an attempt to classify music re-cordings to predefined classes. In [3] a set of classifica-tion schemes are proposed that are based on extractingrhythmic patterns from the signal’s spectrum. The methodproposed in [4] focuses on ballroom dances and is basedon features stemming from the histogram of Inter-OnsetIntervals. Finally, the work in [5] evolves around self sim-ilarity analysis of the music recording. In some of theabove methods, the term “rhythmic signature” is used toas a means to encode fundamental rhythmic characteris-tics of the music recordings.This paper focuses on rhythmic similarity in non West-ern music, i.e., Traditional Greek and African music, whi-c© 2007 Austrian Computer Society (OCG).ch have so far received little attention in the field of MIR.Such traditions impose a number of research challenges,mainly due to the complexity of the music meters, the sys-tem of music intervals and the highly improvisational at-titude of the music performers. The latter gives an addi-tional research challenge as serves the preservation of cul-tural heritage and also highlights the importance of MIRsystems to apply to corpora that fall outside the traditionalWestern schemes. In an attempt to measure rhythmic sim-ilarity in such music corpora, this paper exploits the repet-itive nature of the music recordings by means of Self Sim-ilarity Analysis. This type of analysis reveals periodicitiesthat are inherent in the music signal. Such periodicities areexpressed as a sequence of values to which we also referby the term rhythmic signature. To this end, we inves-tigate the possibility of applying an optional thumbnail-ing scheme as a preprocessing step to extracting rhythmicsignatures. Similarity measurement between signatures isperformed by means of a standard Dynamic Time Warpingtechnique.Section 2 presents the proposed audio thumbnailingscheme and Section 3 describes how rhythmic signaturesare extracted from the music signal. The proposed similar-ity measure is presented in Section 4. Results and imple-mentation details are given in Section 5 and conclusionsare drawn in Section 6.2 THUMBNAILING SCHEMEThe proposed audio thumbnailing scheme is optional andis considered to be a variation of the method proposed in[6], in the sense that a different feature extraction schemeis used in this paper.2.1 Feature extractionAt a first step, the music recording is short-term processedby means of a moving window technique. The short-term frames are chosen to be ≃ 186 msecs long, non-overlapping and are multiplied by a Hamming window.Each frame is given as input to a mel-scale filter bank [8]that consists of overlapping triangular filters. The centerfrequencies of the filters coincide with the frequencies ofwhole tones on a chromatic scale, starting from F0 = 110Hz and moving up to ≃ 6.3 KHz, resulting into 36 filterswhich cover approximately six octaves. In the sequel wewill refer to this type of MFCCs as chroma-based MFCCsdue to the similarities it bears with the “chroma vector”[6]. Further details are given in [7].To proceed, let c(n) be the 36 × 1 vector of chroma-based MFCCs from the n-th frame. The sequence of vec-tors can be written in matrix notation asC36×N = [c(1) c(2) . . . c(N)],where N is the number of short-term frames.At a next step, Singular Value Decomposition (SVD)is applied on the transpose, CT , of C, i.e., CT = UΣV,where UN×36 and V36×36 are the projection matrices andΣ36×36 is the matrix of singular values. The first six rowsof the transpose, UT , of U, are finally selected as the fea-ture sequence.2.2 Thumbnail selectionThe SSM is generated from the first six rows of UT usingthe Euclidean Distance function as metric [5]. By its defi-nition the SSM is symmetric around the main diagonal andit therefore suffices to focus on its lower triangle. At a firststep, the SSM is correlated with a rectangular window, w(size D ×D). The window has 1’s on the main diagonaland zeros elsewhere. If (i, j) are the position indices ofan element of SSM, the upper left corner of w is chosento coincide with (i, j). The correlation result, S(i, j), forSSM(i, j) is therefore computed as follows:S(i, j) =D−1∑d1=0D−1∑d2=0SSM(i + d1, j + d2)w(d1, d2)=D−1∑d=0SSM(i + d, j + d) (1)At a second step, let S(k,m) be the lowest value of S.S(k,m) resides on the diagonal with index k−m and el-ements {S(k,m), S(k+1,m+1), . . . , S(k+D−1,m+D − 1)} form a segment on the diagonal that defines thedesired thumbnail. The two corresponding feature subse-quences, i.e., two instances of the thumbnail, are{UTk , UTk+1, . . . UTk+D−1}and{UTm, UTm+1, . . . , UTm+D−1},respectively. Parameter D controls the size of the thumb-nail and is user defined, depending on the corpus understudy (see Section 5).It has to be noted that the proposed thumbnailing sche-me is optional and depends on the dataset under study. Ifit is skipped, the rhythmic signatures (see next section)will be extracted by taking into account the whole audiorecording. This is desirable if one is unsure whether thetwo instances of the extracted thumbnail are indeed repre-sentative of the complete music recording. In the sequel,the term music signal will refer to either the two instancesof the selected thumbnail or the complete music record-ing.3 EXTRACTING RHYTHMIC SIGNATURES3.1 Feature extractionAt a first step, the music signal (i.e., the two thumbnail in-stances or the complete recording) is short-term processedto extract a sequence of chroma-based MFCCs, as in Sec-tion 2.1. However, this time, shorter, overlapping win-dows are used (window length is ≃ 93 msecs and windowstep is 11.6 msecs). Following the notation that was intro-duced in Section 2.1, let C = [c(1) c(2) . . . c(N)], bethe new sequence of MFCCs.At a first step, C is long-term segmented with a mov-ing long-term window (window length is 4 secs and stepis 1 sec). To simplify notation, let Ct = [ct(1) ct(2) . . .ct(M)], be the subsequence that corresponds to the t-thlong-term window, where M is the window length mea-sured in number of frames. The SSM is then calculatedfor each long-term window, using the Euclidean Distancemetric. For the t-th long-term window, the mean value,Rt(k), of each diagonal in the lower SSM triangle is com-puted, i.e., Rt(k) = 1M−k∑Ml=k∥∥ct(l), ct(l − k)∥∥, wherek is the diagonal index and ‖.‖ is the Euclidean distancefunction. Each Rt is treated as a signal. At a next step, themean signal, Rµ, of all Rt’s is computed, i.e.,Rµ(k) =1TT∑t=1Rt(k),where T is the number of long-term windows. Rµ is thennormalized to unity, i.e., Rµ(k) = Rµ(k)max(Rµ) .As can be seen in Figure 1, Rµ exhibits a number ofvalleys (local minima). Each valley corresponds to a pe-riodicity that is inherent in the music signal. Such peri-odicities are related to the rhythmic characteristics of therecording, e.g., music meter and tempo [7]. In what fol-lows, we will refer to Rµ as the rhythmic signature of themusic recording. The main idea behind this approach, isthat, recordings with similar rhythmic characteristics areexpected to yield “similar” signatures (as can be seen inthe upper part of Figure 1). On the contrary, differentrhythmic characteristics will result into “dissimilar” sig-natures (bottom part of Figure 1). Therefore, the nextchallenge is to devise a similarity measure for signatures.4 SIMILARITY MEASURE FOR SIGNATURESIf L is the number of music recordings in a corpus, Lrhythmic signatures are first extracted and stored as meta-data. In order to measure similarity between signatures, astandard Dynamic Time Warping cost has been employed.As is the case with DTW techniques [8], a set of localpath constraints needs to be first defined. In our study weexperimented with two types of constraints, i.e., Sakoe-Chiba and Itakura and adopted the former.If a rhythmic signature is drawn from the corpus, itsmatching cost against the remaining L-1 signatures is cal-culated using the adopted DTW technique. This proce-dure yields L−1 cost values which are sorted in ascending0 0.5 1 1.5 2 2.5 3 3.5 400.20.40.60.81Time (sec)Normalized Mean Values Of SSM Diagonals For Chroma Based MFCCs0 0.5 1 1.5 2 2.5 3 3.5 400.20.40.60.81Normalized Mean Values Of SSM Diagonals For Chroma Based MFCCsTime (sec)song1: 7/8 meter, 268 bpm  song3: 2/4 meter, 91 bpmsong1 :7/8 meter, 268 bpm  song1 :7/8 meter, 260 bpmFigure 1. Top: Signatures from two recordings of musicmeter 78 . Bottom: Signatures from a recording of meter78and a recording of meter 24 .order, with the lowest values indicating highest similar-ity. The next section focuses on evaluating this matchingscheme on two corpora of traditional Greek and Africanmusic.5 EXPERIMENTS AND RESULTS5.1 Corpus of Greek Traditional Dance musicThe first corpus of our study consists of 220 tracks ofGreek Traditional Dance Music, which are drawn fromvarious Greek regions. The tracks were manually cate-gorised into four genres as shown in Table 1. These gen-res exhibit certain variety in terms of instrumentation andrhythm. From the corpus description it can be noticed thatthe longest music meter duration (approximately 2 secs)appears in class 2, for tempo ∼= 90 bpm and music meter34 . By using a thumbnail which is 10 secs long, the longestmusic meter is repeated up to 5 times in each long-termsegment. Our study has revealed that, this expected num-ber of repetitions is sufficient for the extraction of reliablerhythmic signatures and justifies our choice for the lengthof thumbnails. Similarly, the longest meter duration af-fects the length of the long-term window in Section 3.1.By the definition of self similarity analysis, a periodicityof k lags will manifest itself as a valley of Rµ, if the long-term window is at least 2k long. Therefore the length ofthe long-term window was chosen equal to 4secs to cap-ture the periodicity of the longest music meter. Finally,the range of lags of Rµ on which DTW is employed startswith the lag corresponding to the fastest tempo value andreaches up to the lag that corresponds to the longest meter-tempo pair.Table 2 presents the confusion matrix for the Greekcorpus, where the leave-one-out method was applied onthe complete corpus. Table 2 reveals that, when only thebest result (lowest matching cost) was examined, limitedconfusion occured between the classes 3 and 4 and classes1 and 2. Further experimentation revealed that, when thetwo lowest matching costs were taken into account, theconfusion matrix remained the same within statistical con-fidence.class id # of songs meter tempo range (bpm)1 53 2/4 91-952 63 3/4 93-1053 62 7/8 250-2804 42 2/4 150-180Table 1. Description of Greek Traditional Dance corpus.Precision % Class 1 Class 2 Class 3 Class 4Class 1 94.3 3.2 1.7 0Class 2 3.8 96.8 0 0Class 3 1.9 0 96.6 10.9Class 4 0 0 1.7 89.1Recall % Class 1 Class 2 Class 3 Class 4Class 1 94.3 3.8 1.9 0Class 2 3.2 96.8 0 0Class 3 1.6 0 90.3 8.1Class 4 0 0 2.4 97.6Table 2. Precision and recall for Greek Traditional corpus.5.2 Corpus of African musicA collection of 103 pieces was selected from the musicarchives of the Royal Museum of Central-Africa in Ter-vuren (Brussels). This institute has one of the most im-portant collections of African music in the world. 1 Thecurrent selection contains field recordings of Congo andRwanda recorded during the second half of the 20th cen-tury [9]. Similarly to the Greek music corpus, the rhyth-mic structure is highly repetitive and contains a wide rangeof rhythmic structures, including irregular meters that areseldom found in Western music [10].class id # of songs meter1 27 3/42 26 4/43 24 5/44 26 6/4Table 3. Description of the 1st set of the African corpus.The corpus has been manually annotated with a (percei-ved) ground truth which has been used to evaluate thecomputer analysis. Two types of classification were made:one according to the meter, the other focusing on a selec-tion of characteristic repetitive rhythmic patterns. The firstclassification, Table 3, uses four metric classes 34 ,44 ,54 and1 http://music.africamuseum.bePrecision %Class id 1 2 3 41 68.8 4.3 20 02 12.5 82.6 12 3.73 15.6 13 64 3.74 3.1 0 4 92.6Recall %Class id 1 2 3 41 78.6 3.6 17.9 02 14.8 70.4 11.1 3.73 20 12 64 44 3.7 0 3.7 92.6Table 4. Precision and recall (1st set of African corpus).64 . Table 5, is restricted to 44 pieces which can be classi-fied as variants of 5 prototypical patterns: short-long-long(quintuple); short-long-long-short-long-long-long (sextu-ple); long-short-short-long (triple1); short-long (triple2);and short-short-long (duple).class id # of songs patternquintuple 10sextuple 14triple1 8triple2 5duple 7Table 5. Description of the 2nd set of African corpus.The thumbnail that was selected for each piece by theproposed method, often tended to contain parts of the songwhere the most percussive events occurred. Since thiscould lead to a dense regular structure that can easily beconfused with patterns in which each beat is articulatedthe rhythmic signatures, Rµ’s were extracted from wholeaudio recordings and the thumbnail scheme was skipped.Tables 4, 6 reveal that the results of both sets of Africanmusic are promising. When manually checking the prob-lematic cases, mistakes can mostly be related to the occur-rence of variants of the main pattern within one piece. Thepossible variations are mostly the addition of a percussiveevent where a rest used to be, or the opposite: omission ofa percussive event. The meter and beat stay however thesame.6 CONCLUSIONSThis paper presented a music retrieval method based onrhythmic similarity measurement. The method yieldedsatisfactory results on coprora of traditional Greek andAfrican music. In future work, more sophisticated DTWtechniques will be used on larger corpora and the possi-bility to extract multiple rhythmic signatures per musicrecording will be investigated.Precision %Class id 1 2 3 4 51 70.80 3.1 0 10 14.32 4.2 84.4 0 0 03 8.3 3.1 86.7 0 04 8.3 0 13.3 60 05 8.3 9.4 0 30 85.7Recall %Class id 1 2 3 4 51 85 5 0 5 52 3.6 96.4 0 0 03 12.5 6.3 81.3 0 04 20 0 20 60 05 14.3 21.4 0 21.4 42.9Table 6. Precision and recall (2nd set of African corpus).7 REFERENCES[1] J. Paulus and A. Klapuri, “Measuring the similarityof rhythmic patterns.”, Proceedings of ISMIR, Paris,France, September 2002.[2] S. Dixon, F. Gouyon and G. Widmer, “Towards char-acterisation of music via rhythmic patterns.”, Proceed-ings of ISMIR, Barcelona, Spain, 2004.[3] G. Peeters, “Rhythm Classification Using SpectralRhythm Patterns”, Proceedings of ISMIR 2005, Lon-don, September, 2005.[4] F. Gouyon and S. Dixon, “Dance music classifica-tion: a tempo-based approach.”, Proceedings of IS-MIR, Barcelona, Spain, 2004.[5] J. Foote, M. Cooper and U. Nam, “Audio Retrieval byRhythmic Similarity”, Proceedings of ISMIR, Paris,France, September 2002.[6] M. Bartsch and G.H. Wakefield, “Audio thumbnail-ing of popular music using chroma-based representa-tions”, IEEE Transactions on Multimedia, 7(1), 96-104, 2005.[7] A. Pikrakis, I. Antonopoulos and S. Theodoridis,“Music Meter and Tempo Tracking from raw polypho-nic audio”, Proceedings of ISMIR, Barcelona, Spain,2004.[8] S. Theodoridis and K. Koutroumbas, Pattern recogni-tion, Academic Press, 3d Edition, 2006.[9] O. Cornelis et al., “Digitisation of the ethnomusico-logical Sound Archive of the Royal Museum for Cen-tral Africa.”, IASA Journal, pp 35-43, 2005.[10] O. Cornelis et al., “Problems and Opportunities ofApplying Data- & Audio-Mining Techniques to Eth-nic Music”, Journal of Intangible Heritage, (in press),2007.

Music retrieval by rhythmic similarity applied on Greek and African traditional music

https://biblio.ugent.be/publication/423631/file/448560

Music retrieval by rhythmic similarity applied on Greek and African traditional music

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography