The VocalNotes dataset is a collection of audio and annotations for excerpts of vocal performances from five musical traditions - Japanese Minyo, Chinese Hebei Bangzi opera, Russian traditional singing, Alpine yodel and Jewish Romaniote chant. For each tradition the dataset contains: about 10 minutes of audio; documentation for the songs from which annotated fragments originate; f0, independent onset, offset and note pitch annotations created by two or three experts; The dataset was created as part of the VocalNotes project [1]. It is released under CC-BY-NC-SA license and can be accessed by filling out a request form

Chiba, Gakuto

Crowdus, Miranda

Killick, Andrew

Li, Yukun

McBride, John

Nikolaenko, Yulia

Ozaki, Yuto

Phillips, Elizabeth

Proutskova, Polina

Savage, Patrick

Shuster, Lawrence

Velichkina, Olga

Wey, Yannick

Yu, Zhaoxin

Yue, Wei

Zuckerberg, Gabriel

English

Berner Fachhochschule: ARBOR

THE VocalNotes DATASETPolina Proutskova John McBride Yuto Ozaki Gakuto ChibaBBC/QMULproutskova@googlemail.comCentre for Soft and Living Matterjmmcbride@protonmail.comKeio Universityyozaki@sfc.keio.ac.jpKeio Universitygane1222@sfc.keio.ac.jpYukun Li Zhaoxin Yu Wei Yue Miranda CrowdusQueen Mary University of Londonyukun.li@qmul.ac.ukShandong College of Arts943469149@qq.comShandong College of Artsyuewei1981zgyy@126.cmConcordia Universitymiranda.crowdus@concordia.caGabriel Zuckerberg Olga Velichkina Yulia Nikolaenko Yannick WeyBrown Universitygabriel_zuckerberg@brown.eduFrench Society for Ethnomusicologyolga.velichkina@gmail.comindependentmitida173@gmail.comLucerne University of AppliedSciences and Artsyannick.wey@hslu.chLawrence Shuster Patrick E. Savage Elizabeth Phillips Andrew KillickCornell Universitylbs239@cornell.eduKeio Universitypsavage@sfc.keio.ac.jpMcMaster Universityphille10@mcmaster.caUniversity of Sheffielda.killick@sheffield.ac.ukABSTRACTThe VocalNotes dataset is a collection of audio andannotations for excerpts of vocal performances from fivemusical traditions - Japanese Minyo, Chinese HebeiBangzi opera, Russian traditional singing, Alpine yodeland Jewish Romaniote chant. For each tradition thedataset contains: about 10 minutes of audio;documentation for the songs from which annotatedfragments originate; f0, independent onset, offset andnote pitch annotations created by two or three experts;The dataset was created as part of the VocalNotes project[1]. It is released under CC-BY-NC-SA license and canbe accessed by filling out a request form.1. INTRODUCTIONA major challenge in MIR is the lack of annotated data,especially for singing. Recently, large datasets based oncrowd-sourced non-expert annotations, improved by deepneural networks, have been introduced (DALI [2],MIR-ST500 [3]), yet the resulting quality of theannotations is low and is difficult to assess. While thesedatasets comprise mainly Western popular music, newhigh-quality corpora of non-Western vocal traditions haveemerged more recently, including for Georgian [4],Korean [5] and Chinese songs [6]. Cross-cultural datasetshave been published outside MIR, e.g. by theSpeech/Song project [7]. Here we expand on this byreleasing a dataset of vocal performances from fivedifferent traditions – Japanese, Chinese, Russian, Alpineand Jewish – with pitch and note annotations by expertsfrom each tradition.Automated systems continue to fall short of humanexpertise in singing onset and note annotation [8].Figure 1. Fundamental frequency curve of a Russiantraditional song “Vy kumushki kumitesia” from Poozerieregion, played on a keyboard and sung by the traditionalperformer Olga SergeevaTypically in MIR research, one develops algorithms thataim to reproduce a ‘ground truth’. One of the pitfalls isthe assumption that there is one ‘correct’ way totranscribe music, which has been refuted byethnomusicologists [9]–[14]. Transcriptions can differdue to cultural knowledge, or different aims as to whatlevel of detail should be annotated. Limits of humanperception can lead to uncertainties, and transcribers mayhave different strategies to resolve uncertainties. Singingin particular is difficult to transcribe due to inherentlyunstable pitch curves (Fig. 1) and vocal drift [15]. Vocaltechniques such as vibrato, embellishments and glidescontribute to the analytical ambiguity of singing, forhumans and automated systems alike. Following [16] weprovide a larger dataset with the advantage ofindependent expert annotations for each excerpt,exploring the concept of a variable ground truth, whichwe hope will aid the development of more flexibleautomated transcription algorithms that can deal withambiguities.source: https://doi.org/10.24451/arbor.20797 | downloaded: 10.1.2024Figure 2. The VocalNotes project participants map.2. THE VOCALNOTES PROJECTThe VocalNotes project [1] investigated how experttraditional music listeners conceive of notes in vocalperformances by studying similarities and differences intheir transcriptions. Teams of experts from five musicaltraditions (Japanese Min’yo, Chinese Bangzi opera,Russian traditional village singing, Alpine yodelling, andRomaniote Jewish chanting) each transcribed 10 minutesof vocal recordings from their culture, wheretranscription consisted of segmentation and note pitchcorrection. The experts then systematically comparedtheir independent transcriptions, isolated and formalisedcontexts which led to disagreements.Each team was led by an ethnomusicologist andincluded two or three transcribers deeply familiar withthe tradition they were annotating. The instruction was toproduce independent analytic transcriptions based onexpert’s perception. Yet each team had their own researchquestion which shaped the choice of musical fragmentsand the transcription process:● The Japanese team analysed 9 recordings: 3different singers x 3 different folk songs, withone historical performance and two modernperformances by the co-authors for each song.● The Russian team put together a corpus ofethnographic solo and group singing recordings(with one channel per singer) from a variety oflocal traditions and genres (21 monophonictracks from 12 songs)● The Jewish team transcribed 10 excerpts fromthe few existing recordings of Romaniote Torahcantillation - a selection of the recitation of thesame biblical text chanted by differentRomaniote practitioners. They includespeech-like declamation alongside singing.● The Alpine team transcribed 9 solo excerptsfrom the central European yodelling tradition.● The Chinese research team transcribed 11excerpts from Hebei Bangzi, a genre of Chinesetraditional opera, covering a range of temposincluding slow, moderate, fast, and rubatopatterns.3. ANNOTATION METHODOLOGYTranscription was conducted in three phases: pitch curvecorrection, segmentation, note pitch correction. The firsttwo phases were conducted in Tony [17], with note pitchcorrection done in Sonic Visualizer [18].The f0 curve was automatically extracted using pYIN inTony, then manually corrected and agreed within theteam. Segmentation and note pitch correction wereperformed independently by each transcriber. Notesegmentation was done in Tony without relying on Tony’sautomatic segmentation suggestions. It was found that thenote pitch automatically assigned by Tony to be themedian of the pitch curve was not always perceived asaccurate. Therefore note segments were exported to SonicVisualiser and pitch was manually corrected wherenecessary. Sonic Visualiser turned out to be a much lessconvenient and responsive tool for note pitch correctionthan Tony was for note segmentation, therefore oursegmentation annotations, which were performed fromscratch, are of higher quality than the note pitchannotations, which were largely based on the automatedsuggestions.4. THE DATASETThe dataset comprises three components: the audio files,the song documentation files and the annotations. Audiofilenames are the song titles chosen by the team (usuallylyrics incipits) from which the analysed fragmentsoriginated. The documentation files from each teamcontain information about the songs, the performers,where and when the songs were recorded, what is theirsocial or musical function (e.g. “wedding song”, or“Genesis 1:1-11”), the rights holders of the completerecordings and where these recordings can be accessed.The annotations are in the form of .csv files, includingf0, note segments with uncorrected pitch (f0 median),note segments with manually corrected pitch. Thefilenames are constructed of the song title, transcriberabbreviation and data type (pitches, segments or notes):<Song_title__OV__notes.csv>The dataset is available under the CC BY-NC-SA 4.0license to allow non-commercial use and facilitatereplicability and further research. A version of record ofthe annotations is hosted on Zenodo1. The access to thefull dataset including audio can be requested by fillingout a short online form2. The answers will be reviewed bya team member and a link to download the dataset will beprovided. This is in recognition of the fact that the audiofragments in our dataset contain sensitive religious andritual material. We ask the users of the dataset to treat therecordings, their performers and their communities withrespect and to refrain from aggregating the dataset forunrestricted online access.The VocalNotes dataset welcomes new contributions ofaudio and annotations which follow the samemethodology [19]. All the annotations collected duringthe project, the guidelines, video tutorials and softwarescripts are available in the VocalNotes Open ScienceFramework repository3.3 https://osf.io/4n5ry/2 request form at https://forms.gle/W86j2koBwpkfmnBc91 https://doi.org/10.5281/zenodo.100659555. REFERENCES[1] Polina Proutskova et al., ‘VocalNotes –investigating perceptual differences insegmentation and pitch through transcriptions ofvocal performances in five musical traditions’,Anal. Approaches World Musics J., in preparation2024.[2] G. Meseguer-Brocal, A. Cohen-Hadria, and G.Peeters, ‘DALI: A large dataset of synchronizedaudio, lyrics and notes, automatically created usingteacher-student machine learning paradigm’, ArXivPrepr. ArXiv190610606, 2019.[3] J.-Y. Wang and J.-S. R. Jang, ‘On the Preparationand Validation of a Large-Scale Dataset of SingingTranscription’, in ICASSP 2021 - 2021 IEEEInternational Conference on Acoustics, Speech andSignal Processing (ICASSP), Toronto, ON,Canada: IEEE, Jun. 2021, pp. 276–280. doi:10.1109/ICASSP39728.2021.9414601.[4] S. Rosenzweig, F. Scherbaum, D. Shugliashvili, V.Arifi-Müller, and M. Müller, ‘ErkomaishviliDataset: A curated corpus of traditional Georgianvocal music for computational musicology’, Trans.Int. Soc. Music Inf. Retr., vol. 3, no. 1, 2020.[5] S. Choi, W. Kim, S. Park, S. Yong, and J. Nam,‘Children’s song dataset for singing voiceresearch’, in ISMIR Late-Breaking Demo, 2020.[6] R. Gong, R. C. Repetto, and X. Serra, ‘Creating ana cappella singing audio dataset for automaticjingju singing evaluation research’, presented at theProceedings of the 4th International Workshop onDigital Libraries for Musicology, 2017, pp. 37–40.[7] Y. Ozaki et al., ‘Similarities and differences in aglobal sample of song and speech recordings[Stage 1 Registered Report]’, 2022.[8] Y. Ozaki et al., ‘Agreement among human andautomated transcriptions of global songs’,PsyArXiv, preprint, Jul. 2021. doi:10.31234/osf.io/jsa4u.[9] G. List, ‘The Musical Significance of Transcription(Comments on Hood," Musical Significance")’,Ethnomusicology, vol. 7, no. 3, pp. 193–197, 1963.[10] A. Herzog, ‘Transcription and Transnotation inEthnomusicology’, J. Int. Folk Music Counc., vol.16, pp. 100–101, 1964.[11] N. M. England, ‘Symposium on Transcription andAnalysis: A Hukwe Song with Musical Bow’,Ethnomusicology, vol. 8, no. 3, pp. 223–277, 1964.[12] G. List, ‘The reliability of transcription’,Ethnomusicology, pp. 353–377, 1974.[13] Alekseev, E, Notnaya zapis’ narodnoi muzyki[Score notation of folk music]. Moscow: SovetskiyKompositor, 1990.[14] J. Stanyek, ‘Forum on transcription’,Twent.-Century Music, vol. 11, no. 1, pp. 101–161,2014.[15] M. Mauch, K. Frieler, and S. Dixon, ‘Intonation inunaccompanied singing: Accuracy, drift, and amodel of reference pitch memory’, J. Acoust. Soc.Am., vol. 136, no. 1, pp. 401–411, 2014.[16] R. M. Bittner, K. Pasalo, J. J. Bosch, G.Meseguer-Brocal, and D. Rubinstein, ‘vocadito: Adataset of solo vocals with $ f_0 $, note, and lyricannotations’, ArXiv Prepr. ArXiv211005580, 2021.[17] M. Mauch et al., ‘Computer-aided melody notetranscription using the Tony software: Accuracyand efficiency’, presented at the First InternationalConference on Technologies for Music Notationand Representation (TENOR 2015), 2015.[18] C. Cannam, C. Landone, and M. Sandler, ‘Sonicvisualiser: An open source application for viewing,analysing, and annotating music audio files’,presented at the Proceedings of the 18th ACMinternational conference on Multimedia, 2010, pp.1467–1468.[19] Polina Proutskova et al., ‘VocalNotesmethodology: framework, challenges and lessons’,Anal. Approaches World Musics J., in preparation2024.

The VocalNotes Dataset

https://arbor.bfh.ch/20797/1/354_paper.pdf

The VocalNotes Dataset

Abstract

Similar works

Full text

Available Versions

Berner Fachhochschule: ARBOR