5,403 research outputs found
The GTZAN dataset: Its contents, its faults, their effects on evaluation, and its future use
The GTZAN dataset appears in at least 100 published works, and is the
most-used public dataset for evaluation in machine listening research for music
genre recognition (MGR). Our recent work, however, shows GTZAN has several
faults (repetitions, mislabelings, and distortions), which challenge the
interpretability of any result derived using it. In this article, we disprove
the claims that all MGR systems are affected in the same ways by these faults,
and that the performances of MGR systems in GTZAN are still meaningfully
comparable since they all face the same faults. We identify and analyze the
contents of GTZAN, and provide a catalog of its faults. We review how GTZAN has
been used in MGR research, and find few indications that its faults have been
known and considered. Finally, we rigorously study the effects of its faults on
evaluating five different MGR systems. The lesson is not to banish GTZAN, but
to use it with consideration of its contents.Comment: 29 pages, 7 figures, 6 tables, 128 reference
Recommended from our members
The Effect of Keyboarding and Presentation Format on the Recall of Accent Marks in L2 Learners of French
Are there some implementations of technology in the language classroom which lead to measurable advantages over others? Sturm and Golato (in press) found considerable variance on dictée tests within groups of students who practiced a list of accent-bearing target words by handwriting, typing using preprogrammed function keys, or typing using ALT+ numeric codes. These results contradict the results of Gascogine-Lally (2000), who found that students who typed a paragraph recalled accents better than those who wrote the paragraph by hand. The present study seeks to explore the difference between the two studies. Participants were exposed to Gascoigne-Lally’s paragraph, as well as a set of words in both list and paragraph form. One-way ANOVAs revealed no significant differences between groups, although repeated-measures ANOVAs revealed differences within participants on different sets of target words on immediate posttests. The results of this study encourage future research to investigate the results obtained by Gascoigne-Lally as well as Sturm and Golato
- …