A case for reproduciblity in MIR : replication of 'a highly robust audio fingerprinting system'

This article makes a case for reproducibility in MIR research. Claims made in many MIR publications are hard to verify due to the fact that (i) often only a textual description is made available and code remains unpublished - leaving many implementation issues uncovered; (ii) copyrights on music limit the sharing datasets; and (iii) incentives to put effort into reproducible research -- publishing and documenting code and specifics on data -- is lacking. In this article the problems around reproducibility are illustrated by replicating a MIR work. The system and evaluation described in 'A Highly Robust Audio Fingerprinting System' is replicated as closely as possible. The replication is done with several goals in mind: to describe difficulties in replicating the work and subsequently reflect on guidelines around reproducible research. Added contributions are the verification of the reported work, a publicly available implementation and an evaluation method that is reproducible