Arenas-García, Jerónimo

Hansen, Lars Kai

Lehn-Schiøler, Tue

Petersen, Kaare Brandt

Tue Lehn-Schiøler

Jerónimo Arenas-García

Kaare Brandt Petersen

Lars Kai Hansen

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY

A Genre Classification Plug-in for Data Collection.

This demonstration illustrates how the methods developed in the MIR community can be used to provide real-time feedback to music users. By creating a genre classifier plug-in for a popular media player we present users with relevant information as they play their songs. The plug-in can furthermore be used as a data collection platform. After informed consent from a selected set of users the plug-in will report on music consumption behavior back to a central server

Online Research Database In Technology

A Genre Classification Plug-in for Data CollectionTue Lehn-Schiøler, Jero´nimo Arenas-Garcı´a, Kaare Brandt Petersen, Lars Kai HansenThe Technical University of DenmarkInformatics and Mathematical ModellingRichard Petersens Plads Bld 321Kgs. Lyngby DK 2800{tls,jag,kbp,lkh}@imm.dtu.dkAbstractThis demonstration illustrates how the methods developedin the MIR community can be used to provide real-timefeedback to music users. By creating a genre classifier plug-in for a popular media player we present users with rele-vant information as they play their songs. The plug-in canfurthermore be used as a data collection platform. Afterinformed consent from a selected set of users the plug-inwill report on music consumption behavior back to a centralserver.Keywords: Genre classification, media player, plug-in, datacollection1. Media Player PluginsThe goal of this project is to create a data collection plug-in.To obtain users trust and collaboration the plug-in needs tobe seamlessly intergraded with the player, and furthermorethe user should find the plug-in attractive to use. The lattercan be achieved by offering added value. In this demonstra-tion we propose on-line genre classification and later ad-vanced music based search functionality as a reward. Theperspective is that our users prefer the player and the plug-inwe can collect information about their music related behav-iors. This way, a large collection of songs and usage datacan be collected; this information includes meta data, usageinformation and content based features. Once sufficient datahas been collected, the plug-in can be extended to serve asa search engine; along with the online genre classification,users can be provided with recommendation for new songsbased on the current song and their general listening habits.Many existing media players support external plug-ins,e.g., for programming of visualizations of the music. Theclassical visualizations for plug-ins are based on a short timeFourier transform of the signal. However, in some mediaplayers it is possible to get access to the original sound streamthus making it possible to exploit the content based tech-niques developed in the MIR community to create visual-Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page.c© 2006 University of VictoriaFigure 1. Screen-shot of the real-time genre classifier.izations. Such methods have been described among othersby [2, 4, 7, 8].2. Genre Prediction SystemIn this application we make use of aggregated features asdescribed in [5]. On a 20 ms scale with 10 ms overlap sevenMFCC’s are extracted and the first one discarded. Thesefeatures are collected during a frame of one second thuscreating a six dimensional time series with 100 samples.For each second the time series is modeled by a multivari-ate autoregressive (MAR) process with three lags: xk =∑3l=1Alxk−l+ek. The values of the threeA-matrixes, themean and the covariance of the residuals ek (3 · 62 + 6 +(62 + 6)/2 = 135) are finally concatenated in a vector zkrepresenting each second of the song.2.1. Predicting the genre from AR coefficientsThe machine learning component of the system used thegenre prediction task, is implemented with a simple linearregression model with a softmax activation function [3]:yˆk = softmax [Bzk + b] , (1)where yˆk is a C dimensional vector containing the predic-tions for the different genres (C = 15 in our system), andzk are the parameters of the MAR model characterizing onesecond of the song. The softmax activation allows to inter-pret yˆk as estimations of the a posteriori probabilities of allgenres given some input vector.To adapt the parameters in the model, namely B and b,we used a set of 10000 training songs whose genre labelswere known in advance. These data were used to producea set of training pairs {zk,yk}Nk=1, with yk being the truelabels associated to zk. Then, B and b were optimized bygradient descent of the cost functionE =N∑k=1‖yk − yˆk‖22. (2)To improve the performance of this simple classifier, weadded the following improvements:• Dimensionality reduction: Instead of using the orig-inal MAR coefficients, we only retain the most rele-vant projections, so that the regression model equa-tion (1) operates on z˜k = UT zk. The projection ma-trix U is also optimized to minimize equation (2) us-ing so-called Orthonormalized Partial Least Squares(OPLS) [1]. In the reported experiments, z˜k was 14-dimensional, the maximum number of projections thatOPLS can provide in this situation (with C=15 genrelabels).• Non-linear projection: To increase the expressive powerof the overall classifier, we used a kernel implementa-tion [6] of the dimensionality reduction method, i.e.,data was first projected to a high dimensional space,where features with improved discrimination capabil-ities can be obtained.2.2. Data PostprocessingThe genre is predicted every second but for visual purposesthe results are smoothed before they are sent to the display.The genre vector presented is a running average over theprevious four predictions. A screen-shot of the plug-in isshown in figure 1.3. EvaluationBefore the online implementation the genre predictor wastested on a set of unseen data and error an rate of 62.5 %was achieved. In the test, a song was classified according toby the sum of the one-second predictions in the song. Figure2 shows the confusion matrix. The confusion is consistentwith known similarity between genre. When the predictorfails for blues it is most often because it predicts jazz, coun-try or rock. The confusion matrix also illustrates that somegenres are harder to predict than others; Christian musicis for example often miss-classified. This reflects the factClassification labelTrue LabelA B Ch Cl Co D F J L N O P R&BRa RoAlt RockBluesChristianClassicalCountryDanceFolkJazzLatinNew AgeOperaPopR&BRapRock102030405060708090Figure 2. The confusion matrix illustrates that the majorityof errors are due to ’reasonable’ confusions such as ’Alter-native rock’ - ’Rock’ or ’Folk’ - ’Country’.that the ’Christian’ label is not directly related to the mu-sical content, but rather derived from the context and pos-sibly the lyrics. Hence, a song labeled ’Christian’ could,based on the music alone, also be classified in categoriessuch as ’folk’, ’rock’ or ’jazz’. Whether people will findit interesting to use the plug-in remains to be seen; this infact is the evaluation that really matters. The plan is tomake the plug-in publicly available before the conferenceat http://www.intelligentsound.org/References[1] T. W. Anderson. An Introduction to Multivariate StatisticalAnalysis. Wiley-Interscience, NY, 3rd edition, 2003.[2] Jean-Julien Aucouturier and Francois Pachet. Representingmusical genre: A state of the art. Journal of New MusicResearch, 32:83–93, 2003.[3] C. M. Bishop. Neural Networks for Pattern Recognition. Ox-ford University Press, 1995.[4] Martin F. McKinney and Jeroen Breebart. Features for audioand music classification. In Proceedings of the InternationalConference on Music Information Retrieval (ISMIR), 2003.[5] A. Meng, P. Ahrendt, and J. Larsen. Improving music genreclassification by short-time feature integration. In IEEE In-ternational Conference on Acoustics, Speech, and SignalProcessing, volume V, pages 497–500, mar 2005.[6] B Schlkopf and A. J. Smola. Learning with Kernels. MITPress, 2001.[7] George Tzanetakis and Perry Cook. Music genre classifi-cation of audio signals. IEEE Transactions on Speech andAudio Processing, 10(5):293–302, July 2002.[8] Erling Wold, Thom Blum, Douglas Keislar, and JamesWheaton. Content-based classification, search, and retrievalof audio. IEEE Multimedia, 3(3):27–36, 1996.

A Genre Classification Plug-in for Data Collection

http://orbit.dtu.dk/ws/files/5800859/imm4520.pdf

A Genre Classification Plug-in for Data Collection

Abstract

Similar works

Full text

Available Versions

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY

Online Research Database In Technology