Music is inherently expressive of emotion meaning and affects the mood of people. In this paper, we present a novel EMIR (Emotional Music Information Retrieval) System that uses latent emotion elements both in music and non-descriptive queries (NDQs) to detect implicit emotional association between users and music to enhance Music Information Retrieval (MIR). We try to understand the latent emotional intent of queries via machine learning for emotion classification and compare the performance of emotion detection approaches on different feature sets. For this purpose, we extract music emotion features from lyrics and social tags crawled from the Internet, label some for training and model them in high-dimensional emotion space and recognize latent emotion of users by query emotion analysis. The similarity between queries and music is computed by verified BM25 model

Gurrin, Cathal

Lin, Hongfei

Zhou, Lijuan Marissa

English

DCU Online Research Access Service

EMIR: A Novel Emotion-based Music Retrieval System Lijuan Zhou1, Hongfei Lin1, Cathal Gurrin2,Wenfei Liu1  1School of Computer Science and Technology, Dalian University of Technology 2 School of Computing, Dublin City University marissa.zhou.cn@gmail.com, hflin@dlut.edu.cn, cgurrin@computing.dcu.ie liuwenfei@mail.dlut.edu.cn Abstract.Musicis inherently expressive of emotion meaning and affects the mood of people. In this paper, we present a novel EMIR (Emotional Music Information Retrieval) System that uses latent emotion elements both in music and non-descriptive queries (NDQs) to detect implicit emotional association between users and music to enhance Music Information Retrieval (MIR). We try to understand the latent emotional intent of queries via machine learning for emotion classification and compare the performance of emotion detection approaches on different feature sets. For this purpose, we extract music emotion features from lyrics and social tags crawled from the Internet, label some for training and model them in high-dimensional emotion space and recognize latent emotion of users by query emotion analysis. The similarity between queries and music is computed by verified BM25 model.  Keywords: Music Information Retrieval, Emotion Detection, Machine Learning, Human Computer Interaction 1   Introduction Music is an indispensable element in our daily life and inherently expressive of affective meaning. But prevalent music searching and sharing systems (Google Music, Last.fm and Xiami Music etc.) prompt users to input music descriptive keywords (title, album, artist and singer etc.) to search for songs. But in many cases, MIR system users may have no specific piece of music in mind but only prefer to tell the system what kind of emotional music that they want to listen to. They may just submit a query like “I fall in love now” or “Feeling Depressed”. We refer to these kind of queries as Non-Descriptive Queries (NDQs), in that they containno music descriptive information but really shows the implicit emotional need of searchers. Hence there is a new type of semantic gap to be solved; the gap between a user’s emotional feeling (as expressed in the query) and the emotionally expressive content of music. NDQs are more freely input, coherent and inform the system what the user is seeking, though not in a manner that allows easily identifiable content. We try to exploit emotional context to deal with NDQs to develop effective Music IR systems. Text emotion can be categorized into six basic emotion categories (ANGER, DISGUST, FEAR, JOY, SADNESS and SURPRISE) according to WordNet-Affect[5]. Contextual text information such as lyrics has been proved to be effective in music emotion recognition[6]. This work on contextual text information has signiﬁcant impact on music emotion recognition, but very few researchers have explored the application of emotion detection in music information retrieval. Likewise, collaborative social tags have become an essential part of solutions to many MIRproblems as well as lyrics[3,4]. We propose using emotion detection to enrich MIR. The similarity between music and queries is computed in a higher level of emotion space compared with original key attribute matching method.  2   Idea and Implementation With the consideration of emotion need of MIR system users and inherent emotional expression of songs, we design and implement a demo EMIR System.  2.1   Main Idea The main idea of EMIR system is to recognize the implicit emotional need of music searchers. To design such an Emotional Music Information Retrieval system, we need to tackle following problems: 1) Model music representation in an emotion spaceusing themood definitions fromWordNet-Affect. 2) Analyze structure of the lyrics of the music to discover potentialrelationship between emotion and sentence structure. 3) Select the best performingfeature selection method in emotion detection. 4) Design an algorithm to computeemotion similarityand rank retrieval results. To solve all above significant problems, we consider modeling the process of the retrieval in an emotional space and employ machine learning to assist emotion recognition and presentation. Due to the complicacy of emotion based similarity computing method (explicit keyword terms to implicit emotions), we use an increasing saturation to weight the emotions in queries. Finally, we use a revised BM25 to modify the traditional tf-idf weighting method, proposed in our previous work[1,2]. The main idea of the demo system can be briefly described in Fig. 1.   Fig. 1.Flow chart of EMIR system Login Log Analysis Random Emotion Search Emotion Detection Similarity Computing Click & Play Ranking Randomly Play Music Representation 2.2   Prototype MIR System Our EMIR music retrieval system mainly deals with user music retrieval need by analyzing the emotion of users according to their queries and retrieval logs. Our demo system is implementedto work on Android 2.2 smartphones. As shown in Fig. 2 (l to r), users enter a emotion descriptive textual query through the search interface and our emotion detection algorithm detects the implicit emotion of the query by analyzing the keywords for emotionally meaningful clues, querying the music documents encoded into the emotion space and using our ranking algorithm, we rank the music according to the emotional relevance and present to the user as a ranked list.In the result interface, users can view more detailed information of the song or click the play button to listen as well as add it into personal playlist. The EMIR system also collect user click and input messages to generate the mood curve of users which can be used to show the mood fluctuation of the user over time. In later work, we can analyze this mood fluctuation over time to begin to recommend music to users without requiring an expressive query. Other future work is concerned with increasing the dimensionality of the emotion-space, which is currently six emotions.          Fig. 2.Screenshots of Search, Result, Playback and Emotion Charting References 1. Zhou, L.J., Lin, H.F., Liu, W.F.: Enriching Music Information Retrieval Using EmotionDetection. SIGIR 2011 Workshop on Enriching Information Retrieval (ENIR 2011), July 28, 2011, Beijing, China. 2. J. Li, H. Lin, and L. Zhou. Emotion tag based music retrieval algorithm. pages 599–609, Taibei, Taiwan, December 2010. Proceedings of The Sixth Asia Information Retrieval Symposium. 3. P. Lamere. Social tagging and music information retrieval. Raleigh, NC, USA, April 2010. WWW 2010. 4. C. Laurier, M. Sordo, J. Serr`a, and P. Herrera. Musicmood representations from social tags.pages 381–386,Kobe, Japan, October 2009. 10th International Societyof Music Information Retrieval Conference. 5. R. Picard. Aﬀective Computing. MIT press,Cambridge, MA, USA, 2000. 6. M. V. Zaanen and P. Kanters. Automatic moodclassiﬁcation using tf*idf based on lyrics.pages 75–80,Utrecht, Netherlands, August 2010. The 11thInternational Society of Music Information RetrievalConference. 

EMIR: A novel emotion-based music retrieval system

Abstract

Similar works

Full text

Available Versions

DCU Online Research Access Service

Irish Universities

Name not available