Semantic Federation of Musical and Music-Related Information for Establishing a Personal Music Knowledge Base

Music is perceived and described very subjectively by every individual. Nowadays, people often get lost in their steadily growing, multi-placed, digital music collection. Existing music player and management applications get in trouble when dealing with poor metadata that is predominant in personal music collections. There are several music information services available that assist users by providing tools for precisely organising their music collection, or for presenting them new insights into their own music library and listening habits. However, it is still not the case that music consumers can seamlessly interact with all these auxiliary services directly from the place where they access their music individually. To profit from the manifold music and music-related knowledge that is or can be available via various information services, this information has to be gathered up, semantically federated, and integrated into a uniform knowledge base that can personalised represent this data in an appropriate visualisation to the users. This personalised semantic aggregation of music metadata from several sources is the gist of this thesis. The outlined solution particularly concentrates on users’ needs regarding music collection management which can strongly alternate between single human beings. The author’s proposal, the personal music knowledge base (PMKB), consists of a client-server architecture with uniform communication endpoints and an ontological knowledge representation model format that is able to represent the versatile information of its use cases. The PMKB concept is appropriate to cover the complete information flow life cycle, including the processes of user account initialisation, information service choice, individual information extraction, and proactive update notification. The PMKB implementation makes use of SemanticWeb technologies. Particularly the knowledge representation part of the PMKB vision is explained in this work. Several new Semantic Web ontologies are defined or existing ones are massively modified to meet the requirements of a personalised semantic federation of music and music-related data for managing personal music collections. The outcome is, amongst others, • a new vocabulary for describing the play back domain, • another one for representing information service categorisations and quality ratings, and • one that unites the beneficial parts of the existing advanced user modelling ontologies. The introduced vocabularies can be perfectly utilised in conjunction with the existing Music Ontology framework. Some RDFizers that also make use of the outlined ontologies in their mapping definitions, illustrate the fitness in practise of these specifications. A social evaluation method is applied to carry out an examination dealing with the reutilisation, application and feedback of the vocabularies that are explained in this work. This analysis shows that it is a good practise to properly publish Semantic Web ontologies with the help of some Linked Data principles and further basic SEO techniques to easily reach the searching audience, to avoid duplicates of such KR specifications, and, last but not least, to directly establish a \"shared understanding\". Due to their project-independence, the proposed vocabularies can be deployed in every knowledge representation model that needs their knowledge representation capacities. This thesis added its value to make the vision of a personal music knowledge base come true.:1 Introduction and Background 11 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Personal Music Collection Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2 Music Information Management 17 2.1 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 Knowledge Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.1.1 Knowledge Representation Models . . . . . . . . . . . . . . . . . 18 2.1.1.2 Semantic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.1.3 Ontologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.2 Knowledge Management Systems . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.2.1 Information Services . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.2.2 Ontology-based Distributed Knowledge Management Systems . . 20 2.1.2.3 Knowledge Management System Design Guideline . . . . . . . . 21 2.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.2 Semantic Web Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.2.1 The Evolution of the World Wide Web . . . . . . . . . . . . . . . . . . . . . 22 Personal Music Knowledge Base Contents 2.2.1.1 The Hypertext Web . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2.1.2 The Normative Principles of Web Architecture . . . . . . . . . . . 23 2.2.1.3 The Semantic Web . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2.2 Common Semantic Web Knowledge Representation Languages . . . . . . 25 2.2.3 Resource Description Levels and their Relations . . . . . . . . . . . . . . . 26 2.2.4 Semantic Web Knowledge Representation Models . . . . . . . . . . . . . . 29 2.2.4.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.4.2 Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.4.3 Context Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2.4.4 Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2.4.5 Providing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.4.6 Consuming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3 Music Content and Context Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.3.1 Categories of Musical Characteristics . . . . . . . . . . . . . . . . . . . . . 37 2.3.2 Music Metadata Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.3.3 Music Metadata Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.3.3.1 Audio Signal Carrier Indexing Services . . . . . . . . . . . . . . . . 41 2.3.3.2 Music Recommendation and Discovery Services . . . . . . . . . . 42 2.3.3.3 Music Content and Context Analysis Services . . . . . . . . . . . 43 2.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.4 Personalisation and Environmental Context . . . . . . . . . . . . . . . . . . . . . . 44 2.4.1 User Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.4.2 Context Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.4.3 Stereotype Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3 The Personal Music Knowledge Base 48 3.1 Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.1.1 Knowledge Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.1.2 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.3.1 User Account Initialisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.3.2 Individual Information Extraction . . . . . . . . . . . . . . . . . . . . . . . . 53 3.3.3 Information Service Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3.4 Proactive Update Notification . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.3.5 Information Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.3.6 Personal Associations and Context . . . . . . . . . . . . . . . . . . . . . . . 56 3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4 A Personal Music Knowledge Base 57 4.1 Knowledge Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.1.1 The Info Service Ontology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.1.2 The Play Back Ontology and related Ontologies . . . . . . . . . . . . . . . . 61 4.1.2.1 The Ordered List Ontology . . . . . . . . . . . . . . . . . . . . . . 61 4.1.2.2 The Counter Ontology . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.1.2.3 The Association Ontology . . . . . . . . . . . . . . . . . . . . . . . 64 4.1.2.4 The Play Back Ontology . . . . . . . . . . . . . . . . . . . . . . . . 65 4.1.3 The Recommendation Ontology . . . . . . . . . . . . . . . . . . . . . . . . 69 4.1.4 The Cognitive Characteristics Ontology and related Vocabularies . . . . . . 72 4.1.4.1 The Weighting Ontology . . . . . . . . . . . . . . . . . . . . . . . 72 4.1.4.2 The Cognitive Characteristics Ontology . . . . . . . . . . . . . . . 73 4.1.4.3 The Property Reification Vocabulary . . . . . . . . . . . . . . . . . 78 4.1.5 The Media Types Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.2 Knowledge Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5 Personal Music Knowledge Base in Practice 87 5.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.1.1 AudioScrobbler RDF Service . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.1.2 PMKB ID3 Tag Extractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.2.1 Reutilisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.2.2 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.2.3 Reviews and Mentions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.2.4 Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6 Conclusion and Future Work 93 6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Toward a Robust Diversity-Based Model to Detect Changes of Context

Being able to automatically and quickly understand the user context during a session is a main issue for recommender systems. As a first step toward achieving that goal, we propose a model that observes in real time the diversity brought by each item relatively to a short sequence of consultations, corresponding to the recent user history. Our model has a complexity in constant time, and is generic since it can apply to any type of items within an online service (e.g. profiles, products, music tracks) and any application domain (e-commerce, social network, music streaming), as long as we have partial item descriptions. The observation of the diversity level over time allows us to detect implicit changes. In the long term, we plan to characterize the context, i.e. to find common features among a contiguous sub-sequence of items between two changes of context determined by our model. This will allow us to make context-aware and privacy-preserving recommendations, to explain them to users. As this is an ongoing research, the first step consists here in studying the robustness of our model while detecting changes of context. In order to do so, we use a music corpus of 100 users and more than 210,000 consultations (number of songs played in the global history). We validate the relevancy of our detections by finding connections between changes of context and events, such as ends of session. Of course, these events are a subset of the possible changes of context, since there might be several contexts within a session. We altered the quality of our corpus in several manners, so as to test the performances of our model when confronted with sparsity and different types of items. The results show that our model is robust and constitutes a promising approach.Comment: 27th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2015), Nov 2015, Vietri sul Mare, Ital

A review of the role of sensors in mobile context-aware recommendation systems

Recommendation systems are specialized in offering suggestions about specific items of different types (e.g., books, movies, restaurants, and hotels) that could be interesting for the user. They have attracted considerable research attention due to their benefits and also their commercial interest. Particularly, in recent years, the concept of context-aware recommendation system has appeared to emphasize the importance of considering the context of the situations in which the user is involved in order to provide more accurate recommendations. The detection of the context requires the use of sensors of different types, which measure different context variables. Despite the relevant role played by sensors in the development of context-aware recommendation systems, sensors and recommendation approaches are two fields usually studied independently. In this paper, we provide a survey on the use of sensors for recommendation systems. Our contribution can be seen from a double perspective. On the one hand, we overview existing techniques used to detect context factors that could be relevant for recommendation. On the other hand, we illustrate the interest of sensors by considering different recommendation use cases and scenarios

The Internet of Musical Things Ontology

The Internet of Musical Things (IoMusT) is an emerging research area consisting of the extension of the Internet of Things paradigm to the music domain. Interoperability represents a central issue within this domain, where heterogeneous objects dedicated to the production and/or reception of musical content (Musical Things) are envisioned to communicate between each other. This paper proposes an ontology for the representation of the knowledge related to IoMusT ecosystems to facilitate interoperability between Musical Things. There was no previous comprehensive data model for the IoMusT domain, however the new ontology relates to existing ontologies, including the SOSA Ontology for the representation of sensors and actuators and the Music Ontology focusing on the production and consumption of music. This paper documents the design of the ontology and its evaluation with respect to specific requirements gathered from an extensive literature review, which was based on scenarios involving IoMusT stakeholders, such as performers and audience members. The IoMusT Ontology can be accessed at: https://w3id.org/iomust#

A hybrid approach for item collection recommendations : an application to automatic playlist continuation

Current recommender systems aim mainly to generate accurate item recommendations, without properly evaluating the multiple dimensions of the recommendation problem. However, in many domains, like in music, where items are rarely consumed in isolation, users would rather need a set of items, designed to work well together, while having some cognitive properties as a whole, related to their perception of quality and satisfaction. In this thesis, a hybrid case-based recommendation approach for item collections is proposed. In particular, an application to automatic playlist continuation, addressing similar cognitive concepts, rather than similar users, is presented. Playlists, that are sets of music items designed to be consumed as a sequence, with a specific purpose and within a specific context, are treated as cases. The proposed recommender system is based on a meta-level hybridization. First, Latent Dirichlet Allocation is applied to the set of past playlists, described as distributions over music styles, to identify their underlying concepts. Then, for a started playlist, its semantic characteristics, like its latent concept and the styles of the included items, are inferred, and Case-Based Reasoning is applied to the set of past playlists addressing the same concept, to construct and recommend a relevant playlist continuation. A graph-based item model is used to overcome the semantic gap between songs’ signal-based descriptions and users’ high-level preferences, efficiently capture the playlists’ structures and the similarity of the music items in those. As the proposed method bases its reasoning on previous playlists, it does not require the construction of complex user profiles to generate accurate recommendations. Furthermore, apart from relevance, support to parameters beyond accuracy, like increased coherence or support to diverse items is provided to deliver a more complete user experience. Experiments on real music datasets have revealed improved results, compared to other state of the art techniques, while achieving a “good trade-off” between recommendations’ relevance, diversity and coherence. Finally, although actually focusing on playlist continuations, the designed approach could be easily adapted to serve other recommendation domains with similar characteristics.Los sistemas de recomendación actuales tienen como objetivo principal generar recomendaciones precisas de artículos, sin evaluar propiamente las múltiples dimensiones del problema de recomendación. Sin embargo, en dominios como la música, donde los artículos rara vez se consumen en forma aislada, los usuarios más bien necesitarían recibir recomendaciones de conjuntos de elementos, diseñados para que se complementaran bien juntos, mientras se cubran algunas propiedades cognitivas, relacionadas con su percepción de calidad y satisfacción. En esta tesis, se propone un sistema híbrido de recomendación meta-nivel, que genera recomendaciones de colecciones de artículos. En particular, el sistema se centra en la generación automática de continuaciones de listas de música, tratando conceptos cognitivos similares, en lugar de usuarios similares. Las listas de reproducción son conjuntos de elementos musicales diseñados para ser consumidos en secuencia, con un propósito específico y dentro de un contexto específico. El sistema propuesto primero aplica el método de Latent Dirichlet Allocation a las listas de reproducción, que se describen como distribuciones sobre estilos musicales, para identificar sus conceptos. Cuando se ha iniciado una nueva lista, se deducen sus características semánticas, como su concepto y los estilos de los elementos incluidos en ella. A continuación, el sistema aplica razonamiento basado en casos, utilizando las listas del mismo concepto, para construir y recomendar una continuación relevante. Se utiliza un grafo que modeliza las relaciones de los elementos, para superar el ?salto semántico? existente entre las descripciones de las canciones, normalmente basadas en características sonoras, y las preferencias de los usuarios, expresadas en características de alto nivel. También se utiliza para calcular la similitud de los elementos musicales y para capturar la estructura de las listas de dichos elementos. Como el método propuesto basa su razonamiento en las listas de reproducción y no en usuarios que las construyeron, no se requiere la construcción de perfiles de usuarios complejos para poder generar recomendaciones precisas. Aparte de la relevancia de las recomendaciones, el sistema tiene en cuenta parámetros más allá de la precisión, como mayor coherencia o soporte a la diversidad de los elementos para enriquecer la experiencia del usuario. Los experimentos realizados en bases de datos reales, han revelado mejores resultados, en comparación con las técnicas utilizadas normalmente. Al mismo tiempo, el algoritmo propuesto logra un "buen equilibrio" entre la relevancia, la diversidad y la coherencia de las recomendaciones generadas. Finalmente, aunque la metodología presentada se centra en la recomendación de continuaciones de listas de reproducción musical, el sistema se puede adaptar fácilmente a otros dominios con características similares.Postprint (published version

CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges