Integration between different data formats, and between data belonging to different collections, is an ongoing challenge in the MIR field. Semantic Web tools have proved to be promising resources for making different types of music information interoperable. However, the use of these technologies has so far been limited and scattered in the field. To address this, the Polifonia project is developing an ontological ecosystem that can cover a wide variety of musical aspects (musical features, instruments, emotions, performances). In this paper, we present the Polifonia Ontology Network, an ecosystem that enables and fosters the transition towards semantic MIR

Albert Meroño-Peñuela

Andrea Poltronieri

Delfina Sol Martinez Pandiani

Fiorela Ciroku

Jacopo de Berardinis

Valentina Anita Carriero

Valentina Presutti

English

Albert Meroño-Peñuela

Archivio istituzionale della ricerca - Alma Mater Studiorum Università di Bologna

SEMANTIC INTEGRATION OF MIR DATASETSWITH THE POLIFONIA ONTOLOGY NETWORKValentina Anita Carriero1 Fiorela Ciroku1 Jacopo de Berardinis2Delfina Sol Martinez Pandiani1 Albert Meroño-Peñuela2 Andrea Poltronieri1 Valentina Presutti11 University of Bologna, Italy2 King’s College London, United Kingdomandrea.poltronieri2@unibo.itABSTRACTIntegration between different data formats, and betweendata belonging to different collections, is an ongoing chal-lenge in the MIR field. Semantic Web tools have proved tobe promising resources for making different types of mu-sic information interoperable. However, the use of thesetechnologies has so far been limited and scattered in thefield. To address this, the Polifonia project 1 is developingan ontological ecosystem that can cover a wide variety ofmusical aspects (musical features, instruments, emotions,performances). In this paper, we present the Polifonia On-tology Network, an ecosystem that enables and fosters thetransition towards Semantic MIR.1. INTRODUCTIONDuring the last 20 years, the field of Music InformationRetrieval (MIR) has seen the introduction of an increasingnumber of music datasets, enabling researchers to train andevaluate algorithms for several tasks, from chord recogni-tion and beat detection, to source separation and mood de-tection. However, the availability of audio data is still lim-ited, and two overlooked issues remain: (i) music datasetsare commonly provided as independent and isolated col-lections, with little or no alignment at the metadata andannotation level; (ii) even when tracks/compositions arecoupled with universal identifiers (e.g. MusicBrainz IDs,ISRC), there is no direct way to access and link hetero-geneous music-related data from online databases, such asWikipedia, Genius [1], and Songfacts [2]. The disconnectamong music datasets jeopardises their potential integra-tion, and hence their extension and the combination of an-notations of different kinds. Simultaneously, the low levelof linkage with other databases discourages multi-modal1 https://polifonia-project.eu© V.A. Carriero, F. Ciroku, J. de Berardinis, D.S. MartinezPandiani, A. Meroño-Peñuela, A. Poltronieri, and V. Presutti. Licensedunder a Creative Commons Attribution 4.0 International License (CC BY4.0). Attribution: V.A. Carriero, F. Ciroku, J. de Berardinis, D.S. Mar-tinez Pandiani, A. Meroño-Peñuela, A. Poltronieri, and V. Presutti, “Se-mantic Integration of MIR Datasets with the Polifonia Ontology Net-work”, in Extended Abstracts for the Late-Breaking Demo Session of the22nd Int. Society for Music Information Retrieval Conf., Online, 2021.research in the field, where the availability of heteroge-neous music-related data (text, images, locations, etc.) isan essential asset. Therefore, MIR practitioners interestedin multi-modal and/or multi-task research across existentmusic collections are commonly left to implement com-plex data collection and integration pipelines [3].Semantic Web (SW) technologies [4] allow many ofthese limitations to be overcome, as they provide a com-mon framework for data to be shared and reused acrossapplication, enterprise, and community boundaries. Inthe Resource Description Framework [5], entities can bedescribed through triples of the form subject-predicate-object, and can have unique IDs – allowing to connectthem to other entities through additional triples and forma Knowledge Graph (KG). Because not only documentsbut any entity of interest – such as musical resources – canbe annotated, SW technologies greatly facilitate effectivedata access and integration, resource discovery, semanticreasoning and knowledge extraction, while also promot-ing interoperability among resources, information models,data providers and consumers. As such, they have beenadopted by both large corporations, such as Oracle, IBM,Google 2 ), and many domains dealing with vast quanti-ties of data gathered with different encoding formats, suchas health care, life sciences, and cultural heritage 3 . Awealth of recent research is dedicated to the adoption ofSW knowledge bases as references for enhancing Infor-mation Extraction tasks [6].In light of this, SW technologies and principles are idealfor MIR, as they could easily address the disconnection andthe low level of linkage of music collections. Nonethe-less, despite the numerous music ontologies to date, highfragmentation and poor maintenance of these contributionsare currently hindering the transition to Semantic MIR (seeSection 2). To fill this gap, we are developing an ontolog-ical ecosystem providing a standard, flexible and expres-sive schema to represent and describe/annotate heteroge-neous musical data of different formats, genres, and prove-nance. By leveraging the different sets of skills and ex-pertise of music scholars in Polifonia, we aim at unifyingviews and identifying requirements across different disci-plines to guide the ontology design activities, while fol-lowing state of the art methodologies for data engineering.2 https://schema.org3 https://w3.org/2001/sw/sweo/public/UseCases/2. MUSIC ONTOLOGIESIn the last two decades several ontologies have been devel-oped for diverse music-related applications, dealing withboth symbolic notations and audio signals at different lev-els of specificity. Some ontologies have been designed fordescribing high-level music-related information, such asthe The Music Ontology [7] and the DOREMUS Ontol-ogy [8]. Other ontologies describe musical notation, bothfrom scores and symbolic representations. For example,the MIDI Linked Data Cloud [9] proposes the interconnec-tion of symbolic music descriptions encoded in MIDI for-mat, and the CHARM ontology [10] aims to describe mu-sical structures based on the CHARM specifications. TheMusic Theory Ontology (MTO) [11] describes theoreticalconcepts related to a music composition, while the Mu-sic Score Ontology [12] represents similar concepts witha focus on music sheet notation. Finally, the Music No-tation Ontology [13] focuses on the core “semantic” infor-mation present in a score. Other ontologies aim to describespecific aspects of the musical domain, such as the ChordOntology, the Tonality Ontology, the Temperament Ontol-ogy [14], and the Segment Ontology [15]. The Audio Fea-tures Ontology [16], the Studio Ontology [17], and the Au-dio Effects Ontology [18] describe audio signals and pro-duction procedures. Others have also been used to modellistening habits and music tastes [19], music-induced emo-tions [20], and to describe musical similarities [21].The focus of these ontologies is generally specific, cov-ering only particular aspects of musical content. However,music consists of a dense connected network of heteroge-neous elements that concert with each other. Furthermore,many of these ontologies were developed as stand-aloneprojects, with little or no alignment to other relevant on-tologies within the same domain.3. THE POLIFONIA ONTOLOGY NETWORKBesides the expressive and modular design, allowing torepresent a wide range of music-related concepts and re-lations, the Polifonia ontology network (ON) also bringstwo key assets desirable for the whole ecosystem. First,music objects are described in a way to be connected withrelevant knowledge, such as their links to tangible objects,cultural and historical contexts, stories about them, as wellas related facts expressed in different styles, disciplines,languages. This will enable the development of an ecosys-tem of computational methods and tools supporting dis-covery, extraction, encoding, interlinking, classification,exploration of, and access to, musical heritage knowledgeon the Web. Second, the development of Polifonia ON 4is driven by eXtreme Design (XD) [22] – an agile ontol-ogy engineering methodology, and makes extensive use ofontology design patterns (ODPs) – small ontologies thatwork as reusable templates for recurrent modelling prob-lems. For example, the common modelling problem inthe music domain “Where was a musical composition per-formed?” could be addressed by an ODP modelling the4 https://github.com/polifonia-project/relation between compositions, performances, and places.XD is executed through multiple iterations of a set ofcollaborative steps: (i) requirements collection; (ii) ontol-ogy design and KG generation; (iii) ontology testing andintegration. A story-based approach guides the collectionof the project’s requirements. A story is a framework forcustomers to describe their needs, and is composed of 4sections: the persona, a description of a typical user; thegoal that she needs to address; the scenario, describinghow she will address the goal(s); the competency questions(CQs), that translate the needs of the persona into require-ments. The ontology modelling starts iteratively from theCQs, and is based on the reuse of ODPs and entities ex-tracted from existing ontologies as templates. For instance,if an existing ontology already addresses one of the CQs,alignment axioms 5 make it evident which part(s) havebeen reused in Polifonia ON and support interoperabilitybetween ontologies. This makes Polifonia ON and KG in-teroperable and reusable beyond the Polifonia project.The identification of the 11 ontology modules emergedfrom the investigation, and thematic classification, of allthe CQs from the 19 stories recorded so far. The Fullmodule is the entry point of the whole network. TheCore module models general-purpose concepts and rela-tionships (e.g. place, time). The Musical Performance andMusical Composition modules represent musical perfor-mances and events, and musical compositions respectively.The Musical Feature module provides a model to describea musical object in regard to the musical properties that canbe objectively attributed or subjectively identified from it.The Source module represents sources of music-related in-formation. The Instrument module proposes a taxonomyof instruments and their technical properties, whereas Bellfocuses on bells. The Music emotion module provides amodel to describe emotions both perceived and induced bya composition wrt the musical features. A module namedComparative Measure defines ODPs describing observa-tions and measures applied to comparative analysis, suchas similarities between musical pieces. Lastly, the Meta-data module aims at supporting the representation of meta-data about musical resources. The Polifonia ON is beingpopulated by data from various existing datasets.4. CONCLUSIONS AND FUTURE WORKIn this paper, we addressed the recurring problem of in-tegration and linkage of MIR datasets. By means of SWtechnologies and best practices, we are laying the founda-tions and providing the infrastructure for Semantic MIR.Not only will this support computational music analysis,but the KGs resulting from the interconnection of MIRdatasets and their integration with other sources can be ex-plored through symbolic reasoning to derive novel musi-cal knowledge and test musicological hypotheses. We arecurrently developing a first version of the Polifonia ontol-ogy network, and future work will iteratively extend theecosystem as a result of continuous expert feedback.5 E.g. an alignment axiom could assert that the concept of Person inPolifonia has the same semantics as the concept of Person in FOAF.Acknowledgements. The Polifonia project has receivedfunding from the EU Horizon 2020 research and innova-tion programme under grant agreement No 101004746.5. REFERENCES[1] “Genius website,” http://www.genius.com, accessed:2021-10-30.[2] “Songfacts website,” http://www.songfacts.com, ac-cessed: 2021-10-30.[3] I. A. P. Santana, F. Pinhelli, J. Donini, L. Catharin,R. B. Mangolin, V. D. Feltrim, M. A. Domingues et al.,“Music4all: A new music database and its applica-tions,” in 2020 International Conference on Systems,Signals and Image Processing (IWSSIP). IEEE, 2020.[4] T. Berners-Lee, J. Hendler, and O. Lassila, “The se-mantic web,” Scientific american, vol. 284, no. 5, 2001.[5] O. Lassila, R. R. Swick et al., “Resource descriptionframework (rdf) model and syntax specification,” 1998.[6] J. L. Martinez-Rodriguez, A. Hogan, and I. Lopez-Arevalo, “Information extraction meets the semanticweb: a survey,” Semantic Web, vol. 11, no. 2, 2020.[7] Y. Raimond, S. Abdallah, M. Sandler, and F. Giasson,“The music ontology,” in Proceedings of the 8th Inter-national Conference on Music Information Retrieval(ISMIR 2007), Vienna, Austria, Sep. 2007.[8] P. Lisena and R. Troncy, “Doing reusable musical data(DOREMUS),” in Proceedings of Workshops and Tu-torials of the 9th International Conference on Knowl-edge Capture (K-CAP2017), Austin, Texas, USA, De-cember 4th, 2017, ser. CEUR Workshop Proceedings,vol. 2065. CEUR-WS.org, 2017.[9] A. Meroño-Peñuela, R. Hoekstra, A. Gangemi,P. Bloem, R. de Valk, B. Stringer, B. Janssen,V. de Boer, A. Allik, S. Schlobach, and K. Page, “Themidi linked data cloud,” in The Semantic Web – ISWC2017. Cham: Springer International Publishing, 2017.[10] N. Harley and G. Wiggins, “An ontology for abstract,hierarchical music representation,” in Demo at the 16thInternational Society for Music Information RetrievalConference (ISMIR 2015), Malaga, Spain, 2015.[11] S. M. Rashid, D. De Roure, and D. L. McGuinness,“A music theory ontology,” in Proceedings of the 1stInternational Workshop on Semantic Applications forAudio and Music, ser. SAAM ’18. New York, NY,USA: Association for Computing Machinery, 2018, p.6–14.[12] J. Jones, D. de Siqueira Braga, K. Tertuliano, andT. Kauppinen, “Musicowl: The music score ontology,”in Proceedings of the International Conference on WebIntelligence, ser. WI ’17. New York, NY, USA: As-sociation for Computing Machinery, 2017.[13] S. S.-s. Cherfi, C. Guillotel, F. Hamdi, P. Rigaux,and N. Travers, “Ontology-based annotation of mu-sic scores,” in Proceedings of the Knowledge CaptureConference, ser. K-CAP 2017. New York, NY, USA:Association for Computing Machinery, 2017.[14] G. Fazekas, Y. Raimond, K. Jacobson, and M. Sandler,“An overview of semantic web activities in the omras2project,” Journal of New Music Research, vol. 39, 122010.[15] B. Fields, K. R. Page, D. D. Roure, and T. Craw-ford, “The segment ontology: Bridging music-genericand domain-specific,” in Proceedings of the 2011 IEEEInternational Conference on Multimedia and Expo,ICME 2011, 11-15 July, 2011, Barcelona, Catalonia,Spain. IEEE Computer Society, 2011.[16] A. Allik, G. Fazekas, and M. B. Sandler, “An ontologyfor audio features,” in Proceedings of the 17th Interna-tional Society for Music Information Retrieval Confer-ence, ISMIR 2016, New York City, United States, Au-gust 7-11, 2016, 2016.[17] G. Fazekas and M. B. Sandler, “The studio ontologyframework,” in Proceedings of the 12th InternationalSociety for Music Information Retrieval Conference,ISMIR 2011, Miami, Florida, USA, October 24-28,2011. University of Miami, 2011.[18] T. Wilmering, G. Fazekas, and M. B. Sandler, “The au-dio effects ontology,” in Proceedings of the 14th Inter-national Society for Music Information Retrieval Con-ference, ISMIR 2013, Curitiba, Brazil, November 4-8,2013, 2013.[19] S. Rho, S. Song, E. Hwang, and M. Kim, “Comus:Ontological and rule-based reasoning for music recom-mendation system,” in Advances in Knowledge Discov-ery and Data Mining. Berlin, Heidelberg: SpringerBerlin Heidelberg, 2009.[20] H. H. Kim, “A semantically enhanced tag-based mu-sic recommendation using emotion ontology,” in In-telligent Information and Database Systems. Berlin,Heidelberg: Springer Berlin Heidelberg, 2013.[21] K. Jacobson, Y. Raimond, and M. B. Sandler, “Anecosystem for transparent music similarity in an openworld,” in Proceedings of the 10th International Soci-ety for Music Information Retrieval Conference, ISMIR2009, Kobe International Conference Center, Kobe,Japan, October 26-30, 2009. International Societyfor Music Information Retrieval, 2009.[22] E. Blomqvist, K. Hammar, and V. Presutti, “Engi-neering ontologies with patterns-the extreme designmethodology.” Ontology Engineering with OntologyDesign Patterns, no. 25, 2016.

Semantic Integration of MIR Datasets with the Polifonia Ontology Network

https://cris.unibo.it/bitstream/11585/874450/1/ISMIR_LBD_2021__Polifonia_ON.pdf

Semantic Integration of MIR Datasets with the Polifonia Ontology Network

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Alma Mater Studiorum Università di Bologna