Learning, Probability and Logic: Toward a Unified Approach for Content-Based Music Information Retrieval

Within the last 15 years, the field of Music Information Retrieval (MIR) has made tremendous progress in the development of algorithms for organizing and analyzing the ever-increasing large and varied amount of music and music-related data available digitally. However, the development of content-based methods to enable or ameliorate multimedia retrieval still remains a central challenge. In this perspective paper, we critically look at the problem of automatic chord estimation from audio recordings as a case study of content-based algorithms, and point out several bottlenecks in current approaches: expressiveness and flexibility are obtained to the expense of robustness and vice versa; available multimodal sources of information are little exploited; modeling multi-faceted and strongly interrelated musical information is limited with current architectures; models are typically restricted to short-term analysis that does not account for the hierarchical temporal structure of musical signals. Dealing with music data requires the ability to tackle both uncertainty and complex relational structure at multiple levels of representation. Traditional approaches have generally treated these two aspects separately, probability and learning being the usual way to represent uncertainty in knowledge, while logical representation being the usual way to represent knowledge and complex relational information. We advocate that the identified hurdles of current approaches could be overcome by recent developments in the area of Statistical Relational Artificial Intelligence (StarAI) that unifies probability, logic and (deep) learning. We show that existing approaches used in MIR find powerful extensions and unifications in StarAI, and we explain why we think it is time to consider the new perspectives offered by this promising research field

An Analytical Methodology for the Investigation of the Relationship of Music and Lyrics in Popular Music

This thesis details the conception and design of a new methodology for examining pop songs holistically; considering both music and lyrics and examining the synergies between the two. Central to this methodology is the application of a data extraction framework, which has been designed to mine information about musical and lyrical phenomena. This framework operates as a common source for producing data about two very different media, avoiding individual interpretation where this is possible. The methodology has been designed to address specific questions about the relationship between music and lyrics, but the main purpose of the thesis is to evaluate the usefulness of the endeavour. In order to examine the efficacy of this approach, the framework was used to populate a dataset made up of a sample of 300 songs, which was subsequently explored and analysed through a series of case studies which investigate combinations of metrics concerned with music and lyrics for the whole sample, as well as analysis of specific subsets defined by a range of parameters. These case studies have demonstrated the various ways this approach might be used, as well as working as proof of concept. The conclusion of the thesis reviews the various case studies in the context of presenting potential uses of the framework as a tool and the broader methodology by other scholars. There is also a consideration of how the overall data might be affected by the inclusion of genres and styles that are not included in the initial sample set

Models for music analysis from a Markov logic networks perspective

International audienceAnalyzing and formalizing the intricate mechanisms of music is a very challenging goal for Artificial Intelligence. Dealing with real audio recordings requires the ability to handle both uncertainty and complex relational structure at multiple levels of representation. Until now, these two aspects have been generally treated separately, probability being the standard way to represent uncertainty in knowledge, while logical representation being the standard way to represent knowledge and complex relational information. Several approaches attempting a unification of logic and probability have recently been proposed. In particular, Markov logic networks (MLNs), which combine first-order logic and probabilistic graphical models, have attracted increasing attention in recent years in many domains. This paper introduces MLNs as a highly flexible and expressive formalism for the analysis of music that encompasses most of the commonly used probabilistic and logic-based models. We first review and discuss existing approaches for music analysis. We then introduce MLNs in the context of music signal processing by providing a deep understanding of how they specifically relate to traditional models, specifically hidden Markov models and conditional random fields. We then present a detailed application of MLNs for tonal harmony music analysis that illustrates the potential of this framework for music processing