Motivic Pattern Classification of Music Audio Signals Combining Residual and LSTM Networks

Motivic pattern classification from music audio recordings is a challenging task. More so in the case of a cappella flamenco cantes, characterized by complex melodic variations, pitch instability, timbre changes, extreme vibrato oscillations, microtonal ornamentations, and noisy conditions of the recordings. Convolutional Neural Networks (CNN) have proven to be very effective algorithms in image classification. Recent work in large-scale audio classification has shown that CNN architectures, originally developed for image problems, can be applied successfully to audio event recognition and classification with little or no modifications to the networks. In this paper, CNN architectures are tested in a more nuanced problem: flamenco cantes intra-style classification using small motivic patterns. A new architecture is proposed that uses the advantages of residual CNN as feature extractors, and a bidirectional LSTM layer to exploit the sequential nature of musical audio data. We present a full end-to-end pipeline for audio music classification that includes a sequential pattern mining technique and a contour simplification method to extract relevant motifs from audio recordings. Mel-spectrograms of the extracted motifs are then used as the input for the different architectures tested. We investigate the usefulness of motivic patterns for the automatic classification of music recordings and the effect of the length of the audio and corpus size on the overall classification accuracy. Results show a relative accuracy improvement of up to 20.4% when CNN architectures are trained using acoustic representations from motivic patterns

Music adapting to the brain: From diffusion chains to neurophysiology

During the last decade, the use of experimental approaches on cultural evolution research has provided novel insights, and supported theoretical predictions, on the principles driving the evolution of human cultural systems. Laboratory simulations of language evolution showed how general-domain constraints on learning, in addition to pressures for language to be expressive, may be responsible for the emergence of linguistic structure. Languages change when culturally transmitted, adapting to fit, among all, the cognitive abilities of their users. As a result, they become regular and compressed, easier to acquire and reproduce. Although a similar theory has been recently extended to the musical domain, the empirical investigation in this field is still scarce. In addition, no study to our knowledge directly addressed the role of cognitive constraints in cultural transmission with neurophysiological investigation. In my thesis I addressed both these issues with a combination of behavioral and neurophysiological methods, in three experimental studies. In study 1 (Chapter 2), I examined the evolution of structural regularities in artificial melodic systems while they were being transmitted across individuals via coordination and alignment. To this purpose I used a new laboratory model of music transmission: the multi-generational signaling games (MGSGs), a variant of the signaling games. This model combines classical aspects of lab-based semiotic models of communication, coordination and interaction (horizontal transmission), with the vertical transmission across generations of the iterated learning model (vertical transmission). Here, two-person signaling games are organized in diffusion chains of several individuals (generations). In each game, the two players (a sender and a receiver) must agree on a common code - here a miniature system where melodic riffs refer to emotions. The receiver in one game becomes the sender in the next game, possibly retransmitting the code previously learned to another generation of participants, and so on to complete the diffusion chain. I observed the gradual evolution of several structures features of musical phrases over generations: proximity, continuity, symmetry, and melodic compression. Crucially, these features are found in most of musical cultures of the world. I argue that we tapped into universal processing mechanisms of structured sequence processing, possibly at work in the evolution of real music. In study 2 (Chapter 3), I explored the link between cultural adaptation and neural information processing. To this purpose, I combined behavioral and EEG study on 2 successive days. I show that the latency of the mismatch negativity (MMN) recorded in a pre-attentive auditory sequence processing task on day 1, predicts how well participants learn and transmit an artificial tone system with affective semantics in two signaling games on day 2. Notably, MMN latencies also predict which structural changes are introduced by participants into the artificial tone system. In study 3 (Chapter 4), I replicated and extended behavioral and neurophysiological findings on the temporal domain of music, with two independent experiments. In the first experiment, I used MGSGs as a laboratory model of cultural evolution of rhythmic equitone patterns referring to distinct emotions. As a result of transmission, rhythms developed a universal property of music structure, namely temporal regularity (or isochronicity). In the second experiment, I anchored this result with neural predictors. I showed that neural information processing capabilities of individuals, as measured with the MMN on day 1, can predict learning, transmission, and regularization of rhythmic patterns in signaling games on day 2. In agreement with study 2, I observe that MMN brain timing may reflect the efficiency of sensory systems to process auditory patterns. Functional differences in those systems, across individuals, may produce a different sensitivity to pressures for regularities in the cultural system. Finally, I argue that neural variability can be an important source of variability of cultural traits in a population. My work is the first to systematically describe the emergence of structural properties of melodic and rhythmic systems in the laboratory, using an explicit game-theoretic model of cultural transmission in which agents freely interact and exchange information. Critically, it provides the first demonstration that social learning, transmission, and cultural adaptation are constrained and driven by individual differences in the functional organization of sensory systems

Weathered Words : Formulaic Language and Verbal Art

Formulaic phraseology presents the epitome of words worn and weathered by trial and the tests of time. Scholarship on weathered words is exceptionally diverse and interdisciplinary. This volume focuses on verbal art, which makes Oral-Formulaic Theory (OFT) a major point of reference. Yet weathered words are but a part of OFT, and OFT is only a part of scholarship on weathered words. Each of the eighteen essays gathered here brings particular aspects of formulaic language into focus. No volume on such a diverse topic can be all-encompassing, but the essays highlight aspects of the phenomenon that may be eclipsed elsewhere: they diverge not only in style, but sometimes even in how they choose to define “formula.” As such, they offer overlapping frames that complement one another both in their convergences and their contrasts. While they view formulaicity from multifarious angles, they unite in a Picasso of perspectives on which the reader can reflect and draw insight.Peer reviewe

Computational analysis of world music corpora

PhDThe comparison of world music cultures has been considered in musicological research since the end of the 19th century. Traditional methods from the field of comparative musicology typically involve the process of manual music annotation. While this provides expert knowledge, the manual input is timeconsuming and limits the potential for large-scale research. This thesis considers computational methods for the analysis and comparison of world music cultures. In particular, Music Information Retrieval (MIR) tools are developed for processing sound recordings, and data mining methods are considered to study similarity relationships in world music corpora. MIR tools have been widely used for the study of (mainly) Western music. The first part of this thesis focuses on assessing the suitability of audio descriptors for the study of similarity in world music corpora. An evaluation strategy is designed to capture challenges in the automatic processing of world music recordings and different state-of-the-art descriptors are assessed. Following this evaluation, three approaches to audio feature extraction are considered, each addressing a different research question. First, a study of singing style similarity is presented. Singing is one of the most common forms of musical expression and it has played an important role in the oral transmission of world music. Hand-designed pitch descriptors are used to model aspects of the singing voice and clustering methods reveal singing style similarities in world music. Second, a study on music dissimilarity is performed. While musical exchange is evident in the history of world music it might be possible that some music cultures have resisted external musical influence. Low-level audio features are combined with machine learning methods to find music examples that stand out in a world music corpus, and geographical patterns are examined. The last study models music similarity using descriptors learned automatically with deep neural networks. It focuses on identifying music examples that appear to be similar in their audio content but share no (obvious) geographical or cultural links in their metadata. Unexpected similarities modelled in this way uncover possible hidden links between world music cultures. This research investigates whether automatic computational analysis can uncover meaningful similarities between recordings of world music. Applications derive musicological insights from one of the largest world music corpora studied so far. Computational analysis as proposed in this thesis advances the state-of-the-art in the study of world music and expands the knowledge and understanding of musical exchange in the world.Queen Mary Principal’s research studentship

The Metaphysics of Improvisation

In The Metaphysics of Improvisation, I criticize wrongheaded metaphysical views of, and theories about, improvisation, and put forward a cogent metaphysical theory of improvisation, which includes action theory, an analysis of the relevant genetic and aesthetic properties, and ontology (work-hood). The dissertation has two Parts. Part I is a survey of the history of many improvisational practices, and of the concept of improvisation. Here I delineate, sketch, and sort out the often vague boundaries between improvising and non-improvising within many art forms and genres, including music, dance, theatre, motion pictures, painting, and literature. In addition, I discuss the concept of non-artistic improvisation in various contexts. I attempt to portray an accurate picture of how improvisation functions, or does not function, in various art forms and genres. Part II addresses metaphysical issues in, and problems and questions of, improvisation in the arts. I argue that that continuum and genus-species models are the most cogent ways to understand the action-types of improvising and composing and their relations. I demonstrate that these models are substantiated by an informed investigation and phenomenology of improvisational practice, action theory conceptual analysis, cognitive neuroscience studies and experiments, cognitive psychology studies and models, and some theories of creativity. In addition, I provide a constraint based taxonomy for classifying improvisations that is compatible with, and supports, the continuum model. Next, I address epistemological and ontological issues involving the genetic properties of improvisations, and the properties improvisatory, and as if improvised. Finally, I show that arguments against treating, or classifying, improvisations as works are weak or erroneous, and by focusing on music, I provide a correct ontological theory of work-hood for artistic improvisations