Signal processing methods for beat tracking, music segmentation, and audio retrieval

The goal of music information retrieval (MIR) is to develop novel strategies and techniques for organizing, exploring, accessing, and understanding music data in an efficient manner. The conversion of waveform-based audio data into semantically meaningful feature representations by the use of digital signal processing techniques is at the center of MIR and constitutes a difficult field of research because of the complexity and diversity of music signals. In this thesis, we introduce novel signal processing methods that allow for extracting musically meaningful information from audio signals. As main strategy, we exploit musical knowledge about the signals\u27 properties to derive feature representations that show a significant degree of robustness against musical variations but still exhibit a high musical expressiveness. We apply this general strategy to three different areas of MIR: Firstly, we introduce novel techniques for extracting tempo and beat information, where we particularly consider challenging music with changing tempo and soft note onsets. Secondly, we present novel algorithms for the automated segmentation and analysis of folk song field recordings, where one has to cope with significant fluctuations in intonation and tempo as well as recording artifacts. Thirdly, we explore a cross-version approach to content-based music retrieval based on the query-by-example paradigm. In all three areas, we focus on application scenarios where strong musical variations make the extraction of musically meaningful information a challenging task.Ziel der automatisierten Musikverarbeitung ist die Entwicklung neuer Strategien und Techniken zur effizienten Organisation großer Musiksammlungen. Ein Schwerpunkt liegt in der Anwendung von Methoden der digitalen Signalverarbeitung zur Umwandlung von Audiosignalen in musikalisch aussagekräftige Merkmalsdarstellungen. Große Herausforderungen bei dieser Aufgabe ergeben sich aus der Komplexität und Vielschichtigkeit der Musiksignale. In dieser Arbeit werden neuartige Methoden vorgestellt, mit deren Hilfe musikalisch interpretierbare Information aus Musiksignalen extrahiert werden kann. Hierbei besteht eine grundlegende Strategie in der konsequenten Ausnutzung musikalischen Vorwissens, um Merkmalsdarstellungen abzuleiten die zum einen ein hohes Maß an Robustheit gegenüber musikalischen Variationen und zum anderen eine hohe musikalische Ausdruckskraft besitzen. Dieses Prinzip wenden wir auf drei verschieden Aufgabenstellungen an: Erstens stellen wir neuartige Ansätze zur Extraktion von Tempo- und Beat-Information aus Audiosignalen vor, die insbesondere auf anspruchsvolle Szenarien mit wechselnden Tempo und weichen Notenanfängen angewendet werden. Zweitens tragen wir mit neuartigen Algorithmen zur Segmentierung und Analyse von Feldaufnahmen von Volksliedern unter Vorliegen großer Intonationsschwankungen bei. Drittens entwickeln wir effiziente Verfahren zur inhaltsbasierten Suche in großen Datenbeständen mit dem Ziel, verschiedene Interpretationen eines Musikstückes zu detektieren. In allen betrachteten Szenarien richten wir unser Augenmerk insbesondere auf die Fälle in denen auf Grund erheblicher musikalischer Variationen die Extraktion musikalisch aussagekräftiger Informationen eine große Herausforderung darstellt

Compositions Combining Acoustic, Electro-Acoustic And Synthetic Instruments For Modern Jazz Performance

Compositions Combining Acoustic, Electro-acoustic and Synthetic Instruments for Modern Jazz Performance presents a series of seven compositions by vibraphonist and composer David Kemp. The seven works primarily explore combining acoustic (trumpet and drums), electro-acoustic (electric guitar, bass guitar and pickup equipped vibraphone) and synthetic (electronically created synthesizer patches on a Roland XV5050 Sound Module) instruments, mixing musical styles, using rhythm and duration as a governing force in composition, using an extensive harmonic palette and incorporating technology. In this document, the scores are presented in full in a Portfolio Volume, accompanied by recorded performances in audio and visual formats and written analyses of the compositions. Included is a discussion of archetypal composers, similar stylistic traits of their music to mine, and conclusions drawn from the project. The document is completed by a bibliography and discography

Ontology of music performance variation

Performance variation in rhythm determines the extent that humans perceive and feel the effect of rhythmic pulsation and music in general. In many cases, these rhythmic variations can be linked to percussive performance. Such percussive performance variations are often absent in current percussive rhythmic models. The purpose of this thesis is to present an interactive computer model, called the PD-103, that simulates the micro-variations in human percussive performance. This thesis makes three main contributions to existing knowledge: firstly, by formalising a new method for modelling percussive performance; secondly, by developing a new compositional software tool called the PD-103 that models human percussive performance, and finally, by creating a portfolio of different musical styles to demonstrate the capabilities of the software. A large database of recorded samples are classified into zones based upon the vibrational characteristics of the instruments, to model timbral variation in human percussive performance. The degree of timbral variation is governed by principles of biomechanics and human percussive performance. A fuzzy logic algorithm is applied to analyse current and first-order sample selection in order to formulate an ontological description of music performance variation. Asynchrony values were extracted from recorded performances of three different performance skill levels to create \timing fingerprints" which characterise unique features to each percussionist. The PD-103 uses real performance timing data to determine asynchrony values for each synthesised note. The spectral content of the sample database forms a three-dimensional loudness/timbre space, intersecting instrumental behaviour with music composition. The reparameterisation of the sample database, following the analysis of loudness, spectral flatness, and spectral centroid, provides an opportunity to explore the timbral variations inherent in percussion instruments, to creatively explore dimensions of timbre. The PD-103 was used to create a music portfolio exploring different rhythmic possibilities with a focus on meso-periodic rhythms common to parts of West Africa, jazz drumming, and electroacoustic music. The portfolio also includes new timbral percussive works based on spectral features and demonstrates the central aim of this thesis, which is the creation of a new compositional software tool that integrates human percussive performance and subsequently extends this model to different genres of music

Investigating the iso principle: the effect of musical tempo manipulation on arousal shift

2020 Fall.Includes bibliographical references.The iso principle is a well-known concept in music therapy practice wherein a clinician meets a client at a current body state with a musical element, then moves them to a new body state by modulating the musical element. However, few scholars agree on what bodily states and musical elements define the iso principle, which limits music therapy clinicians' targeted application of the concept. Further, it appears there have been no studies objectively addressing physiologic change during the iso principle. The purpose of this study was to investigate arousal shift during iso principle-informed tempo change in a musical stimulus. Arousal was measured via physiological responses (galvanic skin response [GSR]) and self-perception (self-assessment manikin [SAM]). Participants' (n = 9) took part in a randomized block design with control in which they completed a mindfulness-based intervention before listening to one of three five-minute auditory stimuli: 1) an iso principle-informed song, 2) a compensation principle-informed song, and 3) a spoken short story. GSR data from participants did not show statistically significant differences between the iso principle and compensation principle, but did show significant differences between musical conditions and speech. While the music was designed to increase arousal using the iso principle, overall there was a reduction in arousal levels over the experimental period. Participants' self-ratings of their arousal shifts (SAM scores of arousal) showed a perceived increase in arousal during all conditions. Limitations, clinical implications, and future directions are discussed

Computational methods for percussion music analysis : the afro-uruguayan candombe drumming as a case study

Most of the research conducted on information technologies applied to music has been largely limited to a few mainstream styles of the so-called `Western' music. The resulting tools often do not generalize properly or cannot be easily extended to other music traditions. So, culture-specific approaches have been recently proposed as a way to build richer and more general computational models for music. This thesis work aims at contributing to the computer-aided study of rhythm, with the focus on percussion music and in the search of appropriate solutions from a culture specifc perspective by considering the Afro-Uruguayan candombe drumming as a case study. This is mainly motivated by its challenging rhythmic characteristics, troublesome for most of the existing analysis methods. In this way, it attempts to push ahead the boundaries of current music technologies. The thesis o ers an overview of the historical, social and cultural context in which candombe drumming is embedded, along with a description of the rhythm. One of the specific contributions of the thesis is the creation of annotated datasets of candombe drumming suitable for computational rhythm analysis. Performances were purposely recorded, and received annotations of metrical information, location of onsets, and sections. A dataset of annotated recordings for beat and downbeat tracking was publicly released, and an audio-visual dataset of performances was obtained, which serves both documentary and research purposes. Part of the dissertation focused on the discovery and analysis of rhythmic patterns from audio recordings. A representation in the form of a map of rhythmic patterns based on spectral features was devised. The type of analyses that can be conducted with the proposed methods is illustrated with some experiments. The dissertation also systematically approached (to the best of our knowledge, for the first time) the study and characterization of the micro-rhythmical properties of candombe drumming. The ndings suggest that micro-timing is a structural component of the rhythm, producing a sort of characteristic "swing". The rest of the dissertation was devoted to the automatic inference and tracking of the metric structure from audio recordings. A supervised Bayesian scheme for rhythmic pattern tracking was proposed, of which a software implementation was publicly released. The results give additional evidence of the generalizability of the Bayesian approach to complex rhythms from diferent music traditions. Finally, the downbeat detection task was formulated as a data compression problem. This resulted in a novel method that proved to be e ective for a large part of the dataset and opens up some interesting threads for future research.La mayoría de la investigación realizada en tecnologías de la información aplicadas a la música se ha limitado en gran medida a algunos estilos particulares de la así llamada música `occidental'. Las herramientas resultantes a menudo no generalizan adecuadamente o no se pueden extender fácilmente a otras tradiciones musicales. Por lo tanto, recientemente se han propuesto enfoques culturalmente específicos como forma de construir modelos computacionales más ricos y más generales. Esta tesis tiene como objetivo contribuir al estudio del ritmo asistido por computadora, desde una perspectiva cultural específica, considerando el candombe Afro-Uruguayo como caso de estudio. Esto está motivado principalmente por sus características rítmicas, problemáticas para la mayoría de los métodos de análisis existentes. Así , intenta superar los límites actuales de estas tecnologías. La tesis ofrece una visión general del contexto histórico, social y cultural en el que el candombe está integrado, junto con una descripción de su ritmo. Una de las contribuciones específicas de la tesis es la creación de conjuntos de datos adecuados para el análisis computacional del ritmo. Se llevaron adelante sesiones de grabación y se generaron anotaciones de información métrica, ubicación de eventos y secciones. Se disponibilizó públicamente un conjunto de grabaciones anotadas para el seguimiento de pulso e inicio de compás, y se generó un registro audiovisual que sirve tanto para fines documentales como de investigación. Parte de la tesis se centró en descubrir y analizar patrones rítmicos a partir de grabaciones de audio. Se diseñó una representación en forma de mapa de patrones rítmicos basada en características espectrales. El tipo de análisis que se puede realizar con los métodos propuestos se ilustra con algunos experimentos. La tesis también abordó de forma sistemática (y por primera vez) el estudio y la caracterización de las propiedades micro rítmicas del candombe. Los resultados sugieren que las micro desviaciones temporales son un componente estructural del ritmo, dando lugar a una especie de "swing" característico. El resto de la tesis se dedicó a la inferencia automática de la estructura métrica a partir de grabaciones de audio. Se propuso un esquema Bayesiano supervisado para el seguimiento de patrones rítmicos, del cual se disponibilizó públicamente una implementación de software. Los resultados dan evidencia adicional de la capacidad de generalización del enfoque Bayesiano a ritmos complejos. Por último, la detección de inicio de compás se formuló como un problema de compresión de datos. Esto resultó en un método novedoso que demostró ser efectivo para una buena parte de los datos y abre varias líneas de investigación

Measuring musical interaction: analysing communication in embodied musical behaviour

This thesis addresses the ubiquity and necessity of embodied interaction to musical activity, using video analysis to observe communication in musical events. Through the specific study of classical North Indian instrumental duo performance, the thesis examines how processes of social interaction may inform human musical activity, using a combined methodology of ethnographic study and quantitative data analysis of original video-recordings. Proposing a pragmatic approach to the study of the meaningful nature of musical events, the thesis keeps sight of the generative context of the human body in social interaction, and offers a model of musical communication that privileges nonlinguistic, socially co-regulative elements in its account of human musical interaction. The socially meaningful nature of the behaviour-in-time of the musicians included in the study is investigated by means of a novel methodology. This combines the qualitative exploration of emic concepts related to the practice of North Indian classical music with an empirical analysis of video data, based on a cognitive ethological framework. The thesis draws on current notions of embodied cognition and contributes to the growing corpus of musicological literature emphasising the embodied and social nature of musical communication. The results of this exploratory study suggest that both social-interaction and music-structural factors contribute to the organisation of the musicians' communicative behaviours and that, to a certain extent, these organisational factors can be separated in analysis

The Presence of Groove in Online Songwriting Projects

Collaboration for groups with members who are disconnected by geography or time is convenient for many reasons, but remains a challenge due to time zone differences, network congestion, and the attenuation of nonverbal communication cues. Virtual collaborators engaging in creative work often deal with these challenges, even more so when tasked with expressing their emotions to distant partners. This study seeks to determine the social factors and tools that impact the quality of an online creative collaboration. Members of the Kompoz.com music composition community were surveyed to solicit projects that had the potential to be optimal collaborations. Judges listened to these songs and measured how much each song prompted them to move. This measure, called groove, was used as an indication of a successful collaboration. Judges assisted in selecting one case that was an exemplar of groove, and another that urged them to move much less, to stand as an exemplar of diminished groove. The comparative case method was used to compare and contrast the tools, social practices, and skills employed in each project, and offers guidelines for the design of and participation in online creative communities

Eye-tracking Film Music

Film music scholars, composers, directors and audiences have always implicitly believed that music can help determine the focus of an audience’s visual attention, but researchers have not as yet been able to prove this empirically. Eye-tracking research—the process of measuring either the point of gaze or the motion of the eyes—has grown exponentially in recent years. This paper reports on a foundational, empirical eye-tracking study that examined the effects of contextual musical attributes on visual attention, emotion and user experience during exploration tasks in moving images. Our results show that music is able to direct how we see by switching attention to target foci more quickly as well as lengthening fixations, and that music can also encourage greater exploration of visual scenes outside targets. Our work contributes the first step in understanding how music shapes visual attention using eye-tracking techniques. We encourage wider adoption of this approach which has the potential to enhance understanding of the complex processes of audiovisual perception in action

Grey Areas: Songs of Memory, Imagination, Intellect, and Death for Baritone Voice and Mixed Instrumental Octet

The central element of this thesis is Grey Areas, a newly-composed dramatic piece in four movements for baritone and chamber ensemble. The work is highly personal in both its subject matter and its musical execution; it also contains a unique movement order that changes depending on the age of the baritone. Along with the written portion of this document and full score, I completed a studio recording of the entire composition as part of the project. Composers (by necessity) are more frequently functioning as their own recording and mixing engineers for their own works. I have been able to gather a variety of skills related to the recording arts and sought out opportunities to employ them in this project. In order to meaningfully add to the greater body of research, I describe both the creative aspects of my compositional process and the technical aspects of the recording process

Redesigning a Performance Practice: Synergising Woodwind Improvisation with Bespoke Software Technology.

This research examines how the designing of a new performance practice based on the incorporation of custom digital signal processing software impacts on solo improvised woodwind performance. Through the development of bespoke software, I investigate how these new technologies can be integrated into solo woodwind performance practice. This research presents a new improvised music practice as well as a suite of new software tools and performance techniques. Through a workshop and performance-­‐based research process, a suite of software processors are developed which respond, and are complementary, to a personalised style of improvised performance. This electronic augmentation of the woodwind instrument (clarinet, bass clarinet, alto saxophone and xaphoon) is tested over the course of thirty solo improvised performances. These performances are documented as audio files and analysed using methods derived from electroacoustic music practice. This research represents an important development in the emerging field of improvised music performance engaging with new digital technologies. The research is practice-­‐led from the viewpoint of an experienced performer and tested in real-­‐world situations, resulting in a useful research outputs embedded in the peer community. Examining the history of live electronic performance practice, this research situates itself within the field of expert performers who use digital processing in free improvisation contexts. A critical understanding of the processes involved allows this researcher to design a new performance practice more effectively. While research necessarily draws on my own performance practice, the knowledge generated will have broad relevance in the field and much of this work is applicable to non-­‐woodwind instrumentalists and singers. The research outputs include freely distributable software created during this project