Linking Sheet Music and Audio - Challenges and New Approaches

Score and audio files are the two most important ways to represent, convey, record, store, and experience music. While score describes a piece of music on an abstract level using symbols such as notes, keys, and measures, audio files allow for reproducing a specific acoustic realization of the piece. Each of these representations reflects different facets of music yielding insights into aspects ranging from structural elements (e.g., motives, themes, musical form) to specific performance aspects (e.g., artistic shaping, sound). Therefore, the simultaneous access to score and audio representations is of great importance. In this paper, we address the problem of automatically generating musically relevant linking structures between the various data sources that are available for a given piece of music. In particular, we discuss the task of sheet music-audio synchronization with the aim to link regions in images of scanned scores to musically corresponding sections in an audio recording of the same piece. Such linking structures form the basis for novel interfaces that allow users to access and explore multimodal sources of music within a single framework. As our main contributions, we give an overview of the state-of-the-art for this kind of synchronization task, we present some novel approaches, and indicate future research directions. In particular, we address problems that arise in the presence of structural differences and discuss challenges when applying optical music recognition to complex orchestral scores. Finally, potential applications of the synchronization results are presented

Breathing New Life into Old Forms: Collaborative Processes Supporting Songwriting and Improvisation

The primary aim of this practice-based research was to reinvigorate my artistic practice in composition, and in so doing provide a useful model for vocalists and singer-songwriters in the contemporary jazz genre. These aims are achieved by identifying the dialectics – the collaborative inputs – in composing, lyric writing, rehearsing and recording. The thesis provides a narrative of processes undertaken in the creative practice journey involved in the production of the albums Get Out of Town (2012), Mandarin Skyline (2013) and Weave (2016). Leading Australian musicians and composers were selected for the writing and recording processes. Through a detailed examination of their input and contribution to the music, along with the input of non-musician participants, an enhanced understanding of the musical and wider interactions related to collaborative processes is revealed. The collaborative processes are outlined song by song and analysed in every stage of music creation, including preproduction discussions, communications with selected co-writers, musicians’ contributions to arrangement and composition, informal discussions related to “best” practice, and negotiation regarding stylistic interpretations pertaining to genre, harmonic movement and improvisation. Improvisation emerges as a key factor in not only the personal creative compositional process, but also in the collaborative process. The concept of collaboration was stretched to encompass personal creative processes, informed as they are by issues of stylistic identity, inspirational figures and the creative milieu within which musicians hone their skills. This process shone light on strategies within the collaborative spectrum that promoted the extension and development of my songwriting and improvisational practices

Composing for improvisers : information flow, collaborative composition and individual freedom in large ensembles

Rooted in the lineages of European ‘free’ improvisation, jazz, the New York ‘Downtown’ scene, Chicago’s AACM, and various ‘indeterminate’ approaches, this research project deals with the creation and evaluation of a portfolio of compositions, each of which explores particular facets of the open-ended and interpretable links between composer, performer and piece, exploiting interactive and real-time elements inherent in collective music-making. The compositional models I have developed here focus specifically on large groups of improvisers, and aim to function malleably in ways that encourage collaboration and prioritise the freedoms, personal voices and creative powers of all involved, whilst maintaining a degree of compositional integrity.Following an iterative methodology of experimentation, performance and reflection, this portfolio evolves, via several pieces that each focus on particular criteria, towards 'Micromotives', a collection of pieces designed to be collectively constructed in real-time by a large improvising ensemble. 'Micromotives' provides a modus operandi that is largely consistent with that of free improvisation, bypassing fixed authority figures, timelines and personnel groupings that are common yet arguably problematic staples of many comparable approaches that have emerged since the 1960s (Butcher, 2011; Stenström, 2009). Instead, performer obligations are removed so that predetermined materials can be referred to as and when they are desired. Any player can try to instigate collective action at any time, and is able to communicate detailed information in real-time by way of a series of bespoke hand signs. Underpinned by an ethos of invitation, as opposed to direction, 'Micromotives' allows larger numbers of improvisers to maintain high levels of individual freedom whilst simultaneously enabling constituent pieces to be distinguishable from one another, encouraging modes of collective synchronicity that are virtually unheard in large ensemble free improvisation

Human-Computer Music Performance: From Synchronized Accompaniment to Musical Partner

Live music performance with computers has motivated many research projects in science, engineering, and the arts. In spite of decades of work, it is surprising that there is not more technology for, and a better understanding of the computer as music performer. We review the development of techniques for live music performance and outline our efforts to establish a new direction, Human-Computer Music Performance (HCMP), as a framework for a variety of coordinated studies. Our work in this area spans performance analysis, synchronization techniques, and interactive performance systems. Our goal is to enable musicians to ncorporate computers into performances easily and effectively through a better understanding of requirements, new techniques, and practical, performance-worthy implementations. We conclude with directions for future work

Real-Time Audio-to-Score Alignment of Music Performances Containing Errors and Arbitrary Repeats and Skips

This paper discusses real-time alignment of audio signals of music performance to the corresponding score (a.k.a. score following) which can handle tempo changes, errors and arbitrary repeats and/or skips (repeats/skips) in performances. This type of score following is particularly useful in automatic accompaniment for practices and rehearsals, where errors and repeats/skips are often made. Simple extensions of the algorithms previously proposed in the literature are not applicable in these situations for scores of practical length due to the problem of large computational complexity. To cope with this problem, we present two hidden Markov models of monophonic performance with errors and arbitrary repeats/skips, and derive efficient score-following algorithms with an assumption that the prior probability distributions of score positions before and after repeats/skips are independent from each other. We confirmed real-time operation of the algorithms with music scores of practical length (around 10000 notes) on a modern laptop and their tracking ability to the input performance within 0.7 s on average after repeats/skips in clarinet performance data. Further improvements and extension for polyphonic signals are also discussed.Comment: 12 pages, 8 figures, version accepted in IEEE/ACM Transactions on Audio, Speech, and Language Processin

Developing a Polyrhythmic Idiolect

This practice-based multi-media study sets out to reveal how procedural methodologies effect transformative change in a polyrhythmic drum-set idiolect, premised on the idea that archetypal variants and phraseological patterning constituting my musical “voice” are, primarily, results of a procedural mind rather than aggregations of replicative ideas acquired from elsewhere. The thesis accordingly sets out a detailed participant-observer study designed to reveal methodological processes and outcomes pertaining to the cultivation of a unique sonic identity. In revealing how structural-organisational processes can evolve personalised ways of manipulating rhythm, this research offers new analytical tools for understanding what improvising drummers do. Two important aims of the study are (a) to effect and document transformative change in my drum-set language through the application of improvisational methodologies, and (b) to reveal these procedures in operation from a participant-observer perspective, thereby showing how sonic identity can be individuated through personal agency and decision-making/selection processes operating within constraints. Original generative methodologies for hybridizing vocabulary and propagating unique archetypal variants – namely, the Iterative Loop Cycle and Transitional Synthesis - are central to this project, which targets six developmental areas: Suspended Primary Pulsation, Densities, Pulse Streaming, Transposing Rhythm, Isochronous Asymmetry and Mixed Rates

Playing the Field: An Australian Case Study of Student Popular Musicians’ Informal Learning in Senior Secondary Classroom Music Education

This thesis explores the field of classroom music education in order to foreground the learning experiences of student popular musicians. The Australian, New South Wales (NSW) context is well able to contribute to the global discussion that is underway in popular music education, as senior secondary curriculum here acknowledges the inclusion of students with “informal learning” backgrounds. Over the past decade, research globally has sought to qualify the nature of informal learning, and develop classroom pedagogies relevant to the study of popular music. Utilising these as a starting point, this thesis examines the relationship between these students’ informal learning and the dynamics of the formal classroom. Research was undertaken on three levels: historical, through an investigation of curriculum documents, reforms and matriculation trends; empirical, through a classroom research project exploring a range of informal and formal tasks; and theoretical, via an overarching explanatory tool known as Legitimation Code Theory (LCT). The research revealed that NSW curricular pathways and classroom pedagogies employed result in the maintenance of a ‘code’ distinction: cultivating traditional knowledge and skills for WAM, but not providing adequate knowledge-building opportunities for student popular musicians. Considering the range of cross-genre music-making evident in the study, and the delineation of a spectrum of knowledge and skills spanning code distinctions, findings highlight the need for a re-evaluation of NSW curriculum and pedagogy, with implications beyond the specificities of the case. A recognition and theorisation of the relationship between different forms of musical knowledge across the informal-formal range is believed to be key to providing both socially relevant, and epistemically challenging classroom music education inclusive of all students in the future