Analysing multi-person timing in music and movement : event based methods

Accurate timing of movement in the hundreds of milliseconds range is a hallmark of human activities such as music and dance. Its study requires accurate measurement of the times of events (often called responses) based on the movement or acoustic record. This chapter provides a comprehensive over - view of methods developed to capture, process, analyse, and model individual and group timing [...] This chapter is structured in five main sections, as follows. We start with a review of data capture methods, working, in turn, through a low cost system to research simple tapping, complex movements, use of video, inertial measurement units, and dedicated sensorimotor synchronisation software. This is followed by a section on music performance, which includes topics on the selection of music materials, sound recording, and system latency. The identification of events in the data stream can be challenging and this topic is treated in the next section, first for movement then for music. Finally, we cover methods of analysis, including alignment of the channels, computation of between channel asynchrony errors and modelling of the data set

Automatied detection of single- and multi-note ornaments in Irish traditional flute playing

[TODO] Add abstract here

Note, Cut and Strike Detection for Traditional Irish Flute Recordings

This paper addresses the topic of note, cut and strike detection inIrish traditional music (ITM). In order to do this we first evaluate state of the art onset detection methods for identifying note boundaries. Our method utilises the results from manually and automatically segmented flute recordings. We then demonstrate how this information may be utilised for the detection of notes and single note articulations idiomatic of this genre for the purposes of player style identification. Results for manually annotated onsets achieve 86%, 70% and 74% accuracies for note, cut and strike classification respectively. Results for automatically segmented recordings are considerably, lower therefore we perform an analysis of the onset detection results per event class to establish which musical patterns contain the most errors

Onset Detection for String Instruments Using Bidirectional Temporal and Convolutional Recurrent Networks

Recent work in note onset detection has centered on deep learning models such as recurrent neural networks (RNN), convolutional neural networks (CNN) and more recently temporal convolutional networks (TCN), which achieve high evaluation accuracies for onsets characterized by clear, well-defined transients, as found in percussive instruments. However, onsets with less transient presence, as found in string instrument recordings, still pose a relatively difficult challenge for state-of-the-art algorithms. This challenge is further exacerbated by a paucity of string instrument data containing expert annotations. In this paper, we propose two new models for onset detection using bidirectional temporal and recurrent convolutional networks, which generalise to polyphonic signals and string instruments. We perform evaluations of the proposed methods alongside state-of-the-art algorithms for onset detection on a benchmark dataset from the MIR community, as well as on a test set from a newly proposed dataset of string instrument recordings with note onset annotations, comprising approximately 40 minutes and over 8,000 annotated onsets with varied expressive playing styles. The results demonstrate the effectiveness of both presented models, as they outperform the state-of-the-art algorithms on string recordings while maintaining comparative performance on other types of music

A supervised classification approach for note tracking in polyphonic piano transcription

In the field of Automatic Music Transcription, note tracking systems constitute a key process in the overall success of the task as they compute the expected note-level abstraction out of a frame-based pitch activation representation. Despite its relevance, note tracking is most commonly performed using a set of hand-crafted rules adjusted in a manual fashion for the data at issue. In this regard, the present work introduces an approach based on machine learning, and more precisely supervised classification, that aims at automatically inferring such policies for the case of piano music. The idea is to segment each pitch band of a frame-based pitch activation into single instances which are subsequently classified as active or non-active note events. Results using a comprehensive set of supervised classification strategies on the MAPS piano data-set report its competitiveness against other commonly considered strategies for note tracking as well as an improvement of more than +10% in terms of F-measure when compared to the baseline considered for both frame-level and note-level evaluations.This research work is partially supported by Universidad de Alicante through the FPU program [UAFPU2014–5883] and the Spanish Ministerio de Economía y Competitividad through project TIMuL [No. TIN2013–48152–C2–1–R, supported by EU FEDER funds]. EB is supported by a UK RAEng Research Fellowship [grant number RF/128]

Interactive user correction of automatically detected onsets: approach and evaluation

Onset detection still has room for improvement, especially when dealing with polyphonic music signals. For certain purposes in which the correctness of the result is a must, user intervention is hence required to correct the mistakes performed by the detection algorithm. In such interactive paradigm, the exactitude of the detection can be guaranteed at the expense of user’s work, being the effort required to accomplish the task, the value that has to be both quantified and reduced. The present work studies the idea of interactive onset detection and proposes a methodology for assessing the user’s workload, as well as a set of interactive schemes for reducing such workload when carrying out this detection task. Results show that the evaluation strategy proposed is able to quantitatively assess the invested user effort. Also, the presented interactive schemes significantly facilitate the correction task compared with the manual annotation.This research work is partially supported by the Vicerrectorado de Investigación, Desarrollo e Innovación de la Universidad de Alicante through FPU program (UAFPU2014–5883) and the Spanish Ministerio de Economía y Competitividad through the TIMuL project (No. TIN2013–48152–C2–1–R, supported by EU FEDER funds)

A novel technique for high-resolution frequency discriminators and their application to pitch and onset detection in empirical musicology

This thesis presents and evaluates software for simultaneous, high-resolution time-frequency discrimination. Whilst this is a problem that arises in many areas of engineering, the software here is developed to assist musicological investigations. In order to analyse musical performances, we must first know what is happening and when; that is, at what time each note begins to sound (the note onset) and what frequencies are present (the pitch). The work presented here focusses on onset detection, although the representation of data used for this task could also be used to track the pitch. A potential method of determining pitch on a sample-to-sample basis is given in the final chapter. Extant software for onset detection uses standard signal processing techniques to search for changes in features like the spectrum or phase. These methods struggle somewhat, as they are constrained by the uncertainty principle, which states that, as time resolution is increased, frequency resolution must decrease and vice versa. However, we can hear changes in frequency to a far greater time resolution than the uncertainty principle would suggest is possible. There is an active process in the inner ear which adds energy and enables this perceptual acuity. The mathematical expression which describes this system is known as the Hopf bifurcation. By building a bank of tuned resonators in software, each of which operates at a Hopf bifurcation, and driving it with audio, changes in frequency can be detected in times that defy the uncertainty relation, as we are not seeking to directly measure the time-frequency features of a system, rather it is used to drive a system. Time and frequency information is then available from the internal state variables of the system. The characteristics of this bank of resonators - called a 'DetectorBank' - are investigated thoroughly. The bandwidth of each resonator ('detector') can be as narrow as 0.922Hz and the system bandwidth is extended to the Nyquist frequency. A nonlinear system may be expected to respond poorly when presented with multiple simultaneous input frequencies; however, the DetectorBank performs well under these circumstances. The data generated by the DetectorBank is then analysed by an OnsetDetector. Both the development and testing of this OnsetDetector are detailed. It is tested using a repository of recordings of individual notes played on a variety of instruments, with promising results. These results are discussed, problems with the current implementation are identified and potential solutions presented. This OnsetDetector can then be combined with a PitchTracker to create a NoteDetector, capable of detecting not only a single note onset time and pitch, but information about changes that occur within a note. Musical notes are not static entities: they contain much variation. Both the performer's intonation and the characteristics of the instrument itself have an effect on the frequency present, as well as features like vibrato. Knowledge of these frequency components, and how they appear or disappear over the course of the note, is valuable information and the software presented here enables the collection of this data

Analysis and resynthesis of polyphonic music

This thesis examines applications of Digital Signal Processing to the analysis, transformation, and resynthesis of musical audio. First I give an overview of the human perception of music. I then examine in detail the requirements for a system that can analyse, transcribe, process, and resynthesise monaural polyphonic music. I then describe and compare the possible hardware and software platforms. After this I describe a prototype hybrid system that attempts to carry out these tasks using a method based on additive synthesis. Next I present results from its application to a variety of musical examples, and critically assess its performance and limitations. I then address these issues in the design of a second system based on Gabor wavelets. I conclude by summarising the research and outlining suggestions for future developments