Computational Modelling and Analysis of Vibrato and Portamento in Expressive Music Performance

PhD, 148ppVibrato and portamento constitute two expressive devices involving continuous pitch modulation and is widely employed in string, voice, wind music instrument performance. Automatic extraction and analysis of such expressive features form some of the most important aspects of music performance research and represents an under-explored area in music information retrieval. This thesis aims to provide computational and scalable solutions for the automatic extraction and analysis of performed vibratos and portamenti. Applications of the technologies include music learning, musicological analysis, music information retrieval (summarisation, similarity assessment), and music expression synthesis. To automatically detect vibratos and estimate their parameters, we propose a novel method based on the Filter Diagonalisation Method (FDM). The FDM remains robust over short time frames, allowing frame sizes to be set at values small enough to accurately identify local vibrato characteristics and pinpoint vibrato boundaries. For the determining of vibrato presence, we test two alternate decision mechanisms—the Decision Tree and Bayes’ Rule. The FDM systems are compared to state-of-the-art techniques and obtains the best results. The FDM’s vibrato rate accuracies are above 92.5%, and the vibrato extent accuracies are about 85%. We use the Hidden Markov Model (HMM) with Gaussian Mixture Model (GMM) to detect portamento existence. Upon extracting the portamenti, we propose a Logistic Model for describing portamento parameters. The Logistic Model has the lowest root mean squared error and the highest adjusted Rsquared value comparing to regression models employing Polynomial and Gaussian functions, and the Fourier Series. The vibrato and portamento detection and analysis methods are implemented in AVA, an interactive tool for automated detection, analysis, and visualisation of vibrato and portamento. Using the system, we perform crosscultural analyses of vibrato and portamento differences between erhu and violin performance styles, and between typical male or female roles in Beijing opera singing

Adaptive Scattering Transforms for Playing Technique Recognition

Playing techniques contain distinctive information about musical expressivity and interpretation. Yet, current research in music signal analysis suffers from a scarcity of computational models for playing techniques, especially in the context of live performance. To address this problem, our paper develops a general framework for playing technique recognition. We propose the adaptive scattering transform, which refers to any scattering transform that includes a stage of data-driven dimensionality reduction over at least one of its wavelet variables, for representing playing techniques. Two adaptive scattering features are presented: frequency-adaptive scattering and direction-adaptive scattering. We analyse seven playing techniques: vibrato, tremolo, trill, flutter-tongue, acciaccatura, portamento, and glissando. To evaluate the proposed methodology, we create a new dataset containing full-length Chinese bamboo flute performances (CBFdataset) with expert playing technique annotations. Once trained on the proposed scattering representations, a support vector classifier achieves state-of-the-art results. We provide explanatory visualisations of scattering coefficients for each technique and verify the system over three additional datasets with various instrumental and vocal techniques: VPset, SOL, and VocalSet

CCOM-HuQin: an Annotated Multimodal Chinese Fiddle Performance Dataset

HuQin is a family of traditional Chinese bowed string instruments. Playing techniques(PTs) embodied in various playing styles add abundant emotional coloring and aesthetic feelings to HuQin performance. The complex applied techniques make HuQin music a challenging source for fundamental MIR tasks such as pitch analysis, transcription and score-audio alignment. In this paper, we present a multimodal performance dataset of HuQin music that contains audio-visual recordings of 11,992 single PT clips and 57 annotated musical pieces of classical excerpts. We systematically describe the HuQin PT taxonomy based on musicological theory and practical use cases. Then we introduce the dataset creation methodology and highlight the annotation principles featuring PTs. We analyze the statistics in different aspects to demonstrate the variety of PTs played in HuQin subcategories and perform preliminary experiments to show the potential applications of the dataset in various MIR tasks and cross-cultural music studies. Finally, we propose future work to be extended on the dataset.Comment: 15 pages, 11 figure

Playing Technique Recognition by Joint Time–Frequency Scattering

Playing techniques are important expressive elements in music signals. In this paper, we propose a recognition system based on the joint time–frequency scattering transform (jTFST) for pitch evolution-based playing techniques (PETs), a group of playing techniques with monotonic pitch changes over time. The jTFST represents spectro-temporal patterns in the time–frequency domain, capturing discriminative information of PETs. As a case study, we analyse three commonly used PETs of the Chinese bamboo flute: acciacatura, portamento, and glissando, and encode their characteristics using the jTFST. To verify the proposed approach, we create a new dataset, the CBF-petsDB, containing PETs played in isolation as well as in the context of whole pieces performed and annotated by professional players. Feeding the jTFST to a machine learning classifier, we obtain F-measures of 71% for acciacatura, 59% for portamento, and 83% for glissando detection, and provide explanatory visualisations of scattering coefficients for each technique

Scattering Transform for Playing Technique Recognition

Playing techniques are expressive elements in music performances that carry important information about music expressivity and interpretation. When displaying playing techniques in the time–frequency domain, we observe that each has a distinctive spectro-temporal pattern. Based on the patterns of regularity, we group commonly-used playing techniques into two families: pitch modulation-based techniques (PMTs) and pitch evolution-based techniques (PETs). The former are periodic modulations that elaborate on stable pitches, including vibrato, tremolo, trill, and flutter-tongue; while the latter contain monotonic pitch changes, such as acciaccatura, portamento, and glissando. In this thesis, we present a general framework based on the scattering transform for playing technique recognition. We propose two variants of the scattering transform, the adaptive scattering and the direction-invariant joint scattering. The former provides highly-compact representations that are invariant to pitch transpositions for representing PMTs. The latter captures the spectro-temporal patterns exhibited by PETs. Using the proposed scattering representations as input, our recognition system achieves start-of-the-art results. We provide a formal interpretation of the role of each scattering component confirmed by explanatory visualisations. Whereas previously published datasets for playing technique analysis focused primarily on techniques recorded in isolation, we publicly release a new dataset to evaluate the proposed framework. The dataset, named CBFdataset, is the first dataset on the Chinese bamboo flute (CBF), containing full-length CBF performances and expert annotations of playing techniques. To provide evidence on the generalisability of the proposed framework, we test it over three additional datasets with a variety of playing techniques. Finally, to explore the applicability of the proposed scattering representations to general audio classification problems, we introduce two additional applications: one applies the adaptive scattering for identifying performers in polyphonic orchestral music and the other uses the joint scattering for detecting and classifying chick calls

Continuity and Change in Eugène Ysaÿe’s Six Sonatas, Op. 27, for Solo Violin

The Six Sonatas, Op. 27, for Solo Violin by the Belgian violinist and composer Eugène Ysaÿe (1858-1931), written in 1923/24, are increasingly adopted into the standard repertoire of violinists. Ysaÿe saw them as containing his legacy to future generations of violinists and composers and also as a statement of his aesthetic identity. However, not much research has been done on the aesthetics reflected in them. Yet, a greater awareness of Ysaÿe’s aesthetics will add to the understanding of this important historical figure who did so much to popularise French and Belgian music of the turn of the twentieth century and very much identified with the circle of composers around César Franck. This thesis focusses on Ysaÿe’s relationship with music history as represented in Op. 27. It explores his aesthetics, in particular his attitude to the past, present and future as well as his insistence on the continuity of history. Part I examines Ysaÿe’s historical and biographical context as well as his aesthetic predilections. It particularly focuses on composers to whom he was close, notably the Franckists, as well as on the violin tradition of which he was part, with an emphasis on Henri Vieuxtemps. As each Sonata is dedicated to a violinist of the generation after Ysaÿe, their personalities and playing styles are also discussed. Part II turns to the Sonatas themselves and explores ways in which Ysaÿe engages with past and contemporaneous composers, notably J. S. Bach, César Franck and Claude Debussy, as well as with the violin tradition and the possible influence of the dedicatees on their Sonata. It also demonstrates Ysaÿe’s contribution to music history, especially to the development of the technical and expressive possibilities of his instrument

Measuring Expressive Music Performances: a Performance Science Model using Symbolic Approximation

Music Performance Science (MPS), sometimes termed systematic musicology in Northern Europe, is concerned with designing, testing and applying quantitative measurements to music performances. It has applications in art musics, jazz and other genres. It is least concerned with aesthetic judgements or with ontological considerations of artworks that stand alone from their instantiations in performances. Musicians deliver expressive performances by manipulating multiple, simultaneous variables including, but not limited to: tempo, acceleration and deceleration, dynamics, rates of change of dynamic levels, intonation and articulation. There are significant complexities when handling multivariate music datasets of significant scale. A critical issue in analyzing any types of large datasets is the likelihood of detecting meaningless relationships the more dimensions are included. One possible choice is to create algorithms that address both volume and complexity. Another, and the approach chosen here, is to apply techniques that reduce both the dimensionality and numerosity of the music datasets while assuring the statistical significance of results. This dissertation describes a flexible computational model, based on symbolic approximation of timeseries, that can extract time-related characteristics of music performances to generate performance fingerprints (dissimilarities from an ‘average performance’) to be used for comparative purposes. The model is applied to recordings of Arnold Schoenberg’s Phantasy for Violin with Piano Accompaniment, Opus 47 (1949), having initially been validated on Chopin Mazurkas.1 The results are subsequently used to test hypotheses about evolution in performance styles of the Phantasy since its composition. It is hoped that further research will examine other works and types of music in order to improve this model and make it useful to other music researchers. In addition to its benefits for performance analysis, it is suggested that the model has clear applications at least in music fraud detection, Music Information Retrieval (MIR) and in pedagogical applications for music education

From Musical Grammars to Music Cognition in the 1980s and 1990s: Highlights of the History of Computer-Assisted Music Analysis

While approaches that had already established historical precedents – computer-assisted analytical approaches drawing on statistics and information theory – developed further, many research projects conducted during the 1980s aimed at the development of new methods of computer-assisted music analysis. Some projects discovered new possibilities related to using computers to simulate human cognition and perception, drawing on cognitive musicology and Artificial Intelligence, areas that were themselves spurred on by new technical developments and by developments in computer program design. The 1990s ushered in revolutionary methods of music analysis, especially those drawing on Artificial Intelligence research. Some of these approaches started to focus on musical sound, rather than scores. They allowed music analysis to focus on how music is actually perceived. In some approaches, the analysis of music and of music cognition merged. This article provides an overview of computer-assisted music analysis of the 1980s and 1990s, as it relates to music cognition. Selected approaches are being discussed

Application of singing synthesis techniquest to bertsolaritza

237 p