Computational Models of Expressive Music Performance: A Comprehensive and Critical Review

Expressive performance is an indispensable part of music making. When playing a piece, expert performers shape various parameters (tempo, timing, dynamics, intonation, articulation, etc.) in ways that are not prescribed by the notated score, in this way producing an expressive rendition that brings out dramatic, affective, and emotional qualities that may engage and affect the listeners. Given the central importance of this skill for many kinds of music, expressive performance has become an important research topic for disciplines like musicology, music psychology, etc. This paper focuses on a specific thread of research: work on computational music performance models. Computational models are attempts at codifying hypotheses about expressive performance in terms of mathematical formulas or computer programs, so that they can be evaluated in systematic and quantitative ways. Such models can serve at least two purposes: they permit us to systematically study certain hypotheses regarding performance; and they can be used as tools to generate automated or semi-automated performances, in artistic or educational contexts. The present article presents an up-to-date overview of the state of the art in this domain. We explore recent trends in the field, such as a strong focus on data-driven (machine learning) approaches; a growing interest in interactive expressive systems, such as conductor simulators and automatic accompaniment systems; and an increased interest in exploring cognitively plausible features and models. We provide an in-depth discussion of several important design choices in such computer models, and discuss a crucial (and still largely unsolved) problem that is hindering systematic progress: the question of how to evaluate such models in scientifically and musically meaningful ways. From all this, we finally derive some research directions that should be pursued with priority, in order to advance the field and our understanding of expressive music performance

The role of gesture and non-verbal communication in popular music performance, and its application to curriculum and pedagogy

Jane Davidson states that ‘the use of the body is vital in generating the technical and expressive qualities of a musical interpretation’ (2002, p. 146). Although technique and expression within music performance are separate elements, ‘they interact with, and depend upon, one another’ (Sloboda, 2000, p. 398) and, therefore, require equal consideration. Although it is possible for a musician to perform with exceptional technical prowess but little expression (Sloboda, 2000), it is important that the significance of the expressive qualities of the performer, and the ramifications of these on the delivery of the given performance, are acknowledged because whilst ‘sound is the greatest result of performance’ (Munoz, 2007, p. 56), music is not exclusively an auditory event; principally because ‘sound is essentially movement’ (Munoz, 2007, p. 56). As a performing art, music relies on the use of the physical self and body in the communicative process, and may require more than technical skill and proficient instrumental handling to be truly communicatively effective not least because, as stated by Juslin and Laukka, ‘music is a means of emotional expression' (2003, p. 774). Through a designed interdisciplinary framework, this thesis examines the use of expressive gesture and non-verbal communication skills in popular music performance, and investigates how these communicative facets can be incorporated into popular music performance education within a higher education curriculum. To do this, this work explores the practices of student and professional musicians, focusing on the areas of gesture, persona and interaction, and uses ethnographic case studies, qualitative interview processes and extracts of video footage of 3 rehearsals and live performances to investigate the importance of the physical delivery of the given musical performance. The findings from these investigations are then applied to existing educational theories to construct a pedagogical approach which will provide student musicians with the knowledge and skill to understand the implications of the art of performance through assimilated study, allowing performers to develop their own unique style of artistic expression, and creating well-rounded, empathetic, and employable musicians who have a visceral understanding of their art form

A expressividade por meio da manipulação de recursos de expressão e evocação de emoção na performance pianística de crianças

O presente estudo, sob a temática da expressividade, buscou investigar meios de execução e de organização dos elementos musicais, tendo a manipulação expressiva (imposta ou sugerida) como estímulo à realização de contrastes entre possíveis diferentes versões de uma mesma peça pelo estudante de piano, bem como o estímulo à reflexão dos estudantes a partir de imagens, metáforas e emoções como alusão ao resultado sonoro atingido ou a ser buscado por estes. Três crianças, com idades entre 7 e 11 anos, e tempo de estudo do piano entre 6 meses e 2 anos e meio, estiveram diante das tarefas de manipulação expressiva, e estas se deram em estratégias de dupla natureza: em um primeiro encontro, como foco de manipulação eram impostos aos estudantes parâmetros específicos em seus diferentes contrastes de andamento, dinâmica e articulação; em um segundo encontro, aos estudantes eram sugeridas emoções básicas como foco de manipulação. Em uma segunda etapa de coleta de dados, avaliadores externos, bem como o próprio doutorando como avaliador, atribuíram conceitos de 1 a 5 em escala de Likert sobre cada elemento musical componente de cada produto de performance manipulado pelas crianças. Os resultados das avaliações demonstraram que esta segunda estratégia de manipulação (com foco em emoções básicas) foram mais efetivas entre os participantes. Ainda, esses resultados indicaram que elementos próprios da realização estrutural da peça (acuidade de notas e precisão rítmica) se mostram como maior preocupação e foco desses estudantes, mas também revelaram o timbre como elemento mais valorizado entre os participantes na segunda sessão de coleta, indicando uma possível expressividade natural nas crianças. Como proposição de tese, um modelo piramidal foi elaborado com função de hierarquizar os elementos componentes da performance (acuidade de notas, precisão e organização rítmica, equilíbrio sonoro, timing, fraseado, timbre e ressonância) em estágios de complexidade de execução.The present study, under the theme of expressiveness, sought to investigate means of execution and organization of musical elements, with expressive manipulation (imposed or suggested) as a stimulus to the realization of contrasts between possible different versions of the same piece by the piano student, as well as the stimulus for students' reflection based on images, metaphors, and emotions as an allusion to the sound result achieved or to be sought by them. Three children, aged between 7 and 11 years, and with piano study of 6 months to 2 and a half years, were faced with expressive manipulation tasks, and these took place in dual-nature strategies: in a first meeting, as focus of manipulation, specific parameters were imposed on the students in their different contrasts of tempo, dynamics and articulation; in a second meeting, students were suggested basic emotions as the focus of manipulation. In a second stage of data collection, external evaluators, as well as the doctoral student himself as evaluator, attributed grades from 1 to 5 on the Likert scale to each musical element that was part of each performance product manipulated by the children. The evaluation results showed that this second strategy of manipulation (focusing on basic emotions) was more effective among participants. Furthermore, these results indicated that elements typical of the structural realization of the piece (acuity of notes and rhythmic precision) are shown to be the greatest concern and focus of these students, but also revealed timbre as the most valued element among participants in the second collection session, indicating possible natural expressiveness in children. As a thesis proposition, a pyramidal model was elaborated in order to hierarchize the component elements of the performance (acuity of notes, precision and rhythmic organization, sound balance, timing, phrasing, timbre and resonance) in stages of performance complexity

JAZZ ENSEMBLE EXPRESSIVE PERFORMANCE MODELING

ABSTRACT Computational expressive music performance studies the analysis and characterisation of the deviations that a musician introduces when performing a musical piece. It has been studied in a classical context where timing and dynamic deviations are modeled using machine learning techniques. In jazz music, work has been done previously on the study of ornament prediction in guitar performance, as well as in saxophone expressive modeling. However, little work has been done on expressive ensemble performance. In this work, we analysed the musical expressivity of jazz guitar and piano from two different perspectives: solo and ensemble performance. The aim of this paper is to study the influence of piano accompaniment into the performance of a guitar melody and vice versa. Based on a set of recordings made by professional musicians, we extracted descriptors from the score, we transcribed the guitar and the piano performances and calculated performance actions for both instruments. We applied machine learning techniques to train models for each performance action, taking into account both solo and ensemble descriptors. Finally, we compared the accuracy of the induced models. The accuracy of most models increased when ensemble information was considered, which can be explained by the interaction between musicians

Jazz ensemble expressive performance modeling

Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA.Computational expressive music performance studies the analysis and characterisation of the deviations that a musician introduces when performing a musical piece. It has been studied in a classical context where timing and dynamic deviations are modeled using machine learning techniques. In jazz music, work has been done previously on the study of ornament prediction in guitar performance, as well as in saxophone expressive modeling. However, little work has been done on expressive ensemble performance. In this work, we analysed the musical expressivity of jazz guitar and piano from two different perspectives: solo and ensemble performance. The aim of this paper is to study the influence of piano accompaniment into the performance of a guitar melody and vice versa. Based on a set of recordings made by professional musicians, we extracted descriptors from the score, we transcribed the guitar and the piano performances and calculated performance actions for both instruments. We applied machine learning techniques to train models for each performance action, taking into account both solo and ensemble descriptors. Finally, we compared the accuracy of the induced models. The accuracy of most models increased when ensemble information was considered, which can be explained by the interaction between musicians.This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R) and the H2020-ICT-688269 TELMI project

Jazz ensemble expressive performance modeling

Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA.Computational expressive music performance studies the analysis and characterisation of the deviations that a musician introduces when performing a musical piece. It has been studied in a classical context where timing and dynamic deviations are modeled using machine learning techniques. In jazz music, work has been done previously on the study of ornament prediction in guitar performance, as well as in saxophone expressive modeling. However, little work has been done on expressive ensemble performance. In this work, we analysed the musical expressivity of jazz guitar and piano from two different perspectives: solo and ensemble performance. The aim of this paper is to study the influence of piano accompaniment into the performance of a guitar melody and vice versa. Based on a set of recordings made by professional musicians, we extracted descriptors from the score, we transcribed the guitar and the piano performances and calculated performance actions for both instruments. We applied machine learning techniques to train models for each performance action, taking into account both solo and ensemble descriptors. Finally, we compared the accuracy of the induced models. The accuracy of most models increased when ensemble information was considered, which can be explained by the interaction between musicians.This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R) and the H2020-ICT-688269 TELMI project