Music Information Retrieval in Live Coding: A Theoretical Framework

The work presented in this article has been partly conducted while the first author was at Georgia Tech from 2015–2017 with the support of the School of Music, the Center for Music Technology and Women in Music Tech at Georgia Tech. Another part of this research has been conducted while the first author was at Queen Mary University of London from 2017–2019 with the support of the AudioCommons project, funded by the European Commission through the Horizon 2020 programme, research and innovation grant 688382. The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Music information retrieval (MIR) has a great potential in musical live coding because it can help the musician–programmer to make musical decisions based on audio content analysis and explore new sonorities by means of MIR techniques. The use of real-time MIR techniques can be computationally demanding and thus they have been rarely used in live coding; when they have been used, it has been with a focus on low-level feature extraction. This article surveys and discusses the potential of MIR applied to live coding at a higher musical level. We propose a conceptual framework of three categories: (1) audio repurposing, (2) audio rewiring, and (3) audio remixing. We explored the three categories in live performance through an application programming interface library written in SuperCollider, MIRLC. We found that it is still a technical challenge to use high-level features in real time, yet using rhythmic and tonal properties (midlevel features) in combination with text-based information (e.g., tags) helps to achieve a closer perceptual level centered on pitch and rhythm when using MIR in live coding. We discuss challenges and future directions of utilizing MIR approaches in the computer music field

Developing Student Agency in the Choral Classroom: A Case Study

This multiple-case study sought to describe how student agency is cultivated within two high school choir programs. Research questions were: (a) In what ways did participating teachers incorporate student agency? (b) How did their students describe their experience? (c) What opportunities/difficulties emerged when students assumed leadership and decision-making roles? (d) Why do participating teachers offer these to students? (e) How do opportunities for student agency serve student learning and program goals? Data collection included semi-structured teacher interviews in two stages, as well as a student questionnaire. Other data included choir handbooks, student leadership information, social media presence, and classroom signs. Emergent themes were examined through Emirbayer and Mische’s (1998) temporal-relational view of agency, in which actors make choices within a “temporally embedded process of social engagement, informed by the past, but oriented towards the future and… present”(p. 963) . Iterative agency was exercised as students took actions based on their previous choir and leadership experiences. Opportunities for practical-evaluative agency included student-led sectionals, planning and executing choir events, and making interpretive musical decisions. Teachers considered students’ projective agency through intentional music literacy instruction (“literate, thinking musicians”) and promoting virtues (“better people”). A culture of strong relationships and a safe environment promoted agency. Implications for teaching practice include incorporating student-centered instructional activities into rehearsal: student-led sectionals, discussions about musical interpretation, and questioning techniques. Teachers could choose student conductors or allowing students to rehearse the entire ensemble when directors are absent. Student leadership structures that allow students to participate throughout high school were beneficial, including regular opportunities for director guidance and communication. While directors still retain primary authority, students can be given more agency and autonomy in certain areas, such as culturally relevant performances and social aspects of choir life. All data was collected during the COVID-19 pandemic; therefore, observations were not possible. Gatekeepers in one district did not allow student interviews. As a result, implications for future research include multiple periodic observations over the course of a school year, and incorporation of student interviews. Future studies could investigate the presence of student agency in urban schools, and those lacking strong feeder programs

Haptic communication between partner dancers and swing as a finite state machine

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Vita.Includes bibliographical references (p. 129-138).To see two expert partners, one leading and the other following, swing dance together is to watch a remarkable two-agent communication and control system in action. Even blindfolded, the follower can decode the leader's moves from haptic cues. The leader composes the dance from the vocabulary of known moves so as to complement the music he is dancing to. Systematically addressing questions about partner dance communication is of scientific interest and could improve human-robotic interaction, and imitating the leader's choreographic skill is an engineering problem with applications beyond the dance domain. Swing dance choreography is a finite state machine, with moves that transition between a small number of poses. Two automated choreographers are presented. One uses an optimization and randomization scheme to compose dances by a sequence of shortest path problems, with edge lengths measuring the dissimilarity of dance moves to each bar of music. The other solves a two-player zero-sum game between the choreographer and a judge. Choosing moves at random from among moves that are good enough is rational under the game model.(cont.) Further, experiments presenting conflicting musical environments to two partners demonstrate that although musical expression clearly guides the leader's choice of moves, the follower need not hear the same music to properly decode the leader's signals. Dancers embody gentle interaction, in which each participant extends the capabilities of the other, and their cooperation is facilitated by a shared understanding of the motions to be performed. To demonstrate that followers use their understanding of the move vocabulary to interact better with their leaders, an experiment paired a haptic robot leader with human followers in a haptically cued dance to a swing music soundtrack. The subjects' performance differed significantly between instances when the subjects could determine which move was being led and instances when the subjects could not determine what the next move would be. Also, two-person teams that cooperated haptically to perform cyclical aiming tasks showed improvements in the Fitts' law or Schmidt's law speed-accuracy tradeoff consistent with a novel endpoint compromise hypothesis about haptic collaboration.by Sommer Elizabeth Gentry.Ph.D

INTERACTIVE SONIFICATION STRATEGIES FOR THE MOTION AND EMOTION OF DANCE PERFORMANCES

The Immersive Interactive SOnification Platform, or iISoP for short, is a research platform for the creation of novel multimedia art, as well as exploratory research in the fields of sonification, affective computing, and gesture-based user interfaces. The goal of the iISoP’s dancer sonification system is to “sonify the motion and emotion” of a dance performance via musical auditory display. An additional goal of this dissertation is to develop and evaluate musical strategies for adding layer of emotional mappings to data sonification. The result of the series of dancer sonification design exercises led to the development of a novel musical sonification framework. The overall design process is divided into three main iterative phases: requirement gathering, prototype generation, and system evaluation. For the first phase help was provided from dancers and musicians in a participatory design fashion as domain experts in the field of non-verbal affective communication. Knowledge extraction procedures took the form of semi-structured interviews, stimuli feature evaluation, workshops, and think aloud protocols. For phase two, the expert dancers and musicians helped create test-able stimuli for prototype evaluation. In phase three, system evaluation, experts (dancers, musicians, etc.) and novice participants were recruited to provide subjective feedback from the perspectives of both performer and audience. Based on the results of the iterative design process, a novel sonification framework that translates motion and emotion data into descriptive music is proposed and described

Perle Noire: Meditations for Josephine - Aesthetics, Discussion, and Reception

This dissertation, organized in two parts, is comprised of an essay on my song cycle Perle Noire: Meditations for Josephine and its scores. Perle Noire consists of a flexible musical score that I composed for soprano Julia Bullock, select members of the International Contemporary Ensemble, and myself, with two performance editions. The primary edition, which stems from part of a production conceived of by theater director Peter Sellars, includes recited poetry about Josephine Baker by Claudia Rankine and choreography by Michael Schumacher. The second edition is a musical performance with neither poetic texts nor choreography. The world premiere of Perle Noire took place at the 2016 Ojai Music Festival in California, with subsequent performances at Lincoln Center in New York; Da Camera in Houston, Texas; and the Stony Island Arts Center in Chicago, Illinois. The entire performance of the 90-minute song cycle entails the following five recompositions and one “head arrangement” for a mixed ensemble of improviser-performers and soprano: Part I—1. Bye Bye Blackbird, 2. Sous le Ciel d’Afrique, 3. C’est ca le Vrai Bonheur and Madiana (medley); Part II—4. Si J’etais Blanche, 5. C’est Lui, 6. Terre Seche (Negro Spiritual). The essay centers on the compositions in Part II of Perle Noire: “Si J’etais Blanche,” “C’est Lui,” and “Terre Seche” (Negro Spiritual). To begin, I briefly discuss my aesthetic exigencies in relation to the Black creative musics initiated during the latter part of the twentieth century as well as the inspiration informing the creation of this song cycle. Next, I discuss the aforementioned songs to demonstrate how my aesthetics play out in this work. Finally, I detail the controversial critical reception of the song cycle’s world premiere performance and my response to it

Fusion of Multimodal Information in Music Content Analysis

Music is often processed through its acoustic realization. This is restrictive in the sense that music is clearly a highly multimodal concept where various types of heterogeneous information can be associated to a given piece of music (a musical score, musicians\u27 gestures, lyrics, user-generated metadata, etc.). This has recently led researchers to apprehend music through its various facets, giving rise to "multimodal music analysis" studies. This article gives a synthetic overview of methods that have been successfully employed in multimodal signal analysis. In particular, their use in music content processing is discussed in more details through five case studies that highlight different multimodal integration techniques. The case studies include an example of cross-modal correlation for music video analysis, an audiovisual drum transcription system, a description of the concept of informed source separation, a discussion of multimodal dance-scene analysis, and an example of user-interactive music analysis. In the light of these case studies, some perspectives of multimodality in music processing are finally suggested