Masu, R

MCPHERSON

MICI 2017: CHI Workshop on Mixed-Initiative Creative Interfaces

MORREALE, F

English

Queen Mary Research Online

Constraining Control inMixed-Initiative Musical InterfacesFabio MorrealeCentre for Digital MusicSchool of EECSQueen Mary University ofLondon, UKf.morreale@qmul.ac.ukRaul MasuinterAction LabDISIUniversity of TrentoItalyraul.masu@unitn.itAndrew McPhersonCentre for Digital MusicSchool of EECSQueen Mary University ofLondon, UKa.mcpherson@qmul.ac.ukCopyright ©2017 for this paper is held by the author(s). Proceedings of MICI 2017:CHI Workshop on Mixed-Initiative Creative Interfaces.AbstractResearch on musical instrument design suggests that de-liberate constraints can offer new creative experiences tothe performer. At times, design constraints are physicallyembedded in the instrument to limit the interaction possibil-ities; in other cases the constraints are given by delegatingpart of the control to the machine. In this paper we pro-pose a case study related to the latter form of constraints:in Chimney the musician delegates control on timing to anautonomous agent. Elaborating on opinions collected frommusicians interacting with Chimney, we reflect on the con-sequences of delegating part of the control to the machine.Without the possibility of influencing the temporal evolutionof the piece, the human performer is pushed to find creativeworkarounds and to surrender to a more balanced collabo-ration with the machine.Author Keywordsdigital musical instruments; creativity; constraints; interfacedesignACM Classification KeywordsH.5.5 [Sound And Music Computing]: SystemsIntroductionIn Mixed-Initiative Creative Interfaces humans and ma-chines collaborate to produce creative outputs. Efforts areusually made for the two parts to dialogue, each employ-ing their own assets and delegating their limitations to theother. However, efforts can also be made to specificallyconstrain human control on the creative process. Con-straints are not ruthless attempt to limit human creativity;rather the opposite can be true as advocated by MargaretA. Boden in her seminal book on human creativity [1] :“People often claim that talk of ’rules’ and ’constraints’...mustbe irrelevant to creativity, which is an expression of humanfreedom. But far from being the antithesis of creativity, con-straints on thinking are what make it possible...Constraintsmap out a territory of structural possibilities which can thenbe explored, and perhaps transformed to give another one”In the musical domain, physical constraints contribute tocharacterise the instrument expressive scope and influenceits playing possibilities and the development of a personalstyle [8]. For instance, the constraints of the piano defineits expressive character, which, in turn, defines the pianist’sstyle of playing and the compositions possibilites [4].New digital musical instruments (DMIs) also sometimes in-tegrate design constraints for creative explorations. Suchconstraints can limit the number of inputs and the outcomesthat are mapped to those inputs. For example, Gurevichand colleagues designed a purposely over-simplified in-strument, in which performer interaction is very limited [3].Despite its limitations, or perhaps because of them, the per-formers exhibited a significant degree of stylistic diversity intheir interaction with the instrument.In a similar study, Zappi’s and McPherson’s Cube Instru-ment was designed to show the role of interaction con-straints in encouraging “creative (mis)uses of technology”[8]. Several performers that used the Cube Instrument re-ported that constraints themselves were conducive to ex-ploring subtle musical variations.Another form of constraint in musical interaction is given bydelegating part of the control to a machine. This is the casefor conductor systems, which are essentially tools that allowmusicians to control the playback of pre-recorded music.Performers only control timing, dynamics, and other ex-pressive factors and the machine worries about getting thenotes right [5]. These systems have attracted the interest ofthe general public (Guitar Hero1, Magic Fiddle 2 and similarapplications are forms of conductor programs) by offeringa simple interaction that does not require extensive trainingon the instrument [2].This paper proposes another example in which part of thecontrol is delegated to the machine. Chimney, a softwareinstrument developed by two of the authors [6], forces themusician to delegate timing decisions to a non-responsiveautonomous agent. As a consequence, the human agenthas limited control on the temporal evolution of the piece.The behaviour of Chimney is in a sense the opposite kind ofdelegation compared to a conductor system. With conduc-tor systems a musicians can mainly get time control; withChimney timing is the main thing the musician lacks. Thespecific behaviour of the agent is secondary to the scopeof this paper, which rather focuses on the consequencesof divesting control on timing to an unpredictable algorith-mic agent. The implications on the experience of the per-former of delegating control on timing to the machine arepresented in the Discussion section.Figure 1: Screenshot of Chimney. The white path is the trace leftby the algorithmic walker. The circles are the sonic material addedby the human performer.Chimney: Delegating TimingAn example of DMI in which the musician divests control ontiming to the machine is offered by Chimney [6]. In this vir-tual DMI the control on the temporal evolution of the pieceis delegated to a computer agent, an algorithmic randomwalker. The walker autonomously roams throughout a vir-tual space following a specific statistical distribution mod-elled on an adapted version of the Perlin Noise [7].The musician can interact with the composition by adding tothe virtual space pre-recorded sonic material. These soundsources are displayed as circles that can be resized, reposi-tioned, or deleted in real time by the musician. When a newcircle is added to the canvas, the system sets it in idle state1https://www.guitarhero.com/uk/en/2https://www.smule.com/sunset/magicfiddleand mutes it. As the random walker enters a circle, the am-plitude of the sound connected to that circle increases. Themaximum level is reached when the walker is at the centreof the circle.Player interaction is then reduced to deciding the sonic ob-jects, and their likelihood of being played. Under these con-ditions, the musician cannot organise the temporal struc-ture, which is entirely controlled by the algorithmic agent.An informal evaluation was conducted at a public concert inwhich Chimney was perfomed in a duo with a trumpet [6].Both the musician that was controlling Chimney (the secondauthor of this paper) and the trumpeter were interviewed atthe end of the performace.The lack of control over timing caused interesting experi-ences for both musicians. The performer who was control-ling Chimney reported emotional reactions that usually donot belong to the palette of emotions in music. Unable tocontrol every aspect of the music, he could only wish for thealgorithm to answer in a particular way. As a consequence,he experienced hope, surprise, frustration, and resignation.Other interesting comments were collected from the trum-peter. Understanding all the details of the music played byChimney was particularly demanding. In particular, as op-posed to traditional improvisation, he did not feel free topropose new musical material. He was always answering toChimney proposals. The only exception to such approachconcerned big changes in dynamics (e.g. very intensecrescendo or diminuendo). In this cases, the trumpeterexplicitly communicated to the Chimney performer to addsonic material to the canvas to obtain a crescendo, and toremove some for the diminuendo. Furthermore, the trum-peter commented that the specific behaviour of Chimneyof being non-responsive fostered him to find novel musi-cal strategies. For instance, in those situations in which hewould have preferred a higher complexity in the music, hecompensated for this perceived deficiency by increasing therhythm complexity or the loudness of his instrument.DiscussionThis paper reviewed and compared different ways in whichconstraining the interaction possibilities of a peformer canbe a vector of new forms of creativity. In particular, our pre-liminary investigations with Chimney suggest that delegat-ing some initiative to the machine produces a set of creativereactions and feelings that are at times similar and at timesricher than simply constraining the interaction possibilies.For instance, [8] found that players who got more degreesof freedom on the Box Instrument (they could control pitch)exhibtited less unexpected techniques than those with asingle degree of freedom (no control on pitch). In this case,players who could not control such an important musicalfeature needed to came up with original approaches toplaying. Similarly, the musicians that performed with Chim-ney, even if they could control every other aspect of the per-formance, lacked control of timing, which fundamentallychanged how the performance was structured.One difference that we found is presence of a range ofaffective states like surpise and hope that are present inChimney and other MICIs but seem to be absent from sim-ple DMIs and conductor systems. The self-frustration expe-rienced by the performer that operated Chimney is a typicalfeeling experienced in music - for instance by instrumen-talists when they cannot play a passage right. However, inthis case, the frustration was accredited to the impossibilityof precisely expressing one’s personal musical initiative, nomatter how hard he could try.Finally, it should be noted that in typical MICIs, the com-puter acquires the status of creative agent by using formsof artificial intelligence that actively take decisions on tasksthat are typically performed by humans. Although the arti-ficial intelligence element is limited or absent in the exam-ples presented in this paper, we believe that our reflectionsabout the consequences of divesting control to a machinecan be of interest for the disussions at the workshop.References[1] Margaret A Boden. 2004. The creative mind: Mythsand mechanisms. Psychology Press.[2] Elaine Chew and Andrew McPherson. Forthcoming.Performing music: humans, computers and electron-ics. In The Routledge Companion to Music Cognition,R Ashley and R Timmers (Eds.). NY: Taylor and Fran-cis.[3] Michael Gurevich, Paul Stapleton, and AdnanMarquez-Borbon. 2010. Style and Constraint in Elec-tronic Musical Instruments.. In NIME. 106–111.[4] Thor Magnusson. 2010. Designing constraints: Com-posing and performing with digital musical systems.Computer Music Journal 34, 4 (2010), 62–73.[5] Max V Mathews. 1991. The radio baton and conductorprogram, or: Pitch, the most important and least ex-pressive part of music. Computer Music Journal 15, 4(1991), 37–46.[6] Fabio Morreale and Raul Masu. 2016. RenegotiatingResponsibilities in Human-Computer Ensembles. Pro-ceedings of Colloquio di Informatica Musicale (2016),41–45.[7] Ken Perlin. 1985. An image synthesizer. ACM Sig-graph Computer Graphics 19, 3 (1985), 287–296.[8] Victor Zappi and Andrew McPherson. 2014. Designand use of a hackable digital instrument. In NIME.

Constraining Control in Mixed-Initiative Musical Interfaces

https://qmro.qmul.ac.uk/xmlui/bitstream/123456789/25285/2/Morreale%20Constraining%20Control%20in%20Mixed%202017%20Published.pdf

Constraining Control in Mixed-Initiative Musical Interfaces

Abstract

Similar works

Full text

Available Versions

Queen Mary Research Online