Presented at the 15th International Conference on Auditory Display (ICAD2009), Copenhagen, Denmark, May 18-22, 2009In this paper we extend the VIVO system with MIR function- ality that allows us to extract the VIVO rule database from a sym- bolic score. As a reference material we use a collection of se- lected Chorale harmonizations by J.S.Bach. Our extended system takes symbolic scores as input and generates a harmonic progres- sion database as output. The VIVO and the new extended sys- tem are realized inside the PWGL environment. Complete work- ing patches are given as examples. As a proof of concept the database-extracted from the repertoire-is then used in the VIVO to re-harmonize a short melody

Kuuskankare, Mika

Laurson, Mikael

English

Scholarly Materials And Research @ Georgia Tech

Proceedings of the 15th International Conference on Auditory Display, Copenhagen, Denmark May 18 - 22, 2009EXTENDING VIVO AS A MIR SYSTEMMika Kuuskankare and Mikael LaursonSibelius AcademyCentre for Music and TechnologyP.O.Box 86, FIN-02510, Finland{mkuuskan,laurson}@siba.fiABSTRACTIn this paper we extend the VIVO system with MIR function-ality that allows us to extract the VIVO rule database from a sym-bolic score. As a reference material we use a collection of se-lected Chorale harmonizations by J.S.Bach. Our extended systemtakes symbolic scores as input and generates a harmonic progres-sion database as output. The VIVO and the new extended sys-tem are realized inside the PWGL environment. Complete work-ing patches are given as examples. As a proof of concept thedatabase-extracted from the repertoire-is then used in theVIVOto re-harmonize a short melody.1. INTRODUCTIONSeveral different techniques have bee applied in producingchoraleharmonizations automatically. One of the most known exampleis arguably the rule-based choral harmonizer system by KemalEbcioglu [1]. Schottstaed [2] takes rules from Fux [3] to producespecies counterpoint. [4], in turn, applies genetic programmingtechniques to harmonization. Furthermore, [5] uses neuralnet-works and [6] Markov models to generate harmonic structures.An overview of constraint-based harmonization systems is givenin [7] and a comparison of constraint-based systems and geneticalgorithms can be found in [8].VIVO [9] is a system that can be used to visually create harmo-nization rules. It uses ENP [10] as the primary user-interface com-ponent. Rules are defined by giving examples in standard westernmusic notation. VIVO includes a set of special visualization de-vices that can be inserted in music notation to define aspectsofvoice-leading, for example. The visual rules are translated intotextual rules by the VIVO compiler. The advantages of VIVO arethat it allows us: (1) to represent the harmonic progressionandvoice-leading rules visually, (2) to represent the resultsin standardmusic notation, (3) to mix textual representation of rules with thevisual ones, and (4) to transparently mix heuristics and counter-point rules on top of the existing ones.Our previous work [9] demonstrated how the system can beused to prepare some traditional voice-leading assignments, suchas, suspensions, four-six progressions (V64–V), etc. While it is thepurpose of VIVO to let the user to define the rules by hand it isalso possible to create the whole rule database algorithmically.In this paper our objective is to let VIVO learn harmonic pro-gressions from the repertoire, i.e., to analyze existing scores toproduce the visual VIVO rule database. We augment the VIVOsystem with MIR functionality using an object-oriented scriptinglanguage that allows us to extract the VIVO rule database froma collection of symbolic scores. One of the benefits of the newMIR functionality is that it is possible to extract a rich setof musi-cal information using a sophisticated and object-orientedscriptinglanguage. Owing to it’s clean and simple syntax it is also straight-forward to convert these findings to generative rules (and also viceversa). All the tools described in this paper are realized insidePWGL [11].In the rest of this paper we first discuss and describe the re-trieval apparatus used in extracting the rule database fromexist-ing repertoire. In the remainder of the paper we use this databasein conjunction with the original VIVO system to generate sampleharmonization of a choral melody.2. THE RETRIEVAL APPARATUSIn the center of our retrieval system is the rule-based scripting lan-guage ENP-Script [12]. ENP-Script can be used to produce side-effects to or to extract information from an ENP score. The syntaxof the language is not presented here, instead the reader is advisedto study, for example [13]. Furthermore, recent developments inthe syntax are covered in detail in [14].In this paper we use ENP-script to define a collection of scriptsthat extract interesting properties from ENP scores. The propertieswe are primarily interested in are melodic movement, voicing, har-monic progression, and voice-leading.Currently, the rules fall into two main categories: (1) rulesthat define harmonic progression, and (2) rules that define voice-leading.In the earlier system described in [9], it was not possible todefine progressions that are key dependent. Here, the VIVO sys-tem has been augmented so that when a key signature is given inthe database score, VIVO interprets the individual harmonic rulesas absolute instead of relative. This allows us to define specificfunctions (e.g.,I–IV–V) inside one key. In case of chorale har-monization this is an important feature as the harmonic regionsseldom move very far from the key center.Simplicity in mind, we use a small collection of hand-pickedChorale harmonizations by J.S.Bach as a starting point. TheBWV248.5, 244.62, 244.54, 244.44, and 244.17 of the choraleHerzlichtut mich verlangenwere used. All the chorales were transposed ina-minor.The main retrieval process consists of the following steps:(1) Analyze all the scores for common tertian harmonies.This analysis is stored into a special variable called:common-chords (see the script given below).(2) Run the retrieval script on all the scores to collect all the woconsecutive sounding harmonies and record their relation toeach other. Here we use the :common-chords property to dis-tinguish between cases that contain passing tones from theones that do not.2.1. The Retrieval PatchFigure 1 shows the ’retrieval’ patch. The left part of the patchimports a collection of MIDI files in a loop and converts themICAD09-1Proceedings of the 15th International Conference on Auditory Display, Copenhagen, Denmark May 18 - 22, 2009into ENP scores. The scores are not quantized here. On the right,there is a box called ’Multi-Score-Editor’ displaying one of theimported scores. The Multi-Score-Editor can contain an arbitr rynumber of scores one of which is displayed at a time. The rest ofthe patch (below Multi-Score-Editor) deals with the retrieval of theproperties.2.2. The Retrieval ScriptNext, we discuss the ENP-Script for retrieving harmonic progres-sions and voice-leading (see Figure 2).It is important to realize that the script presented here isbased on a special pattern-matching language that is used tox-tract relevant objects from the score. In the first line of thescript (a) we define the pattern that we are interested in. Here,we want to record the relation between two consecutive har-monies in the score, thus the pattern(* ?source-chord?target-chord :harmony. The keyword:harmony indi-cates to the script that in this case we are interested in the ’har-mony’ property of the score. Each of the consecutive harmoniesare in turn bound to the two variables?source-chord and?target-chord.The scripting engine is constructed so that it goes through allthe partial harmonies (i.e., constructing every harmony note bynote from bottom up).Thus, it is necessary to tell the systemthatin this case we are only dealing with complete harmonies. Seethekeyword:complete? used in the condition found in (c). Therest of the script is executed only when both of the harmoniescon-tain all the sounding notes.The rest of the script deals with assigning the current harmonicsituation into one of the rule categories described above. Th sim-ple harmonic progression case is handled in (d-m) and the morelaborate one, in turn, in (o-z).The former case is quite straightforward. When both of theharmonies are simple tertian they are just pushed into a variblecalled:vivo-pre-analysis (see h-m). This becomes thenone of the harmonic progression rules in VIVO.The latter case, however, requires some extra work. Here, theassumption is that one or both of the harmonies contain pitchesthat are the result of passing tone texture, for example. To assurecorrect voice-leading, in this case, we constrain the movement ofthe individual voices. In VIVO, this was done visually by insertingcertain expressions in the notation to denote how the voicesshouldmove. Now, we simulate this algorithmically by collecting thenotes in each of the harmonies (o-r), looping them through (s-z)and inserting an appropriate ’voice-leading’ expression betweenthe two melodic notes in each of the voices (v-x). Finally, the newconstruction is pushed in the list of VIVO rules (y-z).Figure 3 illustrates how the retrieval script functions. Caseswith two consecutive tertian chords are marked with rectanglesvertically covering the notes of the harmony (see the rst andsecondmeasure). Cases where one (or both) of the chords is not a com-mon tertian chord are in turn marked with lines indicating the pass-ing tone movement (see, for example, the third and fourth mea-sure). Note, that visualization is automatically generated. Here,we use a slightly modified retrieval script that instead of buildingthe database inserts an appropriate expression in the score.In this section we described the procedure behind retrievingharmonic progressions and voice-leading from ENP scores. In thenext Section we attempt to validate the method by using the auto-matically extracted and generated visual VIVO rules to produce are-harmonization of an existing melody.a (* ?source-chord ?target-chord :harmonyb (?ifc (?incase (m ?target-chord :complete? t)d (cond ((and (member (sc-name (m ?source-chord))e (pwgl-value :common-chords))f (member (sc-name (m ?target-chord))g (pwgl-value :common-chords)))h (pwgl-value :vivo-pre-analysisi :pushj (list (duplicate-instancek (m ?source-chord :object t))l (duplicate-instancem (m ?target-chord :object t)))))n (To (let ((c1 (duplicate-instancep (m ?source-chord :object t)))q (c2 (duplicate-instancer (m ?target-chord :object t))))s (loop for n1 in (sort c1 ’< :key #’midi)t for n2 in (sort c2 ’< :key #’midi)u dov (insert-expressionw (list n1 n2)x (make-instance ’voice-leading)))y (pwgl-value :vivo-pre-analysisz :push (list c1 c2))))))))Figure 2: The retrieval script realized with ENP-script language.3. AN EXAMPLE RE-HARMONIZATIONNext, we use the harmonic progression database extracted fromthe example score to re-harmonize a short chorale melody. Fig-ure 4 gives a complete VIVO patch consisting of (1) the visualrule database, (2) the textual rule generator, (3) the inputscorewith a pre-constrained melody, (4) the output score with there-harmonized chorale melody, (5) basic voice-leading rules (includ-ing rules excluding parallel movement, for example), (6) heuristicrules, and (7) the solver box.When the patch is evaluated the visual rules (1) are convertedto textual ones by the VIVO interpreter (2). The textual rulesare used by the solver box (7). In (3) the user can enter a pre-constrained melody or to let VIVO create a new one according touser defined rules. The result is displayed in (4) in standardmu-sic notation. A special ’sieve’ plug-in is applied to the result thatconnects smaller durational units–consisting of pitch repetitions–into larger ones. The basic pulse of the three lower voices can bedefined in the input-score.Finally, Figure 5 shows a sample re-harmonization of the firstphrase ofHerzlich tut mich verlangen. A quick analysis revealsseveral points of interest. Generally speaking, the harmonic fu c-tions appear to be correct. The movement of the voices is quitesatisfactory apart from the Tenor part which is probably tooagileas in this type of music the middle voices usually tend to be quitestatic. This could also be due to the fact that the melody is setunconventionally high. The setting is not optimal but this aspectis not covered with the rules. Likewise, the handling of the lead-ing tone also leaves something to wish for. It has also to be not dthat currently this system has no notion of a cadence. Thus, teexample here was selected as it was the first run that in the endproduced a cadence in a-minor. Sound examples can be found atwww.siba.fi/pwgl/demos/cmmr09-vivo.html.4. DISCUSSIONOne of the benefits of our system is that once the VIVO databaseisgenerated it is possible to visually examine and edit it. This allowsus to take away progressions we want to omit or to add new ones.It is also possible to validate the database by listening to the result.This should prove to be a quick and intuitive method of checkingthe integrity of the retrieval process as well. Furthermore, th sys-tem allows easily to merge databases and to add other rules whnICAD09-2Proceedings of the 15th International Conference on Auditory Display, Copenhagen, Denmark May 18 - 22, 2009Figure 1: The basic retrieval patch.Figure 3: The rectangle shapes indicate common tertian harmonies. The line shapes indicate the presence of passing tonem v ment and/orincomplete tertian harmonies.Figure 4: A VIVO patch that comprises of (1) the visual rule database, (2) the textual rule generator, (3) the input score with a pre-constrained melody, (4) the output score, (5) basic voice-leading rules, (6) heuristic rules, and (7) the solver box.ICAD09-3Proceedings of the 15th International Conference on Auditory Display, Copenhagen, Denmark May 18 - 22, 2009Figure 5: A sample re-harmonization of a chorale melody.it is used as a generative tool.The repertoire of the retrieval system should be extended tocover other aspects of the music, such as, setting, melody, etc. Theretrieval process should also analyze the input material for phrasesand cadences to create more convincing harmonic functions.Fur-thermore, our system should take advantage of other traditionalMIR techniques such as pitch histograms and N-grams as consid-ered by several authors (see for example [15]).It is evident that in order to realize more ambitious examplesa larger corpus of analyzed data should be used. The method pre-sented here serves as a proof of concept and should be exploredfurther. However, the re-harmonizations that the current sys emcreates have already proven to be quite practical.5. ACKNOWLEDGMENTSThe work of Mika Kuuskankare and Mikael Laurson has been sup-ported by the Academy of Finland (SA 114116 and SA 105557).6. REFERENCES[1] Kemal Ebcioglu, “An Expert System for HarmonizingChorales in the Style of J. S. Bach,”Journal of Logic Pro-gramming, vol. 8, no. 1, pp. 145–185, 1990.[2] B. Schottstaedt, “Automatic species counterpoint,” Ccrma,Stanford University, 1989.[3] Johann Joseph Fux,Gradus ad Parnassum, 1725.[4] R. A. McIntyre, “Bach in a box: The evolution of four partbaroque harmony using the genetic algorithm,” inIEEE Con-ference on Evolutionary Computation, 1994, pp. 852–857.[5] Johannes Feulner Hermann Hild and Wolfram Menzel,“HARMONET: A neural net for harmonizing chorales in thestyle of J.S. Bach,” inAdvances in Neural Information Pro-cessing. 1991, vol. 4, pp. 267–274, Morgan Kaufmann.[6] Geraint Wiggins Dan Ponsford and Chris Mellish, “Statisti-cal learning of harmonic movement,”Journal of New MusicResearch, 1999.[7] François Pachet and Pierre Roy, “Musical harmonizationwith constraints: A survey,”Constraints, vol. 6, no. 1, pp.7–19, 2001.[8] S. Phon-Amnuaisuk and G. A. Wiggins, “The four-part har-monisation problem: a comparison between genetic algo-rithms and a rule-based system,” inAISB’99 Symposium onMusical Creativity, 1999.[9] Mika Kuuskankare and Mikael Laurson, “VIVO - visual-izing harmonic progressions and voice-leading in PWGL,”in International Symposium on Music Information Retrieval,2007, pp. 289–290.[10] Mika Kuuskankare and Mikael Laurson, “Expressive Nota-tion Package,”Computer Music Journal, vol. 30, no. 4, pp.67–79, 2006.[11] Mikael Laurson, Mika Kuuskankare, and Vesa Norilo, “AnOverview of PWGL, a Visual Programming Environment forMusic,” Computer Music Journal, vol. 33, no. 1, 2009.[12] Mika Kuuskankare and Mikael Laurson, “Intelligent Script-ing in ENP using PWConstraints,” inProceedings of Inter-national Computer Music Conference, Miami, USA, 2004,pp. 684–687.[13] Mikael Laurson, PATCHWORK: A Visual ProgrammingLanguage and some Musical Applications, Studia mu-sica no.6, doctoral dissertation, Sibelius Academy, Helsinki,1996.[14] Mikael Laurson and Mika Kuuskankare, “Extensible Con-straint Syntax Through Score Accessors,” inJournéesd’Informatique Musicale, Paris, France, 2005.[15] J. S. Downie, Evaluating a Simple Approach to Music In-formation Retrieval: Conceiving Melodic N-grams as Text,Ph.D. thesis, University of Western Ontario, 1999.ICAD09-4

In this paper we extend the VIVO system with MIR function- ality that allows us to extract the VIVO rule database from a sym- bolic score. As a reference material we use a collection of se- lected Chorale harmonizations by J.S.Bach. Our extended system takes symbolic scores as input and generates a harmonic progres- sion database as output. The VIVO and the new extended sys- tem are realized inside the PWGL environment. Complete work- ing patches are given as examples. As a proof of concept the database-extracted from the repertoire-is then used in the VIVO to re-harmonize a short melody.

https://smartech.gatech.edu/bitstream/1853/51299/1/KuuskankareLaurson2009.pdf

Abstract

Similar works

Full text

Available Versions

Scholarly Materials And Research @ Georgia Tech