On the Human Role in Generative Art: A Case Study of AI-driven Live Coding

The evolution of philosophical views on art is interwoven with trajectories of accelerating technological amelioration. In the emergence of generative algorithms there is a need for making sense of modern technologies that step in a realm previously reserved for humans – creativity. This paper aims to understand the role of the human in generative art by demystifying implications of black-box generative algorithms and their applications for artistic purposes. First, we shortly canvas the current state of practice and research in generative art, especially music. Then, we introduce Anastatica, a part performance, part installation built using data-driven generative live coding. Finally, we discuss implications of AI in art through a case study of Anastatica’s development and performance. We trace the path from algorithms to intelligence, applying musical and computer science theory to a practical case of generating a live coding musical performance, while focusing on aesthetic, compositional, conceptual, and phenomenological implications

Toward an Artist-Centred AI

Awareness about the immense impact that artificial intelligence (AI) might have or already has made on the social, economic, political, and cultural realities of our world has become part of the mainstream public discourse. Attributes such as ethical, responsible, or explainable emerge as associative and descriptive nominal references in guidelines that influence perspectives on AI application and development. This paper contextualizes the notions of suitability and desirability of principles, practices, and tools related to the use of AI in the arts. The result is a framework drafted as a set of atomic attributes that summarize the values of AI deemed important for artistic creativity. It was composed by examining the challenges that AI poses to art production, distribution, consumption, and monetization. Considering the differentiating potentials of AI and taking a perspective aside from the purely technical ontology, we argue that artistically pertinent AI should be unexpected, diversified, affordant, and evolvable

Visual programming language for sound synthesis and computer music creation

Vizualni programski jezici često su zbog svojih paradigmi i značajki prilagođeni stručnjacima u neračunarskim područjima. Osmišljeni su tako da omoguće izražavanje vizualnim simbolima bliskima domeni za koju su namijenjeni, da budu jednostavni za upotrebu i da sakrivaju tehničke aspekte jezika od programera. U tom su smislu posebno zanimljivi i često korišteni vizualni jezici u području glazbe. Postojeći vizualni jezici u domeni glazbe koriste audio signale, odnosno matematičkim formulacijama opisan zvuk kao temeljni tip podataka. Međutim, rad s audio signalima zahtijeva poznavanje obrade signala i programerske vještine kako bi se na željeni način moglo njima upravljati. Glazbenicima ovo otežava korištenje jezika. U radu se stoga predlaže jezik koji se oslanja na koncepte timbralnih atributa kako bi se približio vokabularu i predznanju glazbenika. Jezik pritom pokušava zadržati osnovni koncept rada koji je glazbenicima poznat iz postojećih jezika. Kako bi se ostvario novi jezik, istraženi su različiti postupci preslikavanja timbralnih atributa u parametre audio sintetizatora. Zatim su timbralni atributi uklopljeni u tok podataka jezika sličan onome kod postojećih rješenja. Većina uobičajenih blokova i funkcionalnosti takvih jezika prilagođena je za rad s timbralnim atributima. Prilikom oblikovanja, validacije i evaluacije predloženog jezika korištene su metode kvantitativne i kvalitativne analize pomoću anketa te formalne analize pomoću kognitivnih dimenzija. Dodatno, predložena su specifična poboljšanja sučelja blokova u jeziku tako da olakšaju manipulaciju timbralnim atributima. Predstavljena su i rješenja za vizualizaciju rezultata programa i izlaznog zvuka u domeni timbralnih atributa. Konačno, predloženo je rješenje za vizualno uspoređivanja programa čiji je cilj lakše praćenje promjena među inačicama programa kao i otkrivanje pogrešaka.The paradigms and features of visual programming languages are often based on the needs of experts in fields outside computer science. They enable programming using visual symbols closely related to specific domains. Visual languages are prominent in the field of music. Existing visual languages for music use audio signals as their main data type. The manipulation of audio signals requires understanding of signal processing and programming which inhibits musicians from using these languages. This PhD thesis thus proposes a language that employs timbral attributes to bring programming closer to the vocabulary and knowledge of musicians. Simultaneously, the new language maintains existing best practices and basic principles. To model the new language, algorithms that translate timbral attributes into sound synthesis parameters were researched first. Then, timbral attributes were included in the language's data flow. Most of the usual objects and functionalities were adapted to work with timbral attributes. The process of modeling, validating, and evaluating the language was driven by quantitative and qualitative analysis based on questionnaires and formal analysis based on the cognitive dimensions framework. Improvements to object's visual interfaces are also introduced to simplify manipulation of timbral attributes. Similarly, methods of audio signal visualization in the domain of timbral attributes are proposed. Finally, a solution for visual comparisons of programs, that simplifies change tracking and error detection, is demonstrated. Scientific contributions: 1.Visual programming language for sound synthesis based on attribute flow and for computer music creation based on time-domain manipulations. 2.Models of exploration and visualization of sound synthesis results and for the intuitive control of a sound synthesizer and its parameters. 3. Improved and domain-adjusted methods for visual programs comparison

Visual programming language for sound synthesis and computer music creation

Vizualni programski jezici često su zbog svojih paradigmi i značajki prilagođeni stručnjacima u neračunarskim područjima. Osmišljeni su tako da omoguće izražavanje vizualnim simbolima bliskima domeni za koju su namijenjeni, da budu jednostavni za upotrebu i da sakrivaju tehničke aspekte jezika od programera. U tom su smislu posebno zanimljivi i često korišteni vizualni jezici u području glazbe. Postojeći vizualni jezici u domeni glazbe koriste audio signale, odnosno matematičkim formulacijama opisan zvuk kao temeljni tip podataka. Međutim, rad s audio signalima zahtijeva poznavanje obrade signala i programerske vještine kako bi se na željeni način moglo njima upravljati. Glazbenicima ovo otežava korištenje jezika. U radu se stoga predlaže jezik koji se oslanja na koncepte timbralnih atributa kako bi se približio vokabularu i predznanju glazbenika. Jezik pritom pokušava zadržati osnovni koncept rada koji je glazbenicima poznat iz postojećih jezika. Kako bi se ostvario novi jezik, istraženi su različiti postupci preslikavanja timbralnih atributa u parametre audio sintetizatora. Zatim su timbralni atributi uklopljeni u tok podataka jezika sličan onome kod postojećih rješenja. Većina uobičajenih blokova i funkcionalnosti takvih jezika prilagođena je za rad s timbralnim atributima. Prilikom oblikovanja, validacije i evaluacije predloženog jezika korištene su metode kvantitativne i kvalitativne analize pomoću anketa te formalne analize pomoću kognitivnih dimenzija. Dodatno, predložena su specifična poboljšanja sučelja blokova u jeziku tako da olakšaju manipulaciju timbralnim atributima. Predstavljena su i rješenja za vizualizaciju rezultata programa i izlaznog zvuka u domeni timbralnih atributa. Konačno, predloženo je rješenje za vizualno uspoređivanja programa čiji je cilj lakše praćenje promjena među inačicama programa kao i otkrivanje pogrešaka.The paradigms and features of visual programming languages are often based on the needs of experts in fields outside computer science. They enable programming using visual symbols closely related to specific domains. Visual languages are prominent in the field of music. Existing visual languages for music use audio signals as their main data type. The manipulation of audio signals requires understanding of signal processing and programming which inhibits musicians from using these languages. This PhD thesis thus proposes a language that employs timbral attributes to bring programming closer to the vocabulary and knowledge of musicians. Simultaneously, the new language maintains existing best practices and basic principles. To model the new language, algorithms that translate timbral attributes into sound synthesis parameters were researched first. Then, timbral attributes were included in the language's data flow. Most of the usual objects and functionalities were adapted to work with timbral attributes. The process of modeling, validating, and evaluating the language was driven by quantitative and qualitative analysis based on questionnaires and formal analysis based on the cognitive dimensions framework. Improvements to object's visual interfaces are also introduced to simplify manipulation of timbral attributes. Similarly, methods of audio signal visualization in the domain of timbral attributes are proposed. Finally, a solution for visual comparisons of programs, that simplifies change tracking and error detection, is demonstrated. Scientific contributions: 1.Visual programming language for sound synthesis based on attribute flow and for computer music creation based on time-domain manipulations. 2.Models of exploration and visualization of sound synthesis results and for the intuitive control of a sound synthesizer and its parameters. 3. Improved and domain-adjusted methods for visual programs comparison

Visual programming language for sound synthesis and computer music creation

Vizualni programski jezici često su zbog svojih paradigmi i značajki prilagođeni stručnjacima u neračunarskim područjima. Osmišljeni su tako da omoguće izražavanje vizualnim simbolima bliskima domeni za koju su namijenjeni, da budu jednostavni za upotrebu i da sakrivaju tehničke aspekte jezika od programera. U tom su smislu posebno zanimljivi i često korišteni vizualni jezici u području glazbe. Postojeći vizualni jezici u domeni glazbe koriste audio signale, odnosno matematičkim formulacijama opisan zvuk kao temeljni tip podataka. Međutim, rad s audio signalima zahtijeva poznavanje obrade signala i programerske vještine kako bi se na željeni način moglo njima upravljati. Glazbenicima ovo otežava korištenje jezika. U radu se stoga predlaže jezik koji se oslanja na koncepte timbralnih atributa kako bi se približio vokabularu i predznanju glazbenika. Jezik pritom pokušava zadržati osnovni koncept rada koji je glazbenicima poznat iz postojećih jezika. Kako bi se ostvario novi jezik, istraženi su različiti postupci preslikavanja timbralnih atributa u parametre audio sintetizatora. Zatim su timbralni atributi uklopljeni u tok podataka jezika sličan onome kod postojećih rješenja. Većina uobičajenih blokova i funkcionalnosti takvih jezika prilagođena je za rad s timbralnim atributima. Prilikom oblikovanja, validacije i evaluacije predloženog jezika korištene su metode kvantitativne i kvalitativne analize pomoću anketa te formalne analize pomoću kognitivnih dimenzija. Dodatno, predložena su specifična poboljšanja sučelja blokova u jeziku tako da olakšaju manipulaciju timbralnim atributima. Predstavljena su i rješenja za vizualizaciju rezultata programa i izlaznog zvuka u domeni timbralnih atributa. Konačno, predloženo je rješenje za vizualno uspoređivanja programa čiji je cilj lakše praćenje promjena među inačicama programa kao i otkrivanje pogrešaka.The paradigms and features of visual programming languages are often based on the needs of experts in fields outside computer science. They enable programming using visual symbols closely related to specific domains. Visual languages are prominent in the field of music. Existing visual languages for music use audio signals as their main data type. The manipulation of audio signals requires understanding of signal processing and programming which inhibits musicians from using these languages. This PhD thesis thus proposes a language that employs timbral attributes to bring programming closer to the vocabulary and knowledge of musicians. Simultaneously, the new language maintains existing best practices and basic principles. To model the new language, algorithms that translate timbral attributes into sound synthesis parameters were researched first. Then, timbral attributes were included in the language's data flow. Most of the usual objects and functionalities were adapted to work with timbral attributes. The process of modeling, validating, and evaluating the language was driven by quantitative and qualitative analysis based on questionnaires and formal analysis based on the cognitive dimensions framework. Improvements to object's visual interfaces are also introduced to simplify manipulation of timbral attributes. Similarly, methods of audio signal visualization in the domain of timbral attributes are proposed. Finally, a solution for visual comparisons of programs, that simplifies change tracking and error detection, is demonstrated. Scientific contributions: 1.Visual programming language for sound synthesis based on attribute flow and for computer music creation based on time-domain manipulations. 2.Models of exploration and visualization of sound synthesis results and for the intuitive control of a sound synthesizer and its parameters. 3. Improved and domain-adjusted methods for visual programs comparison

Desirable Aspects of Visual Programming Languages for Different Applications in Music Creation

(Abstract to follow