AN APPROACH TO MACHINE DEVELOPMENT OF MUSICAL ONTOGENY

This Thesis pursues three main objectives: (i) to use computational modelling to explore how music is perceived, cognitively processed and created by human beings; (ii) to explore interactive musical systems as a method to model and achieve the transmission of musical influence in artificial worlds and between humans and machines; and (iii) to experiment with artificial and alternative developmental musical routes in order to observe the evolution of musical styles. In order to achieve these objectives, this Thesis introduces a new paradigm for the design of computer interactive musical systems called the Ontomemetical Model of Music Evolution - OMME, which includes the fields of musical ontogenesis and memetlcs. OMME-based systems are designed to artificially explore the evolution of music centred on human perceptive and cognitive faculties. The potential of the OMME is illustrated with two interactive musical systems, the Rhythmic Meme Generator (RGeme) and the Interactive Musical Environments (iMe). which have been tested in a series of laboratory experiments and live performances. The introduction to the OMME is preceded by an extensive and critical overview of the state of the art computer models that explore musical creativity and interactivity, in addition to a systematic exposition of the major issues involved in the design and implementation of these systems. This Thesis also proposes innovative solutions for (i) the representation of musical streams based on perceptive features, (ii) music segmentation, (iii) a memory-based music model, (iv) the measure of distance between musical styles, and (v) an impi*ovisation-based creative model

Computational composition strategies in audiovisual laptop performance

We live in a cultural environment in which computer based musical performances have become ubiquitous. Particularly the use of laptops as instruments is a thriving practice in many genres and subcultures. The opportunity to command the most intricate level of control on the smallest of time scales in music composition and computer graphics introduces a number of complexities and dilemmas for the performer working with algorithms. Writing computer code to create audiovisuals offers abundant opportunities for discovering new ways of expression in live performance while simultaneously introducing challenges and presenting the user with difficult choices. There are a host of computational strategies that can be employed in live situations to assist the performer, including artificially intelligent performance agents who operate according to predefined algorithmic rules. This thesis describes four software systems for real time multimodal improvisation and composition in which a number of computational strategies for audiovisual laptop performances is explored and which were used in creation of a portfolio of accompanying audiovisual compositions

Beyond the Electronic Connection: The Technologically Manufactured Cyber-Human and Its Physical Human Counterpart in Performance: A Theory Related to Convergence Identities

This thesis is an investigation of the complex processes and relationships between the physical human performer and the technologically manufactured cyber-human counterpart. I acted as both researcher and the physical human performer, deeply engaged in the moment-to-moment creation of events unfolding within a shared virtual reality environment. As the primary instigator and activator of the cyber-human partner, I maintained a balance between the live and technological performance elements, prioritizing the production of content and meaning. By way of using practice as research, this thesis argues that in considering interactions between cyber-human and human performers, it is crucial to move beyond discussions of technology when considering interactions between cyber-humans and human performers to an analysis of emotional content, the powers of poetic imagery, the trust that is developed through sensory perception and the evocation of complex relationships. A theoretical model is constructed to describe the relationship between a cyber-human and a human performer in the five works created specifically for this thesis, which is not substantially different from that between human performers. Technological exploration allows for the observation and analysis of various relationships, furthering an expanded understanding of ‘movement as content’ beyond the electronic connection. Each of the works created for this research used new and innovative technologies, including virtual reality, multiple interactive systems, six generations of wearable computers, motion capture technology, high-end digital lighting projectors, various projection screens, smart electronically charged fabrics, multiple sensory sensitive devices and intelligent sensory charged alternative performance spaces. They were most often collaboratively created in order to augment all aspects of the performance and create the sense of community found in digital live dance performances/events. These works are identified as one continuous line of energy and discovery, each representing a slight variation on the premise that a working, caring, visceral and poetic content occurs beyond the technological tools. Consequently, a shift in the physical human’s psyche overwhelms the act of performance. Scholarship and reflection on the works have been integral to my creative process throughout. The goals of this thesis, the works created and the resulting methodologies are to investigate performance to heighten the multiple ways we experience and interact with the world. This maximizes connection and results in a highly interactive, improvisational, dynamic, non-linear, immediate, accessible, agential, reciprocal, emotional, visceral and transformative experience without boundaries between the virtual and physical for physical humans, cyborgs and cyber-humans alike.College of Fine Arts at the University of Texas at Austin, Department of Theatre & Dance at the University of Texas at Austi

Investigating the cognitive foundations of collaborative musical free improvisation: Experimental case studies using a novel application of the subsumption architecture

This thesis investigates the cognitive foundations of collaborative musical free improvisation. To explore the cognitive underpinnings of the collaborative process, a series of experimental case studies was undertaken in which expert improvisors performed with an artificial agent. The research connects ecological musicology and subsumption robotics, and builds upon insights from empirical psychology pertaining to the attribution of intentionality. A distinguishing characteristic of free improvisation is that no over-arching framework of formal musical conventions defines it, and it cannot be positively identified by sound alone, which poses difficulties for traditional musicology. Current musicological research has begun to focus on the social dimension of music, including improvisation. Ecological psychology, which focuses on the relation of cognition to agent–environment dynamics using the notion of affordances, has been shown to be a promising approach to understanding musical improvisation. This ecological approach to musicology makes it possible to address the subjective and social aspects of improvised music, as opposed to the common treatment of music as objective and neutral. The subjective dimension of musical listening has been highlighted in music cognition studies of cue abstraction, whereby listeners perceive emergent structures while listening to certain forms of music when no structures are identified in advance. These considerations informed the design of the artificial agent, Odessa, used for this study. In contrast to traditional artificial intelligence (AI), which tends to view the world as objective and neutral, behaviour-based robotics historically developed around ideas similar to those of ecological psychology, focused on agent–environment dynamics and the ability to deal with potentially rapidly changing environments. Behaviour-based systems that are designed using the subsumption architecture are robust and flexible in virtue of their modular, decentralised design comprised of simple interactions between simple mechanisms. The competence of such agents is demonstrated on the basis of their interaction with the environment and ability to cope with unknown and dynamic conditions, which suggests the concept of improvisation. This thesis documents a parsimonious subsumption design for an agent that performs musical free improvisation with human co-performers, as well as the experimental studies conducted with this agent. The empirical component examines the human experience of collaborating with the agent and, more generally, the cognitive psychology of collaborative improvisation. The design was ultimately successful, and yielded insights about cognition in collaborative improvisation, in particular, concerning the central relationship between perceived intentionality and affordances. As a novel application of the subsumption architecture, this research contributes to AI/robotics and to research on interactive improvisation systems. It also contributes to music psychology and cognition, as well as improvisation studies, through its empirical grounding of an ecological model of musical interaction

Score following: An artificially intelligent musical accompanist

Score Following is the process by which a musician can be tracked through their performance of a piece, for the purpose of accompanying the musician with the appropriate notes. This tracking is done by following the progress of the musician through the score (written music) of the piece, using observations of the notes they are playing. Artificially intelligent musical accompaniment is where a human musician is accompanied by a computer musician. The computer musician is able to produce musical accompaniment that relates musically to the human performance. Hidden Markov Models (HMMs) are a stochastic modelling tool that can be used to represent real-world systems in a variety of domains. This project discusses how HMMs can be used in the domain of Score Following and describes the construction and evaluation of a score following system that uses HMMs to implement score following. It explores the hypothesis that using an HMM to represent a musical score is an efficient and practical way to implement score following, and that in particular this method is suitable for providing real-time accompaniment to a human performer. The score followers developed during this project are tested and compared against other score following systems and against human musicians. The resulting performances support the project hypothesis to a large extent

Towards Machine Musicians Who Have Listened to More Music Than Us: Audio Database-led Algorithmic Criticism for Automatic Composition and Live Concert Systems

Databases of audio can form the basis for new algorithmic critic systems, applying techniques from the growing field of music information retrieval to meta-creation in algorithmic composition and interactive music systems. In this article, case studies are described where critics are derived from larger audio corpora. In the first scenario, the target music is electronic art music, and two corpuses are used to train model parameters and then compared with each other and against further controls in assessing novel electronic music composed by a separate program. In the second scenario, a “real-world” application is described, where a “jury” of three deliberately and individually biased algorithmic music critics judged the winner of a dubstep remix competition. The third scenario is a live tool for automated in-concert criticism, based on the limited situation of comparing an improvising pianists' playing to that of Keith Jarrett; the technology overlaps that described in the other systems, though now deployed in real time. Alongside description and analysis of these systems, the wider possibilities and implications are discussed

PLXTRM : prediction-led eXtended-guitar tool for real-time music applications and live performance

peer reviewedThis article presents PLXTRM, a system tracking picking-hand micro-gestures for real-time music applications and live performance. PLXTRM taps into the existing gesture vocabulary of the guitar player. On the first level, PLXTRM provides a continuous controller that doesn’t require the musician to learn and integrate extrinsic gestures, avoiding additional cognitive load. Beyond the possible musical applications using this continuous control, the second aim is to harness PLXTRM’s predictive power. Using a reservoir network, string onsets are predicted within a certain time frame, based on the spatial trajectory of the guitar pick. In this time frame, manipulations to the audio signal can be introduced, prior to the string actually sounding, ’prefacing’ note onsets. Thirdly, PLXTRM facilitates the distinction of playing features such as up-strokes vs. down-strokes, string selections and the continuous velocity of gestures, and thereby explores new expressive possibilities

INSAM Journal of Contemporary Music, Art and Technology 2

The subject of machine learning and creativity, as well as its appropriation in arts is the focus of this issue with our Main theme of – Artificial Intelligence in Music, Arts, and Theory. In our invitation to collaborators, we discussed our standing preoccupation with the exploration of technology in contemporary theory and artistic practice. The invitation also noted that this time we are encouraged and inspired by Catherine Malabou’s new observations regarding brain plasticity and the metamorphosis of (natural and artificial) intelligence. Revising her previous stance that the difference between brain plasticity and computational architecture is not authentic and grounded, Malabou admits in her new book, Métamorphoses de l'intelligence: Que faire de leur cerveau bleu? (2017), that plasticity – the potential of neuron architecture to be shaped by environment, habits, and education – can also be a feature of artificial intelligence. “The future of artificial intelligence,” she writes, “is biological.” We wanted to provoke a debate about what machines can learn and what we can learn from them, especially regarding contemporary art practices. On this note, I am happy to see that our proposition has provoked intriguing and unique responses from various different disciplines including: theory of art, aesthetics of music, musicology, and media studies. The pieces in the (Inter)view section deal with machine and computational creativity, as well as the some of the principles of contemporary art. Reviews give us an insight into a couple of relevant reading points for this discussion and a retrospective of one engaging festival that also fits this theme