Automatic sound synthesizer programming: techniques and applications

The aim of this thesis is to investigate techniques for, and applications of automatic sound synthesizer programming. An automatic sound synthesizer programmer is a system which removes the requirement to explicitly specify parameter settings for a sound synthesis algorithm from the user. Two forms of these systems are discussed in this thesis: tone matching programmers and synthesis space explorers. A tone matching programmer takes at its input a sound synthesis algorithm and a desired target sound. At its output it produces a configuration for the sound synthesis algorithm which causes it to emit a similar sound to the target. The techniques for achieving this that are investigated are genetic algorithms, neural networks, hill climbers and data driven approaches. A synthesis space explorer provides a user with a representation of the space of possible sounds that a synthesizer can produce and allows them to interactively explore this space. The applications of automatic sound synthesizer programming that are investigated include studio tools, an autonomous musical agent and a self-reprogramming drum machine. The research employs several methodologies: the development of novel software frameworks and tools, the examination of existing software at the source code and performance levels and user trials of the tools and software. The main contributions made are: a method for visualisation of sound synthesis space and low dimensional control of sound synthesizers; a general purpose framework for the deployment and testing of sound synthesis and optimisation algorithms in the SuperCollider language sclang; a comparison of a variety of optimisation techniques for sound synthesizer programming; an analysis of sound synthesizer error surfaces; a general purpose sound synthesizer programmer compatible with industry standard tools; an automatic improviser which passes a loose equivalent of the Turing test for Jazz musicians, i.e. being half of a man-machine duet which was rated as one of the best sessions of 2009 on the BBC's 'Jazz on 3' programme

Self-Organised Music

Self-organisation, as manifest, for example, by swarms, flock, herds and other collectives, is a powerful natural force, capable of generating large and sustained structures. Yet the individuals who participate in these social groups may not even be aware of the structures that they are creating. Almost certainly, these structures emerge through the application of simple, local interactions. Improvised music is an uncertain activity, characterised by a lack of top-down organisation and busy, local activity between improvisers. Emerging structures may only be perceivable at a (temporal) distance. The development of higher-level musical structure arises from interactions at lower levels, and we propose here that the self-organisation of social animals provides a very suggestive analogy. This paper builds a model of interactivity based on stigmergy, the process by which social insects communicate indirectly by environment modification. The improvisational element of our model arises from the dynamics of a particle swarm. A process called interpretation extracts musical parameters from the aural sound environment, and uses these parameters to place attractors in the environment of the swarm, after which stigmergy can take place. The particle positions are reinterpreted as parameterised audio events. This paper describes this model and two applications, Swarm Music and Swarm Granulator

Musical Interpretation In Improvised Human-Machine Performance

The development of interactive performance systems is an active area of research in the field of live electronic music. Whilst various models and metaphors of interactivity have been proposed in the literature, the engagement of these systems in improvised performance remains under-researched. This paper explores the notion of musical interpretation in improvised human-machine performance practice from the perspective of a performer-developer. Through a consideration of the notion of the musical text, these creative artefacts and the performance practices they engender are situated within the context of interpretive musical practice. I argue that musical performances with these software systems may be seen as an instantiation of the combined musical ideas of the system developer, the musician navigating this space of ideas, and the live and interactive contributions of a machine to the performance. The paper concludes that the development of interactive software is akin to the creation a form of musical text

Computational composition strategies in audiovisual laptop performance

We live in a cultural environment in which computer based musical performances have become ubiquitous. Particularly the use of laptops as instruments is a thriving practice in many genres and subcultures. The opportunity to command the most intricate level of control on the smallest of time scales in music composition and computer graphics introduces a number of complexities and dilemmas for the performer working with algorithms. Writing computer code to create audiovisuals offers abundant opportunities for discovering new ways of expression in live performance while simultaneously introducing challenges and presenting the user with difficult choices. There are a host of computational strategies that can be employed in live situations to assist the performer, including artificially intelligent performance agents who operate according to predefined algorithmic rules. This thesis describes four software systems for real time multimodal improvisation and composition in which a number of computational strategies for audiovisual laptop performances is explored and which were used in creation of a portfolio of accompanying audiovisual compositions

Artificial Intelligence Music Generators in Real Time Jazz Improvisation: a performer’s view

Μια αμφιλεγόμενη είσοδος γεννητριών μουσικής τεχνητής νοημοσύνης στον κόσμο της μουσικής σύνθεσης και ερμηνείας καλπάζει επί του παρόντος. Γόνιμη έρευνα που πηγάζει απο τομείς όπως η ανάκτηση πληροφοριών μουσικής, τα νευρονικά δίκτυα και η βαθιά μάθηση, μεταξύ άλλων, διαμορφώνει αυτό το μέλλον. Ενσωματωμένα και μη ενσωματωμένα συστήματα τεχνητής νοημοσύνης έχουν εισέλθει στον κόσμο της τζαζ προκειμένου να συνδημιουργήσουν ιδιωματικούς μουσικούς αυτοσχεδιασμούς. Αυτή η διπλωματική εξετάζει τους προκύπτοντες μελωδικούς αυτοσχεδιασμούς που παράγονται από τις γεννήτριες OMax, ImproteK και Djazz (OID) μέσω του φακού των στοιχείων της μουσικής και το κάνει από την άποψη ενός ερμηνευτή. Η ανάλυση βασίζεται κυρίως στην αξιολόγηση των ήδη δημοσιευμένων αποτελεσμάτων, καθώς και σε μια μελέτη περίπτωσης που πραγματοποίηθηκε κατά την ολοκλήρωση αυτής της εργασίας που περιλαμβάνει την απόδοση, την ακρόαση και την αξιολόγηση των παραγόμενων αυτοσχεδιασμών του OMax. Επίσης, η εργασία ασχολείται με φιλοσοφικά ζητήματα, με τα γνωστικά θεμέλια του συναισθήματος και του νοήματος και παρέχει μια ολοκληρωμένη ανάλυση της λειτουργικότητας του OID.A highly controversial entrance of Artificial Intelligence (AI) music generators in the world of music composition and performance is currently advancing. A fruitful research from Music Information Retrieval, Neural Networks and Deep Learning, among other areas, are shaping this future. Embodied and non-embodied AI systems have stepped into the world of jazz in order to co-create idiomatic music improvisations. But how musical these improvisations are? This dissertation looks at the resulted melodic improvisations produced by OMax, ImproteK and Djazz (OID) AI generators through the lens of the elements of music and it does so from a performer’s point of view. The analysis is based mainly on the evaluation of already published results as well as on a case study I carried out during the completion of this essay which includes performance, listening and evaluation of generated improvisations of OMax. The essay also reflects upon philosophical issues, cognitive foundations of emotion and meaning and provides a comprehensive analysis of the functionality of OID

Actions towards freedom: Theoretical and practical perspectives on improvisation and composition

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University London.This thesis, and the accompanying portfolio of pieces, is concerned with investigating practical and theoretical meeting points between improvisation and composition. Such meeting points are evaluated alongside a consideration of ‘freedom’ in improvised music, for which a frame is drawn from George Lewis’s concepts of the ‘Afrological’ (placing emphasis on expression of the ‘self’) and ‘Eurological’ (in which the ‘self’ is explicitly avoided). It is suggested that a reconciliation of these two extremes might be found in a compositional ‘creative displacement’, which might change an improviser’s environment in unforeseen ways and thus stimulate explorations of expressive novelty. Three different compositional approaches to ‘creative displacement’ are investigated: through fixed notation, through electronic real-time notation, and through leadership in a workshop setting. In each case compositional experiments will be undertaken and documented, detailing the creation and realisation of the pieces included in the accompanying portfolio. A terminology for the theoretical consideration of these approaches will draw on theories of complex systems, the philosophy of Gilles Deleuze, and various socio-musicological models such as those of Steven Feld and Charles Keil. Through an evaluation of the portfolio compositions in rehearsal and performance, this thesis will conclude that a reconciliation of Lewis’s ‘Afro’ and ‘Eurological’ can be found through the external application of limitations to improvisational creativity. Such constraints will be described as ‘creatively displacing’ if they provoke a performer towards an exploration of novel expressive approaches. In order to achieve this in practice, limitations must be carefully judged with regard to their degree of abstraction, the manner of their presentation and the nature of their notation; it will be suggested that the presence of a leader is vital in achieving this. These conclusions will lead to a questioning of conventional ideas of improvisation and leadership, and suggest a re-evaluation of indeterminacy within notation

Augmenting Percussion with Electronics in Improvised Music Performance

This commentary augments audio and video recordings that should be considered the essence of the study. The sound recordings comprise an original body of work that resulted from my interest in extending the possibilities of a standard drum set by augmenting it with electronics. It developed from an early interest in analogue, electroacoustic devices – such as the Dexion frames used by Tony Oxley and Paul Lytton to an engagement with digital electronics, specifically Max MSP, that was unknown to me at the outset of the study. The digital tools caused me to re-evaluate my thinking; to go beyond extending the sound-world at my disposal to engage with and consider artificial intelligence and the potential of creating a surrogate, software improviser with a degree of agency that challenged my thinking about human-computer interaction and confounded the issue of whether I was playing in a solo or duo setting. The commentary demonstrates the centrality of free improvisation to my approach and the recordings document my use of technologies, varying from the seemingly primitive (wooden beaters) to the apparently sophisticated (Max MSP) where I fully explore the affordances of each encounter

Explaining Listener Differences in the Perception of Musical Structure

PhDState-of-the-art models for the perception of grouping structure in music do not attempt to account for disagreements among listeners. But understanding these disagreements, sometimes regarded as noise in psychological studies, may be essential to fully understanding how listeners perceive grouping structure. Over the course of four studies in different disciplines, this thesis develops and presents evidence to support the hypothesis that attention is a key factor in accounting for listeners' perceptions of boundaries and groupings, and hence a key to explaining their disagreements. First, we conduct a case study of the disagreements between two listeners. By studying the justi cations each listener gave for their analyses, we argue that the disagreements arose directly from differences in attention, and indirectly from differences in information, expectation, and ontological commitments made in the opening moments. Second, in a large-scale corpus study, we study the extent to which acoustic novelty can account for the boundary perceptions of listeners. The results indicate that novelty is correlated with boundary salience, but that novelty is a necessary but not su cient condition for being perceived as a boundary. Third, we develop an algorithm that optimally reconstructs a listener's analysis in terms of the patterns of similarity within a piece of music. We demonstrate how the output can identify good justifications for an analysis and account for disagreements between two analyses. Finally, having introduced and developed the hypothesis that disagreements between listeners may be attributable to differences in attention, we test the hypothesis in a sequence of experiments. We find that by manipulating the attention of participants, we are able to influence the groupings and boundaries they find most salient. From the sum of this research, we conclude that a listener's attention is a crucial factor affecting how listeners perceive the grouping structure of music.Social Sciences and Humanities Research Council; a PhD studentship from Queen Mary University of London; a Provost's Ph.D. Fellowship from the University of Southern California. This material is also based in part on work supported by the National Science Foundation under Grant No. 0347988