Collaborating with the Behaving Machine: simple adaptive dynamical systems for generative and interactive music

Situated at the intersection of interactive computer music and generative art, this thesis is inspired by research in Artificial Life and Autonomous Robotics and applies some of the principles and methods of these fields in a practical music context. As such the project points toward a paradigm for computer music research and performance which comple- ments current mainstream approaches and develops upon existing creative applications of Artificial Life research. Many artists have adopted engineering techniques from the field of Artificial Life research as they seem to support a richer interactive experience with computers than is often achieved in digital interactive art. Moreover, the low level aspects of life which the research programme aims to model are often evident in these artistic appropriations in the form of bizarre and abstract but curiously familiar digital forms that somehow, despite their silicon make-up, appear to accord with biological convention. The initial aesthetic motivation for this project was very personal and stemmed from interests in adaptive systems and improvisation and a desire to unite the two. In sim- ple terms, I wanted to invite these synthetic critters up on stage and play with them. There has been some similar research in the musical domain, but this has focused on a very small selection of specific models and techniques which have been predominantly applied as compositional tools rather than for use in live generative music. This thesis considers the advantages of the Alife approach for contemporary computer musicians and offers specific examples of simple adaptive systems as components for both compo- sitional and performance tools. These models have been implemented in a range of generative and interactive works which are described here. These include generative sound installations, interactive instal- lations and a performance system for collaborative man-machine improvisation. Public response at exhibitions and concerts suggests that the approach taken here holds much promise

Parameter Search for Aesthetic Design and Composition

PhDThis thesis is about algorithmic creation in the arts – where an artist, designer or composer uses a formal generative process to assist in crafting forms and patterns – and approaches to finding effective input parameter values to these generative processes for aesthetic ends. Framed in three practical studies, approaches to navigating the aesthetic possibilities of generative processes in sound and visuals are presented, and strategies for eliciting the preferences of the consumers of the generated output are explored. The first study presents a musical interface that enables navigation of the possibilities of a stochastic generative process with respect to measures of subjective predictability. Through a mobile phone version of the application, aesthetic preferences are crowd-sourced. The second study presents an eye-tracking based framework for the exploration of the possibilities afforded by generative designs; the interaction between the viewers’ gaze patterns and the system engendering a fluid navigation of the state-space of the visual forms. The third study presents a crowd-sourced interactive evolutionary system, where populations of abstract colour images are shaped by thousands of preference selections from users worldwide For each study, the results of analyses eliciting the attributes of the generated outputs – and their associated parameter values – that are most preferred by the consumers/users of these systems are presented. Placed in a historical and theoretical context, a refined perspective on the complex interrelationships between generative processes, input parameters and perceived aesthetic value is presented. Contributions to knowledge include identified trends in objective aesthetic preferences in colour combinations and their arrangements, theoretical insights relating perceptual mechanisms to generative system design and analysis, strategies for effectively leveraging evolutionary computation in an empirical aesthetic context, and a novel eye-tracking based framework for the exploration of visual generative designs.Engineering and Physical Sciences Research Council (EPSRC) as part of the Doctoral Training Centre in Media and Arts Technology at Queen Mary University of London (ref: EP/G03723X/1)

Content-based retrieval of melodies using artificial neural networks

Human listeners are capable of spontaneously organizing and remembering a continuous stream of musical notes. A listener automatically segments a melody into phrases, from which an entire melody may be learnt and later recognized. This ability makes human listeners ideal for the task of retrieving melodies by content. This research introduces two neural networks, known as SONNETMAP and _ReTREEve, which attempt to model this behaviour. SONNET-MAP functions as a melody segmenter, whereas ReTREEve is specialized towards content-based retrieval (CBR). Typically, CBR systems represent melodies as strings of symbols drawn from a finite alphabet, thereby reducing the retrieval process to the task of approximate string matching. SONNET-MAP and ReTREEwe, which are derived from Nigrin’s SONNET architecture, offer a novel approach to these traditional systems, and indeed CBR in general. Based on melodic grouping cues, SONNETMAP segments a melody into phrases. Parallel SONNET modules form independent, sub-symbolic representations of the pitch and rhythm dimensions of each phrase. These representations are then bound using associative maps, forming a two-dimensional representation of each phrase. This organizational scheme enables SONNET-MAP to segment melodies into phrases using both the pitch and rhythm features of each melody. The boundary points formed by these melodic phrase segments are then utilized to populate the iieTREEve network. ReTREEw is organized in the same parallel fashion as SONNET-MAP. However, in addition, melodic phrases are aggregated by an additional layer; thus forming a two-dimensional, hierarchical memory structure of each entire melody. Melody retrieval is accomplished by matching input queries, whether perfect (for example, a fragment from the original melody) or imperfect (for example, a fragment derived from humming), against learned phrases and phrase sequence templates. Using a sample of fifty melodies composed by The Beatles , results show th a t the use of both pitch and rhythm during the retrieval process significantly improves retrieval results over networks that only use either pitch o r rhythm. Additionally, queries that are aligned along phrase boundaries are retrieved using significantly fewer notes than those that are not, thus indicating the importance of a human-based approach to melody segmentation. Moreover, depending on query degradation, different melodic features prove more adept at retrieval than others. The experiments presented in this thesis represent the largest empirical test of SONNET-based networks ever performed. As far as we are aware, the combined SONNET-MAP and -ReTREEue networks constitute the first self-organizing CBR system capable of automatic segmentation and retrieval of melodies using various features of pitch and rhythm