User serviceable parts: Practice, technology, sociality and method in live electronic musicking

In live electronic musical research there is a need to confront the interrelationships between the social and the technological in order to understand our music as practice. These interrelationships form a complex and dynamic ecosystem that not only forms the context to, but is constitutive of practice. I interrogate from a variety of perspectives the musical practice that has formed over the course of this research in order to reveal the dispositions towards technology, the social situatedness and the musical approach that underlies my work. By taking a disposition towards musical practice-led research that is non-hierarchical, performative, ecological, phenomenological and pragmatic, I place into wider context compositional and technological decisions, in terms of their relationships to improvising, skill, design, performance and research. This work contributes both new theories of live electronic musical practice and new suggestions for practice-led methods aimed at investigating the interplay of social and material factors in musicking, and at interrogating the disciplinary status of our field vis-a-vis musical and technical disciplines

Improvisation, Computers, and Interaction : Rethinking Human-Computer Interaction Through Music

Interaction is an integral part of all music. Interaction is part of listening, of playing, of composing and even of thinking about music. In this thesis the multiplicity of modes in which one may engage interactively in, through and with music is the starting point for rethinking Human-Computer Interaction in general and Interactive Music in particular. I propose that in Human-Computer interaction the methodology of control, interaction-as-control, in certain cases should be given up in favor for a more dynamic and reciprocal mode of interaction, interaction-as-difference: Interaction as an activity concerned with inducing differences that make a difference. Interaction-as-difference suggests a kind of parallelity rather than click-and-response. In essence, the movement from control to difference was a result of rediscovering the power of improvisation as a method for organizing and constructing musical content and is not to be understood as an opposition: It is rather a broadening of the more common paradigm of direct manipulation in Human-Computer Interaction. Improvisation is at the heart of all the sub-projects included in this thesis, also, in fact, in those that are not immediately related to music but more geared towards computation. Trusting the self-organizing aspect of musical improvisation, and allowing it to diffuse into other areas of my practice, constitutes the pivotal change that has radically influenced my artistic practice. Furthermore, is the work-in-movement (re-)introduced as a work kind that encompasses radically open works. The work-in-movement, presented and exemplified by a piece for guitar and computer, requires different modes of representation as the traditional musical score is too restrictive and is not able to communicate that which is the most central aspect: the collaboration, negotiation and interaction. The Integra framework and the relational database model with its corresponding XML representation is proposed as a means to produce annotated scores that carry past performances and version with it. The common nominator, the prerequisite, for interaction-as-difference and a improvisatory and self-organizing attitude towards musical practice it the notion of giving up of the Self. Only if the Self is able and willing to accept the loss the priority of interpretation (as for the composer) or the faithfulness to ideology or idiomatics (performer). Only is one is willing to forget is interaction-as-difference made possible. Among the artistic works that have been produced as part of this inquiry are some experimental tools in the form of computer software to support the proposed concepts of interactivity. These, along with the more traditional musical work make up both the object and the method in this PhD project. These sub-projects contained within the frame of the thesis, some (most) of which are still works-in-progress, are used to make inquiries into the larger question of the significance of interaction in the context of artistic practice involving computers

Data-driven machine translation for sign languages

This thesis explores the application of data-driven machine translation (MT) to sign languages (SLs). The provision of an SL MT system can facilitate communication between Deaf and hearing people by translating information into the native and preferred language of the individual. We begin with an introduction to SLs, focussing on Irish Sign Language - the native language of the Deaf in Ireland. We describe their linguistics and mechanics including similarities and differences with spoken languages. Given the lack of a formalised written form of these languages, an outline of annotation formats is discussed as well as the issue of data collection. We summarise previous approaches to SL MT, highlighting the pros and cons of each approach. Initial experiments in the novel area of example-based MT for SLs are discussed and an overview of the problems that arise when automatically translating these manual-visual languages is given. Following this we detail our data-driven approach, examining the MT system used and modifications made for the treatment of SLs and their annotation. Through sets of automatically evaluated experiments in both language directions, we consider the merits of data-driven MT for SLs and outline the mainstream evaluation metrics used. To complete the translation into SLs, we discuss the addition and manual evaluation of a signing avatar for real SL output

Music Encoding Conference Proceedings

UIDB/00693/2020 UIDP/00693/2020publishersversionpublishe

Choreographing the extended agent : performance graphics for dance theater

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2005.Includes bibliographical references (v. 2, leaves 448-458).The marriage of dance and interactive image has been a persistent dream over the past decades, but reality has fallen far short of potential for both technical and conceptual reasons. This thesis proposes a new approach to the problem and lays out the theoretical, technical and aesthetic framework for the innovative art form of digitally augmented human movement. I will use as example works a series of installations, digital projections and compositions each of which contains a choreographic component - either through collaboration with a choreographer directly or by the creation of artworks that automatically organize and understand purely virtual movement. These works lead up to two unprecedented collaborations with two of the greatest choreographers working today; new pieces that combine dance and interactive projected light using real-time motion capture live on stage. The existing field of"dance technology" is one with many problems. This is a domain with many practitioners, few techniques and almost no theory; a field that is generating "experimental" productions with every passing week, has literally hundreds of citable pieces and no canonical works; a field that is oddly disconnected from modern dance's history, pulled between the practical realities of the body and those of computer art, and has no influence on the prevailing digital art paradigms that it consumes.(cont.) This thesis will seek to address each of these problems: by providing techniques and a basis for "practical theory"; by building artworks with resources and people that have never previously been brought together, in theaters and in front of audiences previously inaccessible to the field; and by proving through demonstration that a profitable and important dialogue between digital art and the pioneers of modern dance can in fact occur. The methodological perspective of this thesis is that of biologically inspired, agent-based artificial intelligence, taken to a high degree of technical depth. The representations, algorithms and techniques behind such agent architectures are extended and pushed into new territory for both interactive art and artificial intelligence. In particular, this thesis ill focus on the control structures and the rendering of the extended agents' bodies, the tools for creating complex agent-based artworks in intense collaborative situations, and the creation of agent structures that can span live image and interactive sound production. Each of these parts becomes an element of what it means to "choreograph" an extended agent for live performance.Marc Downie.Ph.D

Towards hypertextual music

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThis is a study of the way in which digital audio and a number of key associated technologies that rely on it as a framework have changed the creation, production and dissemination of music, as witnessed by my own creative practice. The study is built on my own work as an electronic musician and composer and draws from numerous collaborations with not only other musicians but also researchers and artists, as documented through commissions, performances, academic papers and commercial releases over an 9 year period from 2007 to 2016. I begin by contextualising my own musical practice and outlining some prominent themes associated with the democratisation of computing that the work of this thesis interrogates as a critical framework for the production of musical works. I go on to assess how works using various techniques afforded by digital audio may be interpreted as progressively instantiating a digital ontology of music. In the context of this digital ontology of music I propose a method of analysis and criticism of works explicitly concerned with audio analysis and algorithmic processes based on my interpretation of the concept of `hypertext', wherein the ability for computers to analyse, index and create multi-dimensional, non-linear links between segments of digital audio is best described as hypertextual. In light of this, I contextualise the merits of this reading of music created using these affordances of digital audio through a reading of several key works of 20th century music from a hypertextual perspective, emphasising the role information theory and semiotics have to play in analyses of these works. I proffer this as the beginnings of a useful model for musical composition in the domain of digital audio which I seek to explore through my own practice. I then describe and analyse, both individually and in parallel numerous works I have undertaken that seek to interrogate the intricacies of what it means to work in the domain of digital audio with audio analysis, machine listening, algorithmic and generative computational processes and consider the ways in which aspects of this work might be seen as contributing useful and novel insights into music creation by harnessing properties intrinsic to digital audio as a medium. Finally, I emphasise, based on the music and research presented in the thesis, the extent to which digital audio and the harnessing of increasingly complex computational systems for the production and dissemination of music has changed the ontology of music production, a situation which I interpret as creating both substantial challenges, but also great possibilities for the future of music

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds