Seeking out the spaces between: Using improvisation for collaborative composition and interactive technology

Copyright © 2010 ISASTThis article presents findings from experiments into piano performance live electronics undertaken by the author since early 2007. The use of improvisation has infused every step of the process---both as a methodology to obtain meaningful results using interactive technology and as a way to generate and characterize a collaborative musical space with composers. The technology used has included pre-built MIDI interfaces such as the PianoBar, actuators such as miniature DC motors and sensor interfaces including iCube and the Wii controller. Collaborators have included researchers at the Centre for Digital Music (QMUL), Richard Barrett, Pierre Alexandre Tremblay and Atau Tanaka. In seeking to create responsive “performance environments” at the piano, I explore live, performative control of electronics to create better connections for both performer (providing the same level of interpretive freedom as with a “pure” instrumental performance) and audience (communicating clearly to them). I have been lucky to witness first-hand many live interactive performances and to work with various empathetic composers/performers in flexible working environments. Collaborating with experienced technologists and musicians, I have witnessed time and again what, for me, is a fundamental truth in interactive instrumental performance: As a living, spontaneous form it must be nurtured and informed by the performer’s physicality and imagination as much as by the creativity or knowledge of the composer and/or technologist. Specifically in the case of sensors, their dependence on the detail of each person’s body and reactions is so refined as to necessitate, I would argue, an entirely collaborative approach and therefore one that involves at least directed improvisation and, more likely, fairly extensive improvised exploration. The fundamentally personal and intimate nature of sensor readings---the amount of tension created by each performer, the shape of the ancillary gestures or the level of emotional involvement (especially relevant when using galvanic skin response or EEG)---makes creating pieces with sensors extremely difficult for a composer to do in isolation. Improvisation therefore provides a way for performer and composer to generate a common musical and gestural language. Related to these issues is the fact that the technical or notational parameters in interactive music are not yet (and may never be) standardized, thereby creating a very real and practical need for improvisation to figure at least somewhere in the process.This study is funded by the Arts and Humanities Research Council

Education Unleashed: Participatory Culture, Education, and Innovation in Second Life

Part of the Volume on the Ecology of Games: Connecting Youth, Games, and LearningWhile virtual worlds share common technologies and audiences with games, they possess many unique characteristics. Particularly when compared to massively multiplayer online role-playing games, virtual worlds create very different learning and teaching opportunities through markets, creation, and connections to the real world, and lack of overt game goals. This chapter aims to expose a wide audience to the breadth and depth of learning occurring within Second Life (SL). From in-world classes in the scripting language to mixed-reality conferences about the future of broadcasting, a tremendous variety of both amateurs and experts are leveraging SL as a platform for education. In one sense, this isn't new since every technology is co-opted by communities for communication, but SL is different because every aspect of it was designed to encourage this co-opting, this remixing of the virtual and the real

Generating trails automatically, to aid navigation when you revisit an environment

A new method for generating trails from a person’s movement through a virtual environment (VE) is described. The method is entirely automatic (no user input is needed), and uses string-matching to identify similar sequences of movement and derive the person’s primary trail. The method was evaluated in a virtual building, and generated trails that substantially reduced the distance participants traveled when they searched for target objects in the building 5-8 weeks after a set of familiarization sessions. Only a modest amount of data (typically five traversals of the building) was required to generate trails that were both effective and stable, and the method was not affected by the order in which objects were visited. The trail generation method models an environment as a graph and, therefore, may be applied to aiding navigation in the real world and information spaces, as well as VEs

Sensing and mapping for interactive performance

This paper describes a trans-domain mapping (TDM) framework for translating meaningful activities from one creative domain onto another. The multi-disciplinary framework is designed to facilitate an intuitive and non-intrusive interactive multimedia performance interface that offers the users or performers real-time control of multimedia events using their physical movements. It is intended to be a highly dynamic real-time performance tool, sensing and tracking activities and changes, in order to provide interactive multimedia performances. From a straightforward definition of the TDM framework, this paper reports several implementations and multi-disciplinary collaborative projects using the proposed framework, including a motion and colour-sensitive system, a sensor-based system for triggering musical events, and a distributed multimedia server for audio mapping of a real-time face tracker, and discusses different aspects of mapping strategies in their context. Plausible future directions, developments and exploration with the proposed framework, including stage augmenta tion, virtual and augmented reality, which involve sensing and mapping of physical and non-physical changes onto multimedia control events, are discussed

Off the edge

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University London.Work which takes from elsewhere forms an important thread in European art music. There is a long tradition of music which variously borrows, thieves, pastiches, plagiarises, ironically ‘retakes’, hoaxes, impersonates and appropriates. The music I have written for Off the edge, while seeking to honour and add to this thread, also attempts to zoom in upon and make explicit the idea of an ultimate and irreversible composerly self-annihilation, a kind of one-way exit-gate from the world of authored musical works itself made of pieces of music, which so much of this tradition, I feel, points towards. (Of my nine pieces, it is perhaps Time to go—only, with its ‘à la suicide note’ texts and its music that seems to slide in from far beyond the frame that is ‘composer Luke Stoneham’, which manages to get closest to this.) I have chosen the title Off the edge, because all of my music tries to capture a sense of nocturnal peripheral vision: be content with catching glimpses of the composer Luke Stoneham, because as soon as you turn to look at him face-on, he disappears

Deep Learning Techniques for Music Generation -- A Survey

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.Comment: 209 pages. This paper is a simplified version of the book: J.-P. Briot, G. Hadjeres and F.-D. Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 201