Let's Resonate! How to Elicit Improvisation and Letting Go in Interactive Digital Art

Participatory art allows for the spectator to be a participant or a viewer able to engage actively with interactive art. Real-time technologies offer new ways to create participative artworks. We hereby investigate how to engage participation through movement in interactive digital art, and what this engagement can awaken, focusing on the ways to elicit improvisation and letting go. We analyze two Virtual Reality installations, ''InterACTE'' and ''Eve, dance is an unplaceable place,'' involving body movement, dance, creativity and the presence of an observing audience. We evaluate the premises, the setup, and the feedback of the spectators in the two installations. We propose a model following three different perspectives of resonance: 1. Inter Resonance between Spectator and Artwork, which involves curiosity, imitation, playfulness and improvisation. 2. Inner Resonance of Spectator him/herself, where embodiment and creativity contribute to the sense of being present and letting go. 3. Collective Resonance between Spectator/Artwork and Audience, which is stimulated by curiosity, and triggers motor contagion, engagement and gathering. The two analyzed examples seek to awaken open-minded communicative possibilities through the use of interactive digital artworks. Moreover, the need to recognize and develop the idea of resonance becomes increasingly important in this time of urgency to communicate, understand and support collectivity

Musical and meta-musical conversations

This collaboration emerged out of informal conversation between the authors about improvisation. Ben-Tal is a composer/researcher who has been using Music Information Retrieval (MIR) techniques and AI as tools for composition. Dolan is a performer/improviser and researcher on improvisation, creativity and expressive performance with little knowledge of music technology. Dolan became intrigued but also highly sceptical about Ben-Tal’s ideas of musical dialogues between human and computer as a basis for co-creation. They agreed to meet and trial the possibility of real-time improvisation between piano and computer. By his own admission, Dolan came to this first session assuming he will prove the inadequacy of such a set-up for joint improvisation based on extended tonal music idiom.  He found himself equally surprised and alarmed when he experienced moments that felt, to himself,  as real dialogue with the machine. This proof-of-concept session provided the starting point for an ongoing collaboration: developing a unique duo-improvisation within the context of computationally creative tools, real-time interaction, tonal-music and human-computer interaction. Central to this work are musical dialogues between Dolan on the piano and Ben-Tal’s computing system as they improvise together. These are surrounded and complemented by conversations between the authors about the system, about improvisation, composition, performance, music and AI.  This presentation starts from a description of the current improvisation set-up and the development that allowed us to arrive at this stage. The following section re-enacts some of the conversations that the authors engaged in, which will illuminate the learning and discovery process they underwent together. We will end by drawing out important themes emerging from the musical and meta-musical conversations in relation to current debates around music and AI

One's own soundtrack: Affective music synthesis

Computer music usually sounds mechanical; hence, if musicality and music expression of virtual actors could be enhanced according to the user's mood, the quality of experience would be amplified. We present a solution that is based on improvisation using cognitive models, case based reasoning (CBR) and fuzzy values acting on close-to-affect-target musical notes as retrieved from CBR per context. It modifies music pieces according to the interpretation of the user's emotive state as computed by the emotive input acquisition componential of the CALLAS framework. The CALLAS framework incorporates the Pleasure-Arousal- Dominance (PAD) model that reflects emotive state of the user and represents the criteria for the music affectivisation process. Using combinations of positive and negative states for affective dynamics, the octants of temperament space as specified by this model are stored as base reference emotive states in the case repository, each case including a configurable mapping of affectivisation parameters. Suitable previous cases are selected and retrieved by the CBR subsystem to compute solutions for new cases, affect values from which control the music synthesis process allowing for a level of interactivity that makes way for an interesting environment to experiment and learn about expression in music

Modeling Joint Improvisation between Human and Virtual Players in the Mirror Game

Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. A theoretical model based on observations and analysis of experimental data is proposed to capture the main features of their interaction. A set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Then, the model is used to drive a computer avatar able to improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce the emergence of joint movement between the participants

Improvising with the threnoscope: integrating code, hardware, GUI, network, and graphic scores

Live coding emphasises improvisation. It is an art practice that merges the act of musical composition and performance into a public act of projected writing. This paper introduces the Threnoscope system, which includes a live coding micro-language for drone-based microtonal composition. The paper discusses the aims and objectives of the system, elucidates the design decisions, and introduces in particular the code score feature present in the Threnoscope. The code score is a novel element in the design of live coding systems allowing for improvisation through a graphic score, rendering a visual representation of past and future events in a real-time performance. The paper demonstrates how the system’s methods can be mapped ad hoc to GUI- or hardware-based control

Affect and Metaphor Sensing in Virtual Drama

We report our developments on metaphor and affect sensing for several metaphorical language phenomena including affects as external entities metaphor, food metaphor, animal metaphor, size metaphor, and anger metaphor. The metaphor and affect sensing component has been embedded in a conversational intelligent agent interacting with human users under loose scenarios. Evaluation for the detection of several metaphorical language phenomena and affect is provided. Our paper contributes to the journal themes on believable virtual characters in real-time narrative environment, narrative in digital games and storytelling and educational gaming with social software

Seeking out the spaces between: Using improvisation for collaborative composition and interactive technology

Copyright © 2010 ISASTThis article presents findings from experiments into piano performance live electronics undertaken by the author since early 2007. The use of improvisation has infused every step of the process---both as a methodology to obtain meaningful results using interactive technology and as a way to generate and characterize a collaborative musical space with composers. The technology used has included pre-built MIDI interfaces such as the PianoBar, actuators such as miniature DC motors and sensor interfaces including iCube and the Wii controller. Collaborators have included researchers at the Centre for Digital Music (QMUL), Richard Barrett, Pierre Alexandre Tremblay and Atau Tanaka. In seeking to create responsive “performance environments” at the piano, I explore live, performative control of electronics to create better connections for both performer (providing the same level of interpretive freedom as with a “pure” instrumental performance) and audience (communicating clearly to them). I have been lucky to witness first-hand many live interactive performances and to work with various empathetic composers/performers in flexible working environments. Collaborating with experienced technologists and musicians, I have witnessed time and again what, for me, is a fundamental truth in interactive instrumental performance: As a living, spontaneous form it must be nurtured and informed by the performer’s physicality and imagination as much as by the creativity or knowledge of the composer and/or technologist. Specifically in the case of sensors, their dependence on the detail of each person’s body and reactions is so refined as to necessitate, I would argue, an entirely collaborative approach and therefore one that involves at least directed improvisation and, more likely, fairly extensive improvised exploration. The fundamentally personal and intimate nature of sensor readings---the amount of tension created by each performer, the shape of the ancillary gestures or the level of emotional involvement (especially relevant when using galvanic skin response or EEG)---makes creating pieces with sensors extremely difficult for a composer to do in isolation. Improvisation therefore provides a way for performer and composer to generate a common musical and gestural language. Related to these issues is the fact that the technical or notational parameters in interactive music are not yet (and may never be) standardized, thereby creating a very real and practical need for improvisation to figure at least somewhere in the process.This study is funded by the Arts and Humanities Research Council

Play it again, Duke: jazz performance, improvisation, and the construction of spontaneity

No abstract available

Methods and ideas for the creation of 'transparent' music in the classroom

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Methods and ideas for the creation of ‘transparent’ music in the classroom The aims of this port-folio are as follows; - To provide a coherent sequence of pieces and methods which can be used to create music in an educational context and also encourage students and teachers to develop their own creativity. - To provide pieces which develop student’s confidence in their own ability to create music in a variety of ways including composition, improvisation and creative leadership. - To provide exercises and pieces which help to develop the listening and appreciation skills essential for ensemble musicmaking. - To provide methods that enable the creation of ‘transparent music’. This is music in which the some, or all, of the decision making involved in the creation of a piece is accessible and apparent to an audience during its performance. This submission consists of a teaching book containing thirteen pieces/exercises, instructions giving guidance on their possible use in a teaching context and recorded examples. Also included are separate instructions where appropriate for the use of pieces in a concert or other non-educational setting and two essays giving context and background information on the ideas behind the pieces