AV Zones – Tablet app for audiovisual performance

We have identified potential for tablets to be used as stand-alone tools for audiovisual performance, and not simply as controllers, due to their portability, expressive capabilities of multi-touch, and processing power. To explore this potential, we have developed AV Zones (AudioVisual Zones), an iPad app for audiovisual performance. In a preliminary phase, we conducted interviews with audiovisual performers and a workshop, to understand user needs and desires. We then developed AV Zones, an iPad app for audiovisual performance, composed of an audio sequencer/looper with a visualizer. It explores the interactive potential of a touch screen tablet for integrated musical and visual expression. By default, 3 audiovisual columns or “zones” allow for the manipulation of 3 audio loops. These zones are metaphorical adaptations of channels in a standard audio mixer

AVUI: Designing a toolkit for audiovisual interfaces

The combined use of sound and image has a rich history, from audiovisual artworks to research exploring the potential of data visualization and sonification. However, we lack standard tools or guidelines for audiovisual (AV) interaction design, particularly for live performance. We propose the AVUI (AudioVisual User Interface), where sound and image are used together in a cohesive way in the interface; and an enabling technology, the ofxAVUI toolkit. AVUI guidelines and ofxAVUI were developed in a three-stage process, together with AV producers: 1) participatory design activities; 2) prototype development; 3) encapsulation of prototype as a plug-in, evaluation, and roll out. Best practices identified include: reconfigurable interfaces and mappings; object-oriented packaging of AV and UI; diverse sound visualization; flexible media manipulation and management. The toolkit and a mobile app developed using it have been released as open-source. Guidelines and toolkit demonstrate the potential of AVUI and offer designers a convenient framework for AV interaction design

User-centered design of a tool for interactive computer-generated audiovisuals

The present study aims to design a tool for interactive computer-generated audiovisuals. In this paper, we investigate if the tools for audio- visual performance and composition have caught up with the growing interest and the practices in the field. We have adopted a user-centered design approach for our study, based on interviews and a workshop with practitioners. The interviews identified key themes – expressivity, ease of use and connection with the audience – that were explored in the workshop. During the workshop, a novel methodology was adopt- ed – reboot – which expands upon the bootlegging technique. Key ide- as regarding audiovisual performance gathered from the interviews; sketches for novel audiovisual tools resulting from the workshop; and the reboot technique, are the main contributions of this study

Prototyping audiovisual performance tools: A hackathon approach

We present a user-centered approach for prototyping tools for performance with procedural sound and graphics, based on a hackathon. We also present the resulting prototypes. These prototypes respond to a challenge originating from earlier stages of the research: to combine ease-of-use with expressiveness and visibility of interaction in tools for audiovisual performance. We aimed to convert sketches, resulting from an earlier brainstorming session, into functional prototypes in a short period of time. The outcomes include open-source software base released online. The conclusions reflect on the methodology adopted and the effectiveness of the prototypes

Penguin: Design of a Screen Score Interactive System

In this paper, we present Penguin, a system for live scoring, and Studio I, a piece composed for the system and an accordion. The system is shaped in two modules, one that generates a musical stream in real-time and the other that manages a live scoring process. Penguin is designed to be used in interactive performances alongside traditional instruments. Studio I is a piece for Penguin and accordion. The interaction design of the system and the piece were fine- tuned involving the instrumentalist. We provide a general description of Penguin and present the design process that led to the development of the interactive performance. The design process led two main contributions. Firstly, we identify and frame a new performer role that mixes performing and conducting elements. Secondly, we discuss how the design process of the system affected the ownership of the aesthetic of music

Toward the adoption of design concepts in scoring for Digital Musical Instruments: a case study on affordances and constraints

This paper, we propose the idea that a score for a Digital Musical In- strument (DMIs) can be approached from a design perspective. In particular, we focus on the concepts of affordances and constraints. The paper describes the use of scores in a piece called XXV composed for Chimney, a DMI and Cello. Both the piece and the system are detailed with a focus on design concepts. In the conclusion we compare our approach to other literature that discuss the use of score with DMIs. Based on this, and the results obtained, we present recommendations regarding the use of score in DMIs. Future work is also outlined

Pathways to live visuals in dance performances: a quantitative audience study

INTRODUCTION: We present an audience study investigating the impact of different technologies to create visuals in dance performances. OBJECTIVES: We investigated four conditions: motion capture, sensors, camera image, and minimal interaction; and four variables: how much did the audience perceive a connection between the body and the visuals; the visuals as merely copying the dancer; how much distracting were the visuals; and how much did the audience enjoy the visuals. METHODS: We used a questionnaire to collect data. We analyzed it using Friedman’s test, and Spearman’s correlation test. RESULTS: The audience perceived a stronger connection in the camera condition, but in the same condition, visuals tend to be merely copying the dancer. We also suggest that the perceived connection has a positive correlation with enjoyment, while distraction has a negative correlation. CONCLUSION: Our results help to highlight the impact that different technology have on live visuals for dance

User-Centered Design of a Tool for Interactive Computer-Generated Audiovisuals

There is a growing interest in audiovisual performance and composition. In this paper, we would like to investigate if the tools for audiovisual performance and composition have caught up with this growing interest and the practices in the field. In particular, we are interested in tools that use computer-generated graphics. To address these issues, we have adopted a user-centered design approach for our study, based on interviews and a workshop with practitioners. The interviews identified key themes – expressivity, ease of use and connection with the audience – that were explored in the workshop. During the workshop, a novel methodology was adopted – reboot – which expands upon the bootlegging technique. Key ideas regarding audiovisual performance gathered from the interviews; sketches for novel audiovisual tools resulting from the workshop; and the reboot technique, are the main contributions of this study