Designing constraints: composing and performing with digital musical systems

This paper investigates two central terms in Human Computer Interaction (HCI) – affordances and constraints – and studies their relevance to the design and understanding of digital musical systems. It argues that in the analysis of complex systems, such as new interfaces for musical expression (NIME), constraints are a more productive analytical tool than the common HCI usage of affordances. Constraints are seen as limitations enabling the musician to encapsulate a specific search space of both physical and compositional gestures, proscribing complexity in favor of a relatively simple set of rules that engender creativity. By exploring the design of three different digital musical systems, the paper defines constraints as a core attribute of mapping, whether in instruments or compositional systems. The paper describes the aspiration for designing constraints as twofold: to save time, as musical performance is typically a real-time process, and to minimize the performer’s cognitive load. Finally, it discusses skill and virtuosity in the realm of new musical interfaces for musical expression with regard to constraints

The ixiQuarks: merging code and GUI in one creative space

This paper reports on ixiQuarks; an environment of instruments and effects that is built on top of the audio programming language SuperCollider. The rationale of these instruments is to explore alternative ways of designing musical interaction in screen-based software, and investigate how semiotics in interface design affects the musical output. The ixiQuarks are part of external libraries available to SuperCollider through the Quarks system. They are software instruments based on a non- realist design ideology that rejects the simulation of acoustic instruments or music hardware and focuses on experimentation at the level of musical interaction. In this environment we try to merge the graphical with the textual in the same instruments, allowing the user to reprogram and change parts of them in runtime. After a short introduction to SuperCollider and the Quark system, we will describe the ixiQuarks and the philosophical basis of their design. We conclude by looking at how they can be seen as epistemic tools that influence the musician in a complex hermeneutic circle of interpretation and signification

The acoustic, the digital and the body: a survey on musical instruments

This paper reports on a survey conducted in the autumn of 2006 with the objective to understand people's relationship to their musical tools. The survey focused on the question of embodiment and its different modalities in the fields of acoustic and digital instruments. The questions of control, instrumental entropy, limitations and creativity were addressed in relation to people's activities of playing, creating or modifying their instruments. The approach used in the survey was phenomenological, i.e. we were concerned with the experience of playing, composing for and designing digital or acoustic instruments. At the time of analysis, we had 209 replies from musicians, composers, engineers, designers, artists and others interested in this topic. The survey was mainly aimed at instrumentalists and people who create their own instruments or compositions in flexible audio programming environments such as SuperCollider, Pure Data, ChucK, Max/MSP, CSound, etc

Dynamic mapping strategies for interactive art installations: an embodied combined HCI HRI HHI approach

This paper proposes a theoretical framework for dealing with the paradigm of interactivity in new media art, and how the broad use of the term in different research fields can lead to some misunderstandings. The paper addresses a conceptual view on how we can implement interaction in new media art from an embodied approach that unites views from HCI, HRI and HHI. The focus is on an intuitive mapping of a multitude of sensor data and to extend upon this using the paradigm of (1) finite state machines (FSM) to address dynamic mapping strategies, (2) mediality to address aisthesis and (3) embodiment to address valid mapping strategies originated from natural body movements. The theory put forward is illustrated by a case study

Toward the adoption of design concepts in scoring for Digital Musical Instruments: a case study on affordances and constraints

This paper, we propose the idea that a score for a Digital Musical In- strument (DMIs) can be approached from a design perspective. In particular, we focus on the concepts of affordances and constraints. The paper describes the use of scores in a piece called XXV composed for Chimney, a DMI and Cello. Both the piece and the system are detailed with a focus on design concepts. In the conclusion we compare our approach to other literature that discuss the use of score with DMIs. Based on this, and the results obtained, we present recommendations regarding the use of score in DMIs. Future work is also outlined

The threnoscope: a musical work for live coding performance

This paper introduces a new direction in the field of artistic live coding where musical works are presented as pieces in the form of a live coding system. The system itself and the code affordances become equivalent to score system in an open musical work for strong improvisation

Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts

This thesis’s primary goal is to investigate performance ecologies, that is the compound of humans, artifacts and environmental elements that contribute to the result of a per- formance. In particular, this thesis focuses on designing new interactive technologies for sound and music. The goal of this thesis leads to the following Research Questions (RQs): • RQ1 How can the design of interactive sonic artifacts support a joint expression across different actors (composers, choreographers, and performers, musicians, and dancers) in a given performance ecology? • RQ2 How does each different actor influence the design of different artifacts, and what impact does this have on the overall artwork? • RQ3 How do the different actors in the same ecology interact, and appropriate an interactive artifact? To reply to these questions, a new framework named ARCAA has been created. In this framework, all the Actors of a given ecology are connected to all the Artifacts throughout three layers: Role, Context and Activity. This framework is then applied to one systematic literature review, two case studies on music performance and one case study in dance performance. The studies help to better understand the shaded roles of composers, per- formers, instrumentalists, dancers, and choreographers, which is relevant to better design interactive technologies for performances. Finally, this thesis proposes a new reflection on the blurred distinction between composing and designing a new instrument in a context that involves a multitude of actors. Overall, this work introduces the following contributions to the field of interaction design applied to music technology: 1) ARCAA, a framework to analyse the set of inter- connected relationship in interactive (music) performances, validated through 2 music studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de- signing music interactive system for performance (music or dance), accounting for the needs of the various actors and for the overlapping on music composition and design of in- teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings; 4) Proposal of a methodological approach combining autobiographical and idiographical design approaches in interactive performances.O objetivo principal desta tese é investigar as ecologias performativas, conjunto formado pelos participantes humanos, artefatos e elementos ambientais que contribuem para o resultado de uma performance. Em particular, esta tese foca-se na conceção de novas tecnologias interativas para som e música. O objetivo desta tese originou as seguintes questões de investigação (Research Questions RQs): • RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con- junta entre diferentes atores (compositores, coreógrafos e performers, músicos e dançarinos) numa determinada ecologia performativa? • RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso tem no trabalho artístico global? • RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam de um artefato interativo? Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA. Nesta framework, todos os atores (Actores) de uma dada ecologia estão conectados a todos os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre performance musical e a um estudo de caso em performance de dança. Estes estudos aju- daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes, instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec- nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre a distinção entre compor e projetar um novo instrumento num contexto que envolve uma multiplicidade de atores. Este trabalho apresenta as seguintes contribuições principais para o campo do design de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o conjunto de relações interconectadas em performances interativas, validado através de dois estudos de caso relacionados com a música, um estudo de caso relacionado com a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as necessidades dos vários atores e a sobreposição entre a composição musical e o design de tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram as ecologias performativas no NIME, com base numa análise sistemática da literatura dos artigos apresentados e publicados nestas conferência; 4) Proposta de uma aborda- gem metodológica combinando abordagens de design autobiográfico e idiográfico em performances interativas

Multi-touch interaction principles for collaborative real-time music activities: towards a pattern language

In this paper we give an analysis of the literature on a set of problems that can arise when undertaking the interaction design of multi-touch applications for collaborative real-time music activities, which are designed for multitouch technologies (e.g. smartphones, tablets, interactive tabletops, among others). Each problem is described, and a candidate design pattern (CDP) is suggested in the form of a short sentence and a diagram—an approach inspired by Christopher Alexander’s A Pattern Language. These solutions relate to the fundamental collaborative principles of democratic relationships, identities and collective interplay. We believe that this approach might disseminate forms of best design practice for collaborative music applications, in order to produce real-time musical systems which are collaborative and expressive