13 research outputs found
Towards Gestural Sonic Affordances
We present a study that explores the affordance evoked by sound andsound-gesture mappings. In order to do this, we make use of a sensor systemwith minimal form factor in a user study that minimizes cultural associationThe present study focuses on understanding how participants describe sounds andgestures produced while playing designed sonic interaction mappings. This approach seeks to move from object-centric affordance towards investigatingembodied gestural sonic affordances
Gesture, sound and place.
This paper discusses two multichannel interactive audiovisual artworks, Action A/V and SoundLabyrinth, that explore approaches to the experience of gesture, sound and place. Both works were situated in a geodesic dome frame and built within the Max, Ableton and Max for Live computer programs, and produced ostensibly similar outcomes, however the approaches taken by the two authors differ in intention, processes, and philosophy. These approaches were presented and discussed in workshops delivered at ISEA2013, on Sunday June 9 and Monday June 10 2013. In these workshops participants mprovised with the two systems, both through moving in the dome and by operating the related software, and discussed approaches and understandings of the three terms listed in the title of this paper
‘Listening back’ : exploring the sonic interactions at the heart of historical sound effects performance
The cinematic sound design practice of Foley developed from a rich history of performative sound design through materials, objects and mechanical devices created for theatrical performance, magic lantern shows and silent cinema screenings in the late nineteenth and early twentieth century. Now, as virtual and digital methods become available in film sound design production, it is time to evaluate the contribution of these historical sound design methods through the discipline of Sonic Interaction Design (SID). Exploring the use of everyday objects and sound effects devices in the creation of a soundtrack allows us to ‘listen’ back and forward simultaneously. Our knowledge of historical sound practice can be updated and new practices can be generated, at the same time deepening our understanding of sound for screen that is performative in nature. This article investigates the interactivity inherent in historical sound effects performance through a case study of a mechanical and digital wind machine reconstruction, and explores its potential to inform new interactive digital methods for sound design
The Egocentric Nature of Action-Sound Associations
Actions that produce sounds infuse our daily lives. Some of these sounds are a natural consequence of physical interactions (such as a clang resulting from dropping a pan), but others are artificially designed (such as a beep resulting from a keypress). Although the relationship between actions and sounds has previously been examined, the frame of reference of these associations is still unknown, despite it being a fundamental property of a psychological representation. For example, when an association is created between a keypress and a tone, it is unclear whether the frame of reference is egocentric (gesture-sound association) or exocentric (key-sound association). This question is especially important for artificially created associations, which occur in technology that pairs sounds with actions, such as gestural interfaces, virtual or augmented reality, and simple buttons that produce tones. The frame of reference could directly influence the learnability, the ease of use, the extent of immersion, and many other factors of the interaction. To explore whether action-sound associations are egocentric or exocentric, an experiment was implemented using a computer keyboard’s number pad wherein moving a finger from one key to another produced a sound, thus creating an action-sound association. Half of the participants received egocentric instructions to move their finger with a particular gesture. The other half of the participants received exocentric instructions to move their finger to a particular number on the keypad. All participants were performing the same actions, and only the framing of the action varied between conditions by altering task instructions. Participants in the egocentric condition learned the gesture-sound association, as revealed by a priming paradigm. However, the exocentric condition showed no priming effects. This finding suggests that action-sound associations are egocentric in nature. A second part of the same session further confirmed the egocentric nature of these associations by showing no change in the priming effect after moving to a different starting location. Our findings are consistent with an egocentric representation of action-sound associations, which could have implications for applications that utilize these associations
Form Follows Sound
Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships in order to design rich sonic interactions. We propose a participatory approach to sonic interaction design that first considers the affordances of sounds in order to imagine embodied interaction, and based on this, generates interaction models for interaction designers wishing to work with sound. We describe a series of workshops, called Form Follows Sound, where participants ideate imagined sonic interactions, and then realize working interactive sound prototypes. We introduce the Sonic Incident technique, as a way to recall memorable sound experiences. We identified three interaction models for sonic interaction design: conducting; manipulating; substituting. These three interaction models offer interaction designers and developers a framework on which they can build richer sonic interactions
Form Follows Sound: Designing Interactions from Sonic Memories
Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships in order to design rich sonic interactions. We propose a participatory approach to sonic interaction design that first considers the affordances of sounds in order to imagine embodied interaction, and based on this, generates interaction models for interaction designers wishing to work with sound. We describe a series of workshops, called Form Follows Sound, where participants ideate imagined sonic interactions, and then realize working interactive sound prototypes. We introduce the Sonic Incident technique, as a way to recall memorable sound experiences. We identified three interaction models for sonic interaction design: conducting; manipulating; substituting. These three interaction models offer interaction designers and developers a framework on which they can build richer sonic interactions
Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts
This thesis’s primary goal is to investigate performance ecologies, that is the compound
of humans, artifacts and environmental elements that contribute to the result of a per-
formance. In particular, this thesis focuses on designing new interactive technologies for
sound and music. The goal of this thesis leads to the following Research Questions (RQs):
• RQ1 How can the design of interactive sonic artifacts support a joint expression
across different actors (composers, choreographers, and performers, musicians, and
dancers) in a given performance ecology?
• RQ2 How does each different actor influence the design of different artifacts, and
what impact does this have on the overall artwork?
• RQ3 How do the different actors in the same ecology interact, and appropriate an
interactive artifact?
To reply to these questions, a new framework named ARCAA has been created. In this
framework, all the Actors of a given ecology are connected to all the Artifacts throughout
three layers: Role, Context and Activity. This framework is then applied to one systematic
literature review, two case studies on music performance and one case study in dance
performance. The studies help to better understand the shaded roles of composers, per-
formers, instrumentalists, dancers, and choreographers, which is relevant to better design
interactive technologies for performances. Finally, this thesis proposes a new reflection on
the blurred distinction between composing and designing a new instrument in a context
that involves a multitude of actors.
Overall, this work introduces the following contributions to the field of interaction
design applied to music technology: 1) ARCAA, a framework to analyse the set of inter-
connected relationship in interactive (music) performances, validated through 2 music
studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de-
signing music interactive system for performance (music or dance), accounting for the
needs of the various actors and for the overlapping on music composition and design of in-
teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings;
4) Proposal of a methodological approach combining autobiographical and idiographical
design approaches in interactive performances.O objetivo principal desta tese é investigar as ecologias performativas, conjunto formado
pelos participantes humanos, artefatos e elementos ambientais que contribuem para o
resultado de uma performance. Em particular, esta tese foca-se na conceção de novas
tecnologias interativas para som e música. O objetivo desta tese originou as seguintes
questões de investigação (Research Questions RQs):
• RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con-
junta entre diferentes atores (compositores, coreógrafos e performers, músicos e
dançarinos) numa determinada ecologia performativa?
• RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso
tem no trabalho artÃstico global?
• RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam
de um artefato interativo?
Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA.
Nesta framework, todos os atores (Actores) de uma dada ecologia estão conectados a todos
os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework
foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre
performance musical e a um estudo de caso em performance de dança. Estes estudos aju-
daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes,
instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec-
nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre
a distinção entre compor e projetar um novo instrumento num contexto que envolve uma
multiplicidade de atores.
Este trabalho apresenta as seguintes contribuições principais para o campo do design
de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o
conjunto de relações interconectadas em performances interativas, validado através de
dois estudos de caso relacionados com a música, um estudo de caso relacionado com
a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as
necessidades dos vários atores e a sobreposição entre a composição musical e o design de
tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram
as ecologias performativas no NIME, com base numa análise sistemática da literatura
dos artigos apresentados e publicados nestas conferência; 4) Proposta de uma aborda-
gem metodológica combinando abordagens de design autobiográfico e idiográfico em
performances interativas