miMic: The microphone as a pencil

miMic, a sonic analogue of paper and pencil is proposed: An augmented microphone for vocal and gestural sonic sketching. Vocalizations are classified and interpreted as instances of sound models, which the user can play with by vocal and gestural control. The physical device is based on a modified microphone, with embedded inertial sensors and buttons. Sound models can be selected by vocal imitations that are automatically classified, and each model is mapped to vocal and gestural features for real-time control. With miMic, the sound designer can explore a vast sonic space and quickly produce expressive sonic sketches, which may be turned into sound prototypes by further adjustment of model parameters

A design exploration on the effectiveness of vocal imitations

Among sonic interaction design practices a rising interest is given to the use of the voice as a tool for producing fast and rough sketches. Goal of the EU project SkAT-VG (Sketching Audio Technologies using Vocalization and Gestures, 2014-2016) is to develop vocal sketching as a reference practice for sound design by (i) improving our understanding on how sounds are communicated through vocalizations and gestures, (ii) looking for physical relations between vocal sounds and sound-producing phenomena, (iii) designing tools for converting vocalizations and gestures into parametrized sound models. We present the preliminary outcomes of a vocal sketching workshop held at the Conservatory of Padova, Italy. Research through design activities focused on how teams of potential designers make use of vocal imitations, and how morphological attributes of sound may inform the training of basic vocal techniques

Form Follows Sound: Designing Interactions from Sonic Memories

Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships in order to design rich sonic interactions. We propose a participatory approach to sonic interaction design that first considers the affordances of sounds in order to imagine embodied interaction, and based on this, generates interaction models for interaction designers wishing to work with sound. We describe a series of workshops, called Form Follows Sound, where participants ideate imagined sonic interactions, and then realize working interactive sound prototypes. We introduce the Sonic Incident technique, as a way to recall memorable sound experiences. We identified three interaction models for sonic interaction design: conducting; manipulating; substituting. These three interaction models offer interaction designers and developers a framework on which they can build richer sonic interactions

Sketching sounds: an exploratory study on sound-shape associations

Sound synthesiser controls typically correspond to technical parameters of signal processing algorithms rather than intuitive sound descriptors that relate to human perception of sound. This makes it difficult to realise sound ideas in a straightforward way. Cross-modal mappings, for example between gestures and sound, have been suggested as a more intuitive control mechanism. A large body of research shows consistency in human associations between sounds and shapes. However, the use of drawings to drive sound synthesis has not been explored to its full extent. This pa- per presents an exploratory study that asked participants to sketch visual imagery of sounds with a monochromatic digital drawing interface, with the aim to identify different representational approaches and determine whether timbral sound characteristics can be communicated reliably through visual sketches. Results imply that the development of a synthesiser exploiting sound-shape associations is feasible, but a larger and more focused dataset is needed in followup studies

Using participatory visualisation of soundscapes to compare designers’ and listeners’ experiences of sound designs

There are numerous rules and well-established guidelines to help designers with the visual appearance of interactive technologies. In contrast, when it comes to the use of sound, there is a paucity of practical information regarding design for euphony, excepting musical composition. This paper addresses this hiatus by describing a theoretically based, practical method for evaluating the design of the auditory components of interactive technologies and media. Specifically, the method involves eliciting the auditory experiences of users of these technologies and media and comparing them with what the sound designers had intended. The method has been comprehensively tested in trials involving 100 users (listeners), and the results have been described as “useful” and “invaluable” by a group of 10 professional sound designers

Using participatory visualisation of soundscapes to compare designers’ and listeners’ experiences of sound designs

There are numerous rules and well-established guidelines to help designers with the visual appearance of interactive technologies. In contrast, when it comes to the use of sound, there is a paucity of practical information regarding design for euphony, excepting musical composition. This paper addresses this hiatus by describing a theoretically based, practical method for evaluating the design of the auditory components of interactive technologies and media. Specifically, the method involves eliciting the auditory experiences of users of these technologies and media and comparing them with what the sound designers had intended. The method has been comprehensively tested in trials involving 100 users (listeners), and the results have been described as “useful” and “invaluable” by a group of 10 professional sound designers

Designing From Listening: Embodied Experience and Sonic Interactions

An understanding of the richness of people’s sonic experience can lead to the creation of novel methods for informing design practices. One of the challenges in Sonic Interaction Design (SID) is to deal with the complexity of the “sonic”: its phenomenon, the interactions it creates, its social and cultural contexts. To tackle this challenge, this thesis investigates how we can draw upon people’s everyday sonic experience, particularly listening and remembering sound, to design interactions using body movement, digital sound processing and embodied technologies. Firstly, the research analyses how sound has been studied in its phenomenological, cultural and social aspects in fields such as Sound Studies and Embodied Sound Cognition. Secondly, it involves users in the process of designing sonic interactions, with a user study about gestural-sound relationships during active control of digital sound, and a series of participatory design workshops which draws upon people’s sonic experience for imagining interactions with sound. The thesis provides four main contributions. The first is Retro-Active Listening, a concept which draws attention to sounds heard in the past by remembering listening to them. The second is the Sonic Incident, a technique for SID workshops, which allows designers to explore participants’ past experiences of listening. The third is the Gestural Sound Toolkit, which enables designers to rapidly prototype interactive sound mappings based on human movement. The final contribution is three models for designing embodied sonic interactions. These comprise (1) Substitution, in which users’ movements substitute the cause of the sound, (2) Conduction, where users’ movements have a semantic relationship with the sound, and (3) Manipulation, in which users’ movements manipulate the sound. These contributions help to build a framework for design that addresses lesser-explored matters in SID, such as embodiment and contextual aspects of sound, which are potentially relevant for users