Sonifying drawings: characterization of perceptual attributes of sounds produced by human gestures

Abstract

International audienceFriction sounds produced by the pencil of a person who is drawing on a paper are audible and could bring information about his/her gestures. This study firstly focuses on the perceptual significance of the morphology of such sounds, and to what extent gestures could be retrieved by sounds. Sounds recorded during drawing sessions were used in association tests where subjects had to univocally associate friction sounds with different shapes. Results showed that subjects were able to associate sounds with correct shapes and that the auditory characterization of the shape depended on the velocity profile.\\ A sonification strategy was then proposed for human drawings using a friction sound synthesis model. Inspired by the work of Viviani et al. (1982) who exhibited a 2/3-power law relation between kinetics and shape curvature for visual perception, experiments were carried out with the synthesis model, where subjects were asked to adjust the power law exponent so that the most realistic sound was obtained. Results revealed an exponent close to 2/3 as previously found in vision and thus highlighted a similar power law in the auditory modality, providing an ecological way to determine velocity profiles from static shapes and to generate sounds coherent with human gestures