The high/low frequency balance drives the perception of noisy vibrations

Noisy vibrotactile signals transmitted during tactile explorations of an object provide precious information on the nature of its surface. Linking the properties of such vibrotactile signals to the way they are interpreted by the haptic sensory system remains challenging. In this study, we investigated humans' perception of noisy, stationary vibrations recorded during exploration of textures and reproduced using a vibrotactile actuator. Since intensity is a well-established essential perceptual attribute, an intensity equalization was first conducted, providing a model for its estimation. The equalized stimuli were further used to identify the most salient spectral features in a second experiment using dissimilarity estimations between pairs of vibrations. Based on dimensionally reduced spectral representations, linear models of dissimilarity prediction showed that the balance between low and high frequencies was the most important cue. Formal validation of this result was achieved through a Mushra experiment, where participants assessed the fidelity of resynthesized vibrations with various distorted frequency balances. These findings offer valuable insights into human vibrotactile perception and establish a computational framework for analyzing vibrations as humans do. Moreover, they pave the way for signal synthesis and compression based on sparse representations, holding significance for applications involving complex vibratory feedback

Controlling a non linear friction model for evocative sound synthesis applications

International audienceIn this paper, a flexible strategy to control a synthesis model of sounds produced by non linear friction phenomena is proposed for guidance or musical purposes. It enables to synthesize different types of sounds, such a creaky door, a singing glass or a squeaking wet plate. This approach is based on the action/object paradigm that enables to propose a synthesis strategy using classical linear filtering techniques (source/resonance approach) which provide an efficient implementation. Within this paradigm, a sound can be considered as the result of an action (e.g. impacting, rubbing, ...) on an object (plate, bowl, ...). However, in the case of non linear friction phenomena, simulating the physical coupling between the action and the object with a completely decoupled source/resonance model is a real and relevant challenge. To meet this challenge, we propose to use a synthesis model of the source that is tuned on recorded sounds according to physical and spectral observations. This model enables to synthesize many types of non linear behaviors. A control strategy of the model is then proposed by defining a flexible physically informed mapping between a descriptor, and the non linear synthesis behavior. Finally, potential applications to the remediation of motor diseases are presented. In all sections, video and audio materials are available at the following URL: http://www.lma.cnrs-mrs.fr/~kronland/thoretDAFx2013

From shape to sound: sonification of two dimensional curves by reenaction of biological movements

International audienceSonifying two dimensional data is a common problem. In this study, we propose a method to synthesize sonic metaphors of two dimensional curves based on the mental representation of the sound produced by the friction of the pencil when somebody is drawing or writing on a paper. The relevance of such approach is firstly presented. Secondly, synthesis of friction sounds allows to investigate the relevance of the kinematics in the perception of a gesture underlying a sound. In third part, a biological law linking the curvature of a shape to the velocity of the gesture which have draw the shape is calibrated from the auditory point of view. It enables to generate friction sounds with a physically based synthesis model from a given shape

Sonifying drawings: characterization of perceptual attributes of sounds produced by human gestures

International audienceFriction sounds produced by the pencil of a person who is drawing on a paper are audible and could bring information about his/her gestures. This study firstly focuses on the perceptual significance of the morphology of such sounds, and to what extent gestures could be retrieved by sounds. Sounds recorded during drawing sessions were used in association tests where subjects had to univocally associate friction sounds with different shapes. Results showed that subjects were able to associate sounds with correct shapes and that the auditory characterization of the shape depended on the velocity profile.\\ A sonification strategy was then proposed for human drawings using a friction sound synthesis model. Inspired by the work of Viviani et al. (1982) who exhibited a 2/3-power law relation between kinetics and shape curvature for visual perception, experiments were carried out with the synthesis model, where subjects were asked to adjust the power law exponent so that the most realistic sound was obtained. Results revealed an exponent close to 2/3 as previously found in vision and thus highlighted a similar power law in the auditory modality, providing an ecological way to determine velocity profiles from static shapes and to generate sounds coherent with human gestures

Synchronizing Gestures with Friction Sounds: Work in Progress

International audienceThis paper presents a work in progress dealing with the sensorimotor relation between auditory perception and graphical movements. An experiment where subjects were asked to synchronize their gestures with synthetic friction sounds is presented. A first qualitative analysis enabled to evaluate the influence of different intrinsic sound parameters on the characteristics of the synchronized gesture. This preliminary experiment provides a formal framework for a wider study which aims to evaluate the relation between audition, vision and gestures

Navigating in a space of synthesized interaction-sounds: rubbing, scratching and rolling sounds

International audienceIn this paper, we investigate a control strategy of synthesized interaction-sounds. The framework of our research is based on the {action/object} paradigm that considers that sounds result from an action on an object. This paradigm presumes that there exists some sound invariants, i.e. perceptually relevant signal morphologies that carry information about the action or the object. Some of these auditory cues are considered for rubbing, scratching and rolling interactions. A generic sound synthesis model, allowing the production of these three types of interaction together with a control strategy of this model are detailed. The proposed control strategy allows the users to navigate continuously in an ''action space'', and to morph between interactions, e.g. from rubbing to rolling

Handwriting Movement Sonification for the Rehabilitation of Dysgraphia

International audienceSounds can be used to inform about the correctness of an ongoing movement, without directly interfering with the visual and proprioceptive feedback. Furthermore, the dynamic features of sounds make them particularly appropriate means of accessing the spatiotemporal characteristics of movements. Finally, because of their playful characteristics, sounds are potentially effective for motivating children in particular need of such assistance. Based on these theoretical considerations, the present work investigated the relevance of applying an online auditory feedback to spatiotemporal characteristics of handwriting for the rehabilitation of handwriting learning disabilities

Timbre from Sound Synthesis and High-level Control Perspectives

International audienceExploring the many surprising facets of timbre through sound manipulations has been a common practice among composers and instrument makers of all times. The digital era radically changed the approach to sounds thanks to the unlimited possibilities offered by computers that made it possible to investigate sounds without physical constraints. In this chapter we describe investigations on timbre based on the analysis by synthesis approach that consists in using digital synthesis algorithms to reproduce sounds and further modify the parameters of the algorithms to investigate their perceptual relevance. In the first part of the chapter timbre is investigated in a musical context. An examination of the sound quality of different wood species for xylophone making is first presented. Then the influence of instrumental control on timbre is described in the case of clarinet and cello performances. In the second part of the chapter, we mainly focus on the identification of sound morphologies, so called invariant sound structures responsible for the evocations induced by environmental sounds by relating basic signal descriptors and timbre descriptors to evocations in the case of car door noises, motor noises, solid objects, and their interactions

A window on human and artificial cognition with reverse correlation

International audienc

sleepy voices

Supporting data for the paper: Thoret, E., Andrillon, T., Gauriau, C., Leger, D., Pressnitzer, D. (2022) Sleep deprivation measured by voice analysis, bioRxiv. 10.1101/2022.11.17.516913 10.1101/2022.11.17.51691