Analyse-synthèse de sons impulsifs : approches physique et perceptive

This PhD thesis concerns the analysis and synthesis of impulsive sounds. The main specificity of this work lies in its pluridisciplinary approach. Notably, this study calls on vibratory mechanics, digital non stationary signal processing and psychoacoustics. To establish efficient synthesis methods, the combination of these complementary approaches have shown to be useful and pertinent. From a physical point of view, an impulsive sound is the result of a freely oscillating resonator excited by an impact. From a perceptual point of view, many psychoacoustic tests have shown that human hearing is able to identify on impulsive sound from its temporal damping (which depends on the characteristic of the material) and from its spectral contents correlated to the pitch (and which indicates the vibrating structure geometry). This work leads to the implementation of efficient synthesis methods using digital waveguide models. They can simulate the vibratory behavior of unidimensional simple structures (like beams) and multidimensional ones (like membranes and thin plates). When it is possible, these models are based on the physical modelling of the structure. The model parameters are estimated from the analysis of natural sounds and from the physical features of the structure. Otherwise, a synthesis model is established in the case of complex multidimensional structures where analytical solutions do not exist. The model's parameters are estimated from the analysis of joint sound representations (like time-scale representations) which is well adapted to the analysis of non stationary signals. Finally, an application of this study concerning the synthesis of piano sounds is described. One of the presented models is used to reproduce the impulse response of the piano soundboard. The models parameters are estimated from the analysis of experimental signals obtained from measurements made on a real piano in an anechoic room. From these measurements, transfer functions of the soundboard and its acoustic radiation are estimated to virtually reproduce the acoustic field at the pianist's head level.Ce travail de thèse porte sur l'analyse et la synthèse de sons impulsifs. Une des particularités de cette étude réside dans son approche pluridisciplinaire. Elle fait notamment appel à la mécanique vibratoire, aux méthodes de traitement de signaux non stationnaires et à la psychoacoustique. Dans le but d'établir des méthodes de synthèse efficaces, il est de plus en plus courant d'allier plusieurs approches qui peuvent être complémentaires. D'un point de vue physique, un son impulsif est le résultat de la mise en vibration d'un système résonant en régime d'oscillations libres. D'un point de vue perceptif, de nombreux tests psychoacoustiques montrent que l'oreille est capable d'identifier un son impulsif à partir de son amortissement (caractéristique du matériau) ainsi que de son contenu spectral qui définit la hauteur tonale (caractéristique de la géométrie de la structure). Cette étude a débouché sur la mise en oeuvre de modèles de synthèse permettant de simuler le comportement vibratoire de structures simples unidimensionnelles (du type poutre) et multidimensionnelles (du type membrane ou plaque mince) qui sont basés, lorsque cela est possible, sur la modélisation physique de la structure. Les paramètres des modèles sont définis par l'analyse de sons naturels et des grandeurs physiques caractéristiques. Par ailleurs, un modèle de synthèse a pu être mis en uvre dans le cas de structures plus complexes où les solutions analytiques ne sont pas connues. Dans ce cas, les paramètres du modèle sont directement estimés par l'analyse d'une représentation conjointe du son, du type temps-échelle, en prenant en compte des grandeurs perceptives caractéristiques. Enfin, une application concernant la synthèse du son de piano est décrite. Un des modèles de synthèse a été utilisé pour reproduire la réponse impulsionnelle de la table d'harmonie du piano, qui est une structure multidimensionnelle complexe. Les paramètres du modèle ont été estimés à partir de l'analyse de signaux expérimentaux obtenus par des mesures effectuées sur un piano placé en chambre anéchoïque. Dans le cadre de la réalité virtuelle sonore, des fonctions de transfert de la table d'harmonie et de son rayonnement acoustique ont également été estimées afin de reproduire virtuellement le champ acoustique au niveau de la tête d'un pianiste

An evidence of the role of the cellists' postural movements in the score metric cohesion

International audienceWhile playing expressively, cellists tend to produce postural movements, which seem to be part of their musical discourse. This article actually highlights the existence of a metric embodiment, i.e. a natural encoding of the score metric structure through specific periodicities of the musicians' postural movements. By the means of constrained postu-ral conditions, we assess the degradation of the metric coupling between postural and bowing gestures among several cellists. Results reveal that bowing displacements should be in coherence with their postural behavior in order to ensure a correct encoding of the metric hierarchy

ESPRIT in Gabor Frames

International audienceThis articles tackles the estimation of mode parameters in recorded sounds of resonant objects. High resolution methods such as the ESPRIT method have already proved to be of great use for this sort of purpose. However, these methods being model-sensitive, their application to real-life audio signals can lead to results that are not satisfactory enough for a consistent re-synthesis. This is especially the case when the computational cost makes it impossible to analyse the signal in totality, or when the signal presents a high number of components. Significant improvements have already been achieved by decomposing the signal into several sub-band filtered versions, and by applying the ESPRIT algorithm on each of the resulting signals. It is shown in this article that the ESPRIT algorithm can be efficiently applied on time-frequency representations of the signal obtained using Gabor frames. Numerical tests that highlight the advantages of such an approach are also detailed. In addition to the advantages offered by the sub-band approach, the solid Gabor frame formalism combined with the ESPRIT method allows a flexible and sharp analysis on selected regions of the time-frequency plane, and leads to re-synthesis which are perceptually very close to the original sounds

On the potentiality of abstract sounds in perception research

International audienceRecognition of sound sources and events is an important process in sound perception and has been studied in many research domains. Conversely sounds that cannot be recognized are not often studied except by electroacoustic music composers. Besides, considerations on recognition of sources might help to address the problem of stimulus selection and categorization of sounds in the context of perception research. This paper introduces what we call abstract sounds with the existing musical background and shows their relevance for different applications

Rolling sound synthesis : work in progress

International audienceThis paper presents a physically informed rolling sound synthesis model for the MetaSon synthesis platform. The aim of this sound synthesis platform will be shortly described. As shown in the state of the art, both in terms of sound effects and proposed controls, existing models can be improved. Some details on asymmetric rolling objects will be given and the sound synthesis model will be exposed. Perspectives for further studies and work in progress will be discussed

Une preuve du rôle des mouvements posturaux des violoncellistes dans la cohésion métrique de la partition

International audienceWhile playing expressively, cellists tend to produce postural movements, which seem to be part of their musical discourse. This article actually highlights the existence of a metric embodiment, i.e. a natural encoding of the score metric structure through specific periodicities of the musicians' postural movements. By the means of constrained postu-ral conditions, we assess the degradation of the metric coupling between postural and bowing gestures among several cellists. Results reveal that bowing displacements should be in coherence with their postural behavior in order to ensure a correct encoding of the metric hierarchy

Machines that listen: towards a machine listening model based on perceptual descriptors

International audienceUnderstanding how humans use auditory cues to interpret their surroundings is a challenge in various fields, such as music information retrieval, computational musicology and sound modeling. The most common ways of exploring the links between signal properties and human perception are through di↵erent kinds of listening tests, such as catego-rization or dissimilarity evaluations. Although such tests have made it possible to point out perceptually relevant signal structures linked to specific sound categories, rather small sound corpora (100-200 sounds in a categorization protocol) can be tested this way. The number of subjects generally do not exceed 20-30, since it is also very time consuming for an experimenter to include too many subjects. In this study we wanted to test whether it is possible to evaluate larger sound corpora through machine learning models for automatic timbre characterization. A selection of 1800 sounds produced by either wooden or metallic objects were analyzed by a deep learning model that was either trained on a perceptually salient acoustic descriptor or on a signal descriptor based on the energy contents of the signal. A random selection of 180 sounds from the same corpus was tested perceptually and used to compare sound categories obtained from human evaluations with those obtained from the deep learning model. Results revealed that when the model was trained on the perceptually relevant acoustic descriptors it performed a classification that was very close to the results obtained in the listening test, which is a promising result suggesting that such models can be trained to perform perceptually coherent evaluations of sounds

Exploring the usability of sound strategies for guiding task: toward a generalization of sonification design

International audienceThis article aims at providing a new Parameter Mapping Sonification approach in order to facilitate and generalize sonification design for different applications. First a definition of the target as a con-cept that enables a general sonification strategy that is not limited to specific data types is given. This concept intends to facilitate the sepa-ration between sound and information to display. Rather than directly displaying data dimensions through the variation of a specific sound pa-rameter, the approach aims at displaying the distance between a given data value and the requested value. Then a taxonomy of sound strategies based on sound that allow the construction of several strategy types is presented. Finally, several sound strategies are evaluated with a user ex-periment and the taxonomy is discussed on the basis of user's guidance behavior during a guiding task

Perception of the object attributes for sound synthesis purposes

International audienceThis paper presents a work in progress on the perception of the attributes of the shape of a resonant object. As part of the ecological approach to perception-assuming that a sound contains specific morphologies that convey perceptually relevant information responsible for its recognition, called invariants-the PRISM laboratory has developed an environmental sound synthesizer aiming to provide perceptual and intuitive controls for a non-expert user. Following a brief presentation of the di↵erent strategies for controlling the perceptual attributes of the object, we present an experiment conducted with calibrated sounds generated by a physically-informed synthesis model. This test focuses on the perception of the shape of the object, more particularly its width and thickness since these attributes, especially the thickness, have not been much studied in the literature from a perceptual point of view. The first results show that the perception of width is di cult for listeners, while the perception of thickness is much easier. This study allows us to validate the proposed control strategy. Further works are planned to better characterize the perceptual invariants relevant for shape perception

Controlling a non linear friction model for evocative sound synthesis applications

International audienceIn this paper, a flexible strategy to control a synthesis model of sounds produced by non linear friction phenomena is proposed for guidance or musical purposes. It enables to synthesize different types of sounds, such a creaky door, a singing glass or a squeaking wet plate. This approach is based on the action/object paradigm that enables to propose a synthesis strategy using classical linear filtering techniques (source/resonance approach) which provide an efficient implementation. Within this paradigm, a sound can be considered as the result of an action (e.g. impacting, rubbing, ...) on an object (plate, bowl, ...). However, in the case of non linear friction phenomena, simulating the physical coupling between the action and the object with a completely decoupled source/resonance model is a real and relevant challenge. To meet this challenge, we propose to use a synthesis model of the source that is tuned on recorded sounds according to physical and spectral observations. This model enables to synthesize many types of non linear behaviors. A control strategy of the model is then proposed by defining a flexible physically informed mapping between a descriptor, and the non linear synthesis behavior. Finally, potential applications to the remediation of motor diseases are presented. In all sections, video and audio materials are available at the following URL: http://www.lma.cnrs-mrs.fr/~kronland/thoretDAFx2013