Visual Representation in GENESIS as a tool for Physical Modeling, Sound Synthesis and Musical Composition

International audienceThe motivation of this paper is to highlight the importance of visual representations for artists when modeling and simulating mass-interaction physical networks in the context of sound synthesis and musical composition.GENESIS is a musician-oriented software environment for sound synthesis and musical composition. However, despite this orientation, a substantial amount of effort has been put into building a rich variety of tools based on static or dynamic visual representations of models and of abstractions of their properties. After a quick survey of these tools, we will illustrate the significant role they play in the creative process involved when going from a musical idea and exploration to the production of a complete musical piece. To that aim, our analysis will rely on the work and practice of several artists having used GENESIS in various ways

mi-gen~: An Efficient and Accessible Mass-Interaction Sound Synthesis Toolbox

International audienc

Latent force models for sound: Learning modal synthesis parameters and excitation functions from audio recordings

Latent force models are a Bayesian learning technique that combine physical knowledge with dimensionality reduction - sets of coupled differential equations are modelled via shared dependence on a low-dimensional latent space. Analogously, modal sound synthesis is a technique that links physical knowledge about the vibration of objects to acoustic phenomena that can be observed in data. We apply latent force modelling to sinusoidal models of audio recordings, simultaneously inferring modal synthesis parameters (stiffness and damping) and the excitation or contact force required to reproduce the behaviour of the observed vibrational modes. Exposing this latent excitation function to the user constitutes a controllable synthesis method that runs in real time and enables sound morphing through interpolation of learnt parameters

Designing and Composing for Interdependent Collaborative Performance with Physics-Based Virtual Instruments

Interdependent collaboration is a system of live musical performance in which performers can directly manipulate each other’s musical outcomes. While most collaborative musical systems implement electronic communication channels between players that allow for parameter mappings, remote transmissions of actions and intentions, or exchanges of musical fragments, they interrupt the energy continuum between gesture and sound, breaking our cognitive representation of gesture to sound dynamics. Physics-based virtual instruments allow for acoustically and physically plausible behaviors that are related to (and can be extended beyond) our experience of the physical world. They inherently maintain and respect a representation of the gesture to sound energy continuum. This research explores the design and implementation of custom physics-based virtual instruments for realtime interdependent collaborative performance. It leverages the inherently physically plausible behaviors of physics-based models to create dynamic, nuanced, and expressive interconnections between performers. Design considerations, criteria, and frameworks are distilled from the literature in order to develop three new physics-based virtual instruments and associated compositions intended for dissemination and live performance by the electronic music and instrumental music communities. Conceptual, technical, and artistic details and challenges are described, and reflections and evaluations by the composer-designer and performers are documented

Applications musicales du traitement de signal : synthèse et prospective

L'objet de cette communication est de proposer une synthèse des applications musicales du traitement de signal, des problématiques de recherche qui leur sont liées et des directions prospectives qui se dégagent sur la base de travaux récents dans ce domaine. Après l'exposé de notions préliminaires, relatives au système technique musical et à l'analyse des différentes représentations numériques des informations musicales, cette synthèse se concentre sur trois types de fonctions principales : la synthèse et le traitement des sons musicaux, la spatialisation sonore et les technologies d'indexation et d'accès

Musical Applications of Signal Processing: Synthesis and Prospect

This article aims at providing a synthesis of the musical applications of digital signal processing, of related research issues, and of future directions that emerge from recent works in that field. After introducing preliminary notions related to the music technical system and to the analysis of different digital representations of music information, it focuses on three main function types: audio synthesis and processing, sound spatialization and audio indexing and access technologies.L’objet de cet article est de proposer une synthèse des applications musicales du traitement de signal, des problématiques de recherche qui leur sont liées et des directions prospectives qui se dégagent sur la base de travaux récents dans ce domaine. Après l’exposé de notions préliminaires, relatives au système technique musical et à l’analyse des différentes représentations numériques des informations musicales, cette synthèse se concentre sur trois types de fonctions principales : la synthèse et le traitement des sons musicaux, la spatialisation sonore et les technologies d’indexation et d’accès

Accurate sound synthesis of 3D object collisions in interactive virtual scenarios

Questa tesi affronta lo studio di algoritmi efficienti per la sintesi di suoni risultanti dalla collisione di oggetti generici, partendo da una descrizione fisica del problema. L'obiettivo della ricerca e' lo sviluppo di strumenti in grado di aumentare l'accuratezza del feedback uditivo in ambienti di realta' virtuale attraverso un approccio basato sulla fisica, senza il bisogno quindi di far riferimento a suoni pre-registrati. Data la loro versatilita' nel trattare geometrie complesse, i metodi agli elementi finiti (FEM) sono stati scelti per la discretizzazione spaziale di generici risonatori tridimensionali. Le risultanti equazioni discrete sono riarrangiate in modo da disaccoppiare i modi normali del sistema tramite l'utilizzo di tecniche di Analisi e Sintesi Modale. Queste tecniche, infatti, portano convenientemente ad algoritmi computazionalmente efficienti per la sintesi del suono. Implementazioni di esempio di tali algoritmi sono state sviluppate facendo uso solo di software open-source: questo materiale a corredo della tesi permette una migliore riproducibilita' dei risultati di questa tesi da parte di ricercatori aventi una preparazione nel campo della sintesi audio. I risultati originali presenti in questo lavoro includono: i tecniche efficienti basate sulla fisica che aiutano l'implementazione in tempo reale di algoritmi di sintesi del suono su hardware comune; ii un metodo per la gestione efficiente dei dati provenienti da analisi FEM che, assieme ad un modello espressivo per la dissipazione, permette di calcolare l'informazione caratterizzante un oggetto risonante e salvarla in una struttura dati compatta iii una trasformazione nel dominio discreto del tempo su due diverse rappresentazioni nello spazio degli stati di filtri digitali del secondo ordine, che permette il calcolo esatto di variabili derivate come la velocita' e l'energia di un risonatore anche quando semplici realizzazioni a soli poli sono impiegate i un'efficiente realizzazione multirate di un banco parallelo di risonatori, derivata usando una suddivisione con Quadrature-Mirror-Filters (QMF). Confrontata con lavori simili presenti in letteratura, questa realizzazione permette l'uso di eccitazione nonlineare in feedback per un banco di risonatori in multirate: l'idea chiave consiste nello svolgere un cambio di stato adattivo nel banco di risonatori, muovendo i risonatori dalla frequenza di campionamento elevata, usata per il processamento della fase transiente, ad un insieme di sottofrequenze ridotte usate durante l'evoluzione in stato libero del sistema.This thesis investigates efficient algorithms for the synthesis of sounds produced by colliding objects, starting from a physical description of the problem. The objective of this investigation is to provide tools capable of increasing the accuracy of the synthetic auditory feedback in virtual environments through a physics-based approach, hence without the need of pre-recorded sounds. Due to their versatility in dealing with complex geometries, Finite Element Methods (FEM) are chosen for the space-domain discretization of generic three-dimensional resonators. The resulting state-space representations are rearranged so as to decouple the normal modes in the corresponding equations, through the use of Modal Analysis/Synthesis techniques. Such techniques, in fact, conveniently lead to computationally efficient sound synthesis algorithms. The whole mathematical treatment develops until deriving such algorithms. Finally, implementation examples are provided which rely only on open-source software: this companion material guarantees the reproducibility of the results, and can be handled without much effort by most researchers having a background in sound processing. The original results presented in this work include: i efficient physics-based techniques that help implement real-time sound synthesis algorithms on common hardware; ii a method for the efficient management of FEM data which, by working together with an expressive damping model, allows to pre-compute the information characterizing a resonating object and then to store it in a compact data structure; iii a time-domain transformation of the state-space representation of second-order digital filters, allowing for the exact computation of dependent variables such as resonator velocity and energy, even when simple all-pole realizations are used; iv an efficient multirate realization of a parallel bank of resonators, which is derived using a Quadrature-Mirror-Filters (QMF) subdivision. Compared to similar works previously proposed in the literature, this realization allows for the nonlinear feedback excitation of a multirate filter bank: the key idea is to perform an adaptive state change in the resonator bank, by switching the sampling rate of the resonators from a common highest value, used while processing the initial transient of the signals at full bandwidth, to a set of lower values in ways to enable a multirate realization of the same bank during the steady state evolution of the signals