Physical Interactions with Digital Strings - A hybrid approach to a digital keyboard instrument

A new hybrid approach to digital keyboard playing is presented, where the actual acoustic sounds from a digital keyboard are captured with contact microphones and applied as excitation signals to a digital model of a prepared piano, i.e., an extended wave-guide model of strings with the possibility of stopping and muting the strings at arbitrary positions. The parameters of the string model are controlled through TouchKeys multitouch sensors on each key, combined with MIDI data and acoustic signals from the digital keyboard frame, using a novel mapping. The instrument is evaluated from a performing musician's perspective, and emerging playing techniques are discussed. Since the instrument is a hybrid acoustic-digital system with several feedback paths between the domains, it provides for expressive and dynamic playing, with qualities approaching that of an acoustic instrument, yet with new kinds of control. The contributions are two-fold. First, the use of acoustic sounds from a physical keyboard for excitations and resonances results in a novel hybrid keyboard instrument in itself. Second, the digital model of "inside piano" playing, using multitouch keyboard data, allows for performance techniques going far beyond conventional keyboard playing

Re-Sonification of Objects, Events, and Environments

abstract: Digital sound synthesis allows the creation of a great variety of sounds. Focusing on interesting or ecologically valid sounds for music, simulation, aesthetics, or other purposes limits the otherwise vast digital audio palette. Tools for creating such sounds vary from arbitrary methods of altering recordings to precise simulations of vibrating objects. In this work, methods of sound synthesis by re-sonification are considered. Re-sonification, herein, refers to the general process of analyzing, possibly transforming, and resynthesizing or reusing recorded sounds in meaningful ways, to convey information. Applied to soundscapes, re-sonification is presented as a means of conveying activity within an environment. Applied to the sounds of objects, this work examines modeling the perception of objects as well as their physical properties and the ability to simulate interactive events with such objects. To create soundscapes to re-sonify geographic environments, a method of automated soundscape design is presented. Using recorded sounds that are classified based on acoustic, social, semantic, and geographic information, this method produces stochastically generated soundscapes to re-sonify selected geographic areas. Drawing on prior knowledge, local sounds and those deemed similar comprise a locale's soundscape. In the context of re-sonifying events, this work examines processes for modeling and estimating the excitations of sounding objects. These include plucking, striking, rubbing, and any interaction that imparts energy into a system, affecting the resultant sound. A method of estimating a linear system's input, constrained to a signal-subspace, is presented and applied toward improving the estimation of percussive excitations for re-sonification. To work toward robust recording-based modeling and re-sonification of objects, new implementations of banded waveguide (BWG) models are proposed for object modeling and sound synthesis. Previous implementations of BWGs use arbitrary model parameters and may produce a range of simulations that do not match digital waveguide or modal models of the same design. Subject to linear excitations, some models proposed here behave identically to other equivalently designed physical models. Under nonlinear interactions, such as bowing, many of the proposed implementations exhibit improvements in the attack characteristics of synthesized sounds.Dissertation/ThesisPh.D. Electrical Engineering 201

Model-based digital pianos: from physics to sound synthesis

International audiencePiano is arguably one of the most important instruments in Western music due to its complexity and versatility. The size, weight, and price of grand pianos, and the relatively simple control surface (keyboard) have lead to the development of digital counterparts aiming to mimic the sound of the acoustic piano as closely as possible. While most commercial digital pianos are based on sample playback, it is also possible to reproduce the sound of the piano by modeling the physics of the instrument. The process of physical modeling starts with first understanding the physical principles, then creating accurate numerical models, and finally finding numerically optimized signal processing models that allow sound synthesis in real time by neglecting inaudible phenomena, and adding some perceptually important features by signal processing tricks. Accurate numerical models can be used by physicists and engineers to understand the functioning of the instrument, or to help piano makers in instrument development. On the other hand, efficient real-time models are aimed at composers and musicians performing at home or at stage. This paper will overview physics-based piano synthesis starting from the computationally heavy, physically accurate approaches and then discusses the ones that are aimed at best possible sound quality in real-time synthesis

A digital waveguide-based approach for Clavinet modeling and synthesis

The Clavinet is an electromechanical musical instrument produced in the mid-twentieth century. As is the case for other vintage instruments, it is subject to aging and requires great effort to be maintained or restored. This paper reports analyses conducted on a Hohner Clavinet D6 and proposes a computational model to faithfully reproduce the Clavinet sound in real time, from tone generation to the emulation of the electronic components. The string excitation signal model is physically inspired and represents a cheap solution in terms of both computational resources and especially memory requirements (compared, e.g., to sample playback systems). Pickups and amplifier models have been implemented which enhance the natural character of the sound with respect to previous work. A model has been implemented on a real-time software platform, Pure Data, capable of a 10-voice polyphony with low latency on an embedded device. Finally, subjective listening tests conducted using the current model are compared to previous tests showing slightly improved results

Physically Informed Subtraction of a String's Resonances from Monophonic, Discretely Attacked Tones : a Phase Vocoder Approach

A method for the subtraction of a string's oscillations from monophonic, plucked- or hit-string tones is presented. The remainder of the subtraction is the response of the instrument's body to the excitation, and potentially other sources, such as faint vibrations of other strings, background noises or recording artifacts. In some respects, this method is similar to a stochastic-deterministic decomposition based on Sinusoidal Modeling Synthesis [MQ86, IS87]. However, our method targets string partials expressly, according to a physical model of the string's vibrations described in this thesis. Also, the method sits on a Phase Vocoder scheme. This approach has the essential advantage that the subtraction of the partials can take place \instantly", on a frame-by-frame basis, avoiding the necessity of tracking the partials and therefore availing of the possibility of a real-time implementation. The subtraction takes place in the frequency domain, and a method is presented whereby the computational cost of this process can be reduced through the reduction of a partial's frequency-domain data to its main lobe. In each frame of the Phase Vocoder, the string is encoded as a set of partials, completely described by four constants of frequency, phase, magnitude and exponential decay. These parameters are obtained with a novel method, the Complex Exponential Phase Magnitude Evolution (CSPME), which is a generalisation of the CSPE [SG06] to signals with exponential envelopes and which surpasses the nite resolution of the Discrete Fourier Transform. The encoding obtained is an intuitive representation of the string, suitable to musical processing

Analysis and parametric synthesis of the piano sound

Tässä työssä tutkitaan pianon äänentuottomekanismia sekä akustisia ominaisuuksia. Tarkoituksena on luoda lähtökohdat pianon äänen parametriselle mallintamiselle. Lisäksi tutkitaan pianon äänen tärkeimpiä ominaisuuksia, kuten epäharmonisuutta, osaäänesten monimutkaista vaimenemisprosessia, kaikupohjan ja pedaalin ominaisuuksia sekä näiden tekijöiden vaikutuksia ääneen. Flyygelin ja pystypianon eroja tarkastellaan lyhyesti. Koska digitaalinen aaltojohtomallinnus tarjoaa parhaat lähtökohdat fysikaaliseen soitinmallinnukseen, tämä työ pohjautuu tähän tekniikkaan. Digitaalisen aaltojohtomallinnuksen pääpiirteet esitellään, kuten myös pianon kannalta olennaisimmat mallinnukseen liittyvät asiat. Lisäksi esitellään uusi tekniikka häviösuotimen suunnittelua varten, sekä annetaan muutama esimerkki käytännön suodinsuunnittelusta tällä tekniikalla. Tämän lisäksi tarkastellaan kaikupedaalin mallintamista sekä suoritetaan signaalianalyysi tehokkaan mallinnusalgoritmin löytämiseksi. Analysoitavat signaalit on äänitetty kahdessa äänityssessiossa vuoden 2005 aikana.In this thesis, an overview of the sound production mechanism of the piano is given. The acoustical properties of the instrument are studied in order to make a baseline for a physical and parametric model for the piano. In addition, the most important features of the piano sound, such as inharmonicity, the complicated decay process of the tones and the properties of the soundboard and the pedals, are investigated. The differences between the grand piano and the upright piano are considered in brief. As the digital waveguide technique is the most feasible physics-based sound synthesis technique at the moment, the synthesis procedure that is followed in this thesis is based on this technique. An overview of the main aspects of this synthesis scheme is given, and the most important modeling issues are taken into account from the piano sound synthesis point of view. A novel filter design technique for modeling the losses occurring in the piano sound is presented with some practical design examples. In addition, the modeling of the sustain pedal is discussed and signal analysis is performed in order to gather information for the synthetic sustain pedal algorithm. The analyzed signals are obtained from two recording sessions which were carried out in two parts during the year 2005