Re-Sonification of Objects, Events, and Environments

abstract: Digital sound synthesis allows the creation of a great variety of sounds. Focusing on interesting or ecologically valid sounds for music, simulation, aesthetics, or other purposes limits the otherwise vast digital audio palette. Tools for creating such sounds vary from arbitrary methods of altering recordings to precise simulations of vibrating objects. In this work, methods of sound synthesis by re-sonification are considered. Re-sonification, herein, refers to the general process of analyzing, possibly transforming, and resynthesizing or reusing recorded sounds in meaningful ways, to convey information. Applied to soundscapes, re-sonification is presented as a means of conveying activity within an environment. Applied to the sounds of objects, this work examines modeling the perception of objects as well as their physical properties and the ability to simulate interactive events with such objects. To create soundscapes to re-sonify geographic environments, a method of automated soundscape design is presented. Using recorded sounds that are classified based on acoustic, social, semantic, and geographic information, this method produces stochastically generated soundscapes to re-sonify selected geographic areas. Drawing on prior knowledge, local sounds and those deemed similar comprise a locale's soundscape. In the context of re-sonifying events, this work examines processes for modeling and estimating the excitations of sounding objects. These include plucking, striking, rubbing, and any interaction that imparts energy into a system, affecting the resultant sound. A method of estimating a linear system's input, constrained to a signal-subspace, is presented and applied toward improving the estimation of percussive excitations for re-sonification. To work toward robust recording-based modeling and re-sonification of objects, new implementations of banded waveguide (BWG) models are proposed for object modeling and sound synthesis. Previous implementations of BWGs use arbitrary model parameters and may produce a range of simulations that do not match digital waveguide or modal models of the same design. Subject to linear excitations, some models proposed here behave identically to other equivalently designed physical models. Under nonlinear interactions, such as bowing, many of the proposed implementations exhibit improvements in the attack characteristics of synthesized sounds.Dissertation/ThesisPh.D. Electrical Engineering 201

Impulse Response Modeling of the Box Shaped Acoustic Guitar

Music is the pulse of human lives and is an amazing tool to relieve and re-live. And when it comes to the signal processing, impulse is the pulse of the researchers. The work presented here is focused on impulse response modeling of noted produced by box shaped acoustic guitar. The impulse response is very fundamental behavior of any system. The music note is the convolution of the impulse response and the excitation signal of that guitar. The frequency of the generated music note follows the octave rule. The octave rule can be checked for impulse responses as well. If the excitation signal and impulse response are separated, then an impulse response of a single fret can be used to generate the impulse responses of other frets. Here the music notes are analyzed and synthesized on the basis of the plucking style and plucking expression of the guitar-player. If the impulse response of the musical instrument is known, the output music note can be synthesized in an unusual manner. Researchers have been able to estimate the impulse response by breaking the string of the guitar. Estimating the impulse response from the recorded music notes is possible using the methodology of cepstral domain window. By means of the Adaptive Cepstral Domain Window (ACDW) the author estimated the impulse response of guitar notes. The work has been further extended towards the classification of synthesized notes for plucking style and plucking expression using Neural Network and Machine Learning algorithms

A methodology for investigation of bowed string performance through measurement of violin bowing technique

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (leaves 181-186).Virtuosic bowed string performance in many ways exemplifies the incredible potential of human physical performance and expression. Today, a great deal is known about the physics of the violin family and those factors responsible for its sound capabilities. However, there remains much to be discovered about the intricacies of how players control these instruments in order to achieve their characteristic range and nuance of sound. Today, technology offers the ability to study this player control under realistic, unimpeded playing conditions to lead to greater understanding of these performance skills. Presented here is a new methodology for investigation of bowed string performance that uses a playable hardware measurement system to capture the gestures of right hand violin bowing technique. Building upon previous Hyperstring research, this measurement system was optimized to be small, lightweight, and portable and was installed on a carbon fiber violin bow and an electric violin to enable study of realistic, unencumbered violin performances. Included in the system are inertial and force sensors, and an electric field position sensor. In order to maximize the applicability of the gesture data provided by this system to related fields of interest, all of the sensors were calibrated in SI units.(cont.) The gesture data captured by these sensors are recorded together with the audio data from the violin as they are produced by violinists in typical playing scenarios. To explore the potential of the bowing measurement system created, a study of standard bowing techniques, such as detache, martele and spiccato, was conducted with expert violinist participants. Gesture data from these trials were evaluated and input to a classifier to examine physical distinctions between bowing techniques, as well as between players. Results from this analysis, and their implications on this methodology will be presented. In addition to this examination of bowing techniques, applications of the measurement system for study of bowed string acoustics and digital music instrument performance, with focus on virtual instruments created from physical models, will be discussed.by Diana Young.Ph.D

Physically Informed Subtraction of a String's Resonances from Monophonic, Discretely Attacked Tones : a Phase Vocoder Approach

A method for the subtraction of a string's oscillations from monophonic, plucked- or hit-string tones is presented. The remainder of the subtraction is the response of the instrument's body to the excitation, and potentially other sources, such as faint vibrations of other strings, background noises or recording artifacts. In some respects, this method is similar to a stochastic-deterministic decomposition based on Sinusoidal Modeling Synthesis [MQ86, IS87]. However, our method targets string partials expressly, according to a physical model of the string's vibrations described in this thesis. Also, the method sits on a Phase Vocoder scheme. This approach has the essential advantage that the subtraction of the partials can take place \instantly", on a frame-by-frame basis, avoiding the necessity of tracking the partials and therefore availing of the possibility of a real-time implementation. The subtraction takes place in the frequency domain, and a method is presented whereby the computational cost of this process can be reduced through the reduction of a partial's frequency-domain data to its main lobe. In each frame of the Phase Vocoder, the string is encoded as a set of partials, completely described by four constants of frequency, phase, magnitude and exponential decay. These parameters are obtained with a novel method, the Complex Exponential Phase Magnitude Evolution (CSPME), which is a generalisation of the CSPE [SG06] to signals with exponential envelopes and which surpasses the nite resolution of the Discrete Fourier Transform. The encoding obtained is an intuitive representation of the string, suitable to musical processing

Implementation and optimization of the synthesis of musical instrument tones using frequency modulation

Im Bereich der elektronischen Musik hat die Frequenzmodulation (FM) als eine effiziente Methode zur Klangsynthese in jüngster Zeit enorm an Bedeutung gewonnen. In der vorliegenden Arbeit werden Methoden zur Grundfrequenzschätzung und zur FM-Synthese für Musikinstrumentenklänge untersucht, bewertet und optimiert. Dazu wurde im Rahmen dieser Arbeit eine FM Analyse- und Syntheseumgebung entwickelt, in welcher die hier betrachteten Verfahren implementiert wurden. Zur Grundfrequenzschätzung in Musiksignalen wurde ein neuartiges Verfahren auf Basis von Harmonic Pattern Match (HPM) entwickelt, welches eine höhere Schätzungsgenauigkeit als bisher verwendete Verfahren bietet. Hierzu wird nach Festlegung einer geeigneten Teilmenge der Spektraldaten die Autokorrelation sowohl im Zeitals auch im Frequenzbereich analysiert, um Kandidaten für die Grundfrequenz des Signals zu bestimmen. Anschließend wird die Übereinstimmung jedes dieser Kandidaten mit dem Profil der Harmonischen des Musiksignals nach einem effizienten Verfahren analysiert. Das vorgeschlagene Verfahren wurde analysiert und im Kontext mit anderen Verfahren zur Grundfrequenzschätzung bewertet. Die praktische Anwendbarkeit des HPM Verfahrens konnte gezeigt werden. Zur Implementierung einer FM Synthese wird ein Verfahren zur Approximation eines Spektrums auf Basis Genetischer Algorithmen (GA) vorgestellt. Die Problemstellung des GA einschließlich eines Verfahrens zur Bestimmung optimaler FMParameter wird beschrieben. Des Weiteren wurden im Hinblick auf eine optimierte FM-Synthese die Anforderungen an das Trägersignal sowie an den Modulator untersucht, mit dem Ziel einer Vorab-Festlegung des Parameterraums für akkurate Syntheseresultate. Mit dem Ziel einer Datenreduktion bei der FM-Synthese wurde eine stückweise lineare Approximation der Einhüllenden des Trägersignals entwickelt. Einen weiteren Aspekt der Optimierung stellt die Verknüpfung von Formanten in der Matching-Prozedur dar, wobei die Harmonischen der Formanten mit entsprechenden Faktoren gewichtet werden. Auf diese Weise wird eine deutlich genauere Approximation des Timbres des zu synthetisierenden Klangs erreicht. Hierzu wurden die Schätzung der spektralen Einhüllenden und die Extraktion der Formanten analysiert und implementiert. Die im Rahmen dieser Arbeit entwickelte Testumgebung ermöglicht die Schätzung der Parameter und die Analyse und Bewertung der so erzeugten FM-Syntheseresultate.Frequency modulation (FM) as an efficient method to synthesize musical sounds is of great importance in the area of computer music. In this thesis, the estimation of fundamental frequency, the FM synthesis procedure of musical instrument tones and the optimization on FM synthesis were analysed, evaluated, improved and implemented. A FM analysis and synthesis environment was developed, in which the presented work in this thesis were implemented. For the estimation of fundamental frequency of music signals, an algorithm based on harmonic pattern match (HPM) was designed to achieve more reliable estimation accuracy. After defining the spectrum subset, the autocorrelation was applied on the spectrum subset to exploiting candidates of fundamental frequency, and an efficient mechanism to evaluate the match between each candidate and the harmonic pattern of the musical signal was designed. Evaluation of the proposed algorithm and several other estimation algorithms was performed. For the implementation of FM synthesis, the matching procedure of spectra using genetic algorithm (GA) was described, including the definition of the task in GA and the searching procedure of optimized FM parameters through GA. For the optimization on FM synthesis, the requirements of carrier and modulator were analysed and the parameter space was examined, based on which a method for the predetermination of parameter space was designed to achieve accurate synthesis results. For data reduction in FM synthesis, the piecewise linear approximation of the carrier amplitude envelope was designed. Further step on the FM synthesis optimization was implemented by the combination of formants in the spectra matching procedure, in which the formant harmonics were emphasized by the weighting coefficients to achieve more accurate timbre of the synthesized sounds. The spectral envelope estimation and the formant extraction were analysed and implemented. For the analysis and implementation of FM synthesis, a testing environment program was developed, offering the functionality of parameter estimation and performance evaluation in FM synthesis

Music in Health and Diseases

It is well recognized that music is a unique and cost-effective solution for the rehabilitation of patients with cognitive deficits. However, music can also be used as a non-invasive and non-pharmacological intervention modality not only for the management of various disease conditions but also for maintaining good health overall. Music-based therapeutic strategies can be used as complementary methods to existing diagnostic approaches to manage cognitive deficits as well as clinical and physiological abnormalities of individuals in need. This book focuses on various aspects of music and its role in enhancing health and recovering from a disease. Chapters explore music as a healing method across civilizations and measure the effect of music on human physiology and functions

ARTIFICIAL INTELLIGENCE-BASED APPROACH TO MODELLING OF PIPE ORGANS

The aim of the project was to develop a new Artificial Intelligence-based method to aid modeling of musical instruments and sound design. Despite significant advances in music technology, sound design and synthesis of complex musical instruments is still time consuming, error prone and requires expert understanding of the instrument attributes and significant expertise to produce high quality synthesised sounds to meet the needs of musicians and musical instrument builders. Artificial Intelligence (Al) offers an effective means of capturing this expertise and for handling the imprecision and uncertainty inherent in audio knowledge and data. This thesis presents new techniques to capture and exploit audio expertise, following extended knowledge elicitation with two renowned music technologist/audio experts, developed and embodied into an intelligent audio system. The Al combined with perceptual auditory modeling ba.sed techniques (ITU-R BS 1387) make a generic modeling framework providing a robust methodology for sound synthesis parameters optimisation with objective prediction of sound synthesis quality. The evaluation, carried out using typical pipe organ sounds, has shown that the intelligent audio system can automatically design sounds judged by the experts to be of very good quality, while significantly reducing the expert's work-load by up to a factor of three and need for extensive subjective tests. This research work, the first initiative to capture explicitly knowledge from audio experts for sound design, represents an important contribution for future design of electronic musical instruments based on perceptual sound quality will help to develop a new sound quality index for benchmarking sound synthesis techniques and serve as a research framework for modeling of a wide range of musical instruments.Musicom Lt