Analysis and resynthesis of polyphonic music

This thesis examines applications of Digital Signal Processing to the analysis, transformation, and resynthesis of musical audio. First I give an overview of the human perception of music. I then examine in detail the requirements for a system that can analyse, transcribe, process, and resynthesise monaural polyphonic music. I then describe and compare the possible hardware and software platforms. After this I describe a prototype hybrid system that attempts to carry out these tasks using a method based on additive synthesis. Next I present results from its application to a variety of musical examples, and critically assess its performance and limitations. I then address these issues in the design of a second system based on Gabor wavelets. I conclude by summarising the research and outlining suggestions for future developments

Non-Standard Sound Synthesis with Dynamic Models

Full version unavailable due to 3rd party copyright restrictions.This Thesis proposes three main objectives: (i) to provide the concept of a new generalized non-standard synthesis model that would provide the framework for incorporating other non-standard synthesis approaches; (ii) to explore dynamic sound modeling through the application of new non-standard synthesis techniques and procedures; and (iii) to experiment with dynamic sound synthesis for the creation of novel sound objects. In order to achieve these objectives, this Thesis introduces a new paradigm for non-standard synthesis that is based in the algorithmic assemblage of minute wave segments to form sound waveforms. This paradigm is called Extended Waveform Segment Synthesis (EWSS) and incorporates a hierarchy of algorithmic models for the generation of microsound structures. The concepts of EWSS are illustrated with the development and presentation of a novel non-standard synthesis system, the Dynamic Waveform Segment Synthesis (DWSS). DWSS features and combines a variety of algorithmic models for direct synthesis generation: list generation and permutation, tendency masks, trigonometric functions, stochastic functions, chaotic functions and grammars. The core mechanism of DWSS is based in an extended application of Cellular Automata. The potential of the synthetic capabilities of DWSS is explored in a series of Case Studies where a number of sound object were generated revealing (i) the capabilities of the system to generate sound morphologies belonging to other non-standard synthesis approaches and, (ii) the capabilities of the system of generating novel sound objects with dynamic morphologies. The introduction of EWSS and DWSS is preceded by an extensive and critical overview on the concepts of microsound synthesis, algorithmic composition, the two cultures of computer music, the heretical approach in composition, non- standard synthesis and sonic emergence along with the thorough examination of algorithmic models and their application in sound synthesis and electroacoustic composition. This Thesis also proposes (i) a new definition for “algorithmic composition”, (ii) the term “totalistic algorithmic composition”, and (iii) four discrete aspects of non-standard synthesis

Computational Models of Expressive Music Performance: A Comprehensive and Critical Review

Expressive performance is an indispensable part of music making. When playing a piece, expert performers shape various parameters (tempo, timing, dynamics, intonation, articulation, etc.) in ways that are not prescribed by the notated score, in this way producing an expressive rendition that brings out dramatic, affective, and emotional qualities that may engage and affect the listeners. Given the central importance of this skill for many kinds of music, expressive performance has become an important research topic for disciplines like musicology, music psychology, etc. This paper focuses on a specific thread of research: work on computational music performance models. Computational models are attempts at codifying hypotheses about expressive performance in terms of mathematical formulas or computer programs, so that they can be evaluated in systematic and quantitative ways. Such models can serve at least two purposes: they permit us to systematically study certain hypotheses regarding performance; and they can be used as tools to generate automated or semi-automated performances, in artistic or educational contexts. The present article presents an up-to-date overview of the state of the art in this domain. We explore recent trends in the field, such as a strong focus on data-driven (machine learning) approaches; a growing interest in interactive expressive systems, such as conductor simulators and automatic accompaniment systems; and an increased interest in exploring cognitively plausible features and models. We provide an in-depth discussion of several important design choices in such computer models, and discuss a crucial (and still largely unsolved) problem that is hindering systematic progress: the question of how to evaluate such models in scientifically and musically meaningful ways. From all this, we finally derive some research directions that should be pursued with priority, in order to advance the field and our understanding of expressive music performance

Physically-based auralization : design, implementation, and evaluation

The aim of this research is to implement an auralization system that renders audible a 3D model of an acoustic environment. The design of such a system is an iterative process where successive evaluation of auralization quality is utilized to further refine the model and develop the rendering methods. The work can be divided into two parts corresponding to design and implementation of an auralization system and evaluation of the system employing objective and subjective criteria. The presented auralization method enables both static and dynamic rendering. In dynamic rendering positions and orientations of sound sources, surfaces, or a listener can change. These changes are allowed by modeling the direct sound and early reflections with the image-source method. In addition, the late reverberation is modeled with a time-invariant recursive digital filter structure. The core of the thesis deals with the processing of image sources for auralization. The sound signal emitted by each image source is processed with digital filters modeling such acoustic phenomena as sound source directivity, distance delay and attenuation, air and material absorption, and the characteristics of spatial hearing. The digital filter design and implementation of these filters are presented in detail. The traditional image-source method has also been extended to handle diffraction in addition to specular reflections. The evaluation of quality of the implemented auralization system was performed by comparing recorded and auralized soundtracks subjectively. The compared soundtracks were prepared by recording sound signals in a real room and by auralizing these signals with a 3D model of the room. The auralization quality was assessed with objective and subjective methods. The objective analysis was based on both traditional room acoustic criteria and on a simplified auditory model developed for this purpose. This new analysis method mimics the behavior of human cochlea. Therefore, with the developed method, impulse responses and sound signals can be visualized with similar time and frequency resolution as human hearing applies. The evaluation was completed subjectively by conducting listening tests. The utilized listening test methodology is explained and the final results are presented. The results show that the implemented auralization system provides plausible and natural sounding auralizations in rooms similar to the lecture room employed for evaluation.reviewe

An Abstraction Framework for Tangible Interactive Surfaces

This cumulative dissertation discusses - by the example of four subsequent publications - the various layers of a tangible interaction framework, which has been developed in conjunction with an electronic musical instrument with a tabletop tangible user interface. Based on the experiences that have been collected during the design and implementation of that particular musical application, this research mainly concentrates on the definition of a general-purpose abstraction model for the encapsulation of physical interface components that are commonly employed in the context of an interactive surface environment. Along with a detailed description of the underlying abstraction model, this dissertation also describes an actual implementation in the form of a detailed protocol syntax, which constitutes the common element of a distributed architecture for the construction of surface-based tangible user interfaces. The initial implementation of the presented abstraction model within an actual application toolkit is comprised of the TUIO protocol and the related computer-vision based object and multi-touch tracking software reacTIVision, along with its principal application within the Reactable synthesizer. The dissertation concludes with an evaluation and extension of the initial TUIO model, by presenting TUIO2 - a next generation abstraction model designed for a more comprehensive range of tangible interaction platforms and related application scenarios

A new user interface for musical timbre design

This thesis characterises and addresses problems and issues associated with the design of intuitive user interfaces for timbral control. The usability of a range of synthesis methods and representative implementations of these methods is assessed, and three interface architectures - fixed architecture, architecture specification and direct specification - are identified. The characteristics of each of these architectures, as well as problems of usability inherent to each of them are discussed; it is argued that none of them provide intuitive tools for the manipulation and control of timbre. The study examines the nature of timbre and the notion of timbre space; different kinds of timbre space are considered and criteria are proposed for the selection of suitable timbre spaces as vehicles for synthesis. A number of listening tests, designed to demonstrate the feasibility of subsequent work, were devised and carried out; the results of these tests provide evidence that, where Euclidean distances between sounds located in a given timbre space are reflected in perceptual distances, the ability of subjects to detect relative distances in different parts of the space varies with the perceptual granularity of the space. Three contrasting timbre spaces conforming to the proposed criteria for use in synthesis are constructed; the purpose of these spaces is to provide an environment for a novel user interaction approach for timbral design which incorporates a search strategy based on weighted centroid localization. Two prototypes which exemplify the proposed approach in alternative ways are designed, implemented and tested with potential users in order to validate the approach; a third contrasting prototype which represents a simple contrasting alternative is tested for purposes of comparison. The results of these tests are evaluated and discussed, and areas of further work identified

Un voyage du son par les fils électroacoustiques : l'art et les nouvelles technologies en Amérique Latine

L'histoire de la musique électroacoustique latino-américaine est longue, intéressante et prolifique, mais peu connue, même régionalement. De nombreux compositeurs nés ou vivants en Amérique latine ont été très actifs à ce titre, dans certains pays depuis plus de 50 ans, mais la disponibilité de l'information et des enregistrements de musique électroacoustique à cet égard et dans cette région a posé de sérieux problèmes aux éducateurs, compositeurs, interprètes, chercheurs, étudiants et au public en général.\ud Compte tenu de cette situation, la question suivante s'est imposée comme point de départ de ma thèse: comment s'est développée la tradition de la création musicale avec les médias électroacoustiques en Amérique Latine. Pour y répondre, j'ai adopté une approche historique en utilisant une méthodologie ethnographique (caractérisée par une immersion à long terme dans le domaine, par des contacts personnels avec des compositeurs et par ma participation et mon souci en ce qui concerne l'évolution des arts faisant appel aux nouvelles technologies en Amérique Latine) dans toute ma recherche. Ayant commencé à travailler dans le domaine de la musique électroacoustique au milieu des années 1970 dans mon Argentine natale, il m'a été très difficile d'obtenir de l'information sur les activités reliées à ce domaine dans des pays voisins et même dans ma propre ville. Bien que difficile, il était néanmoins possible de trouver les enregistrements de compositeurs vivant en Europe ou en Amérique du Nord, mais plus ardu de trouver ceux réalisés par des compositeurs locaux ou régionaux. Dans divers pays d'Amérique latine, les universités, les organismes d'état et de grandes fondations privées avaient de temps en temps pris l'initiative de soutenir la recherche en art et le recours aux nouveaux médias, mais la plupart avaient cessé leurs activités avant même de développer les ressources pour documenter les processus et préserver les résuItats. J'ai obtenu chaque enregistrement et information que j'ai rassemblés, depuis le milieu des années 1970, en contactant directement chacun des compositeurs. Avec le temps, j'ai constitué des archives personnelles, modestes mais croissantes, comprenant des notes de programme de concerts, livres, bulletins, magazines et revues, partitions, lettres, courriels et des enregistrements sur bobines, cassettes analogiques et quelques vinyles 33 tours. J'ai décidé de partager mes trésors avec des collègues et étudiants et d'explorer des solutions pour les rendre accessibles au plus grand nombre possible. Il y a quelques années, l'UNESCO m'a demandé de rédiger des rapports sur la musique électroacoustique latino-américaine et les arts médiatiques. Les textes de cette recherche ont contribué à diffuser de l'information sur le travail de beaucoup d'artistes latino-américains. Afin de rendre également accessible au public les oeuvres musicales, et sauvegarder le matériel, j'ai cherché un endroit où la préservation des enregistrements était non seulement importante mais aussi possible. J'estimais que la fondation Daniel Langlois pour l'art, la science et la technologie à Montréal était le lieu idéal pour mon projet. Mes activités continues durant près de 28 mois comme chercheur en résidence à la fondation Daniel Langlois m'ont permis de numériser et convertir des enregistrements à partir de différents formats, faire du montage au besoin et verser dans la base de données de la Fondation tous les renseignements sur les pièces (titre, compositeur, année de composition, instrumentation, notes de programme, studio de production, version, durée, bio du compositeur, etc.). À ce jour, janvier 2006, il y a 2152 fichiers audio numériques qui sont archivés au Centre de recherche et de documentation (CR+D) de la fondation. En complément à cette thèse de doctorat, j'ai développé une collection d'enregistrements musicaux maintenant disponibles au public. Cette collection est constituée du résultat de mes recherches (textes, oeuvres musicales, quelques partitions et photographies historiques, entrevues) et diffusée sur le site Web de la fondation Daniel Langlois. Les archives comptent des pièces pour médias fixes ainsi que des oeuvres mixtes pour instruments acoustiques ou voix et médias fixes ou systèmes électroniques interactifs en direct (1722 compositions). Les archives comprennent aussi des enregistrements audio et audiovisuels d'entrevues avec des compositeurs et des novateurs techniques ainsi que des\ud photographies, des vidéos, et quelques très rares partitions.\ud Une grande partie de l'information textuelle contenue dans la base de données des fichiers de musique est accessible par le site Web de la fondation Daniel Langlois. L'information complète (ex. notes de programme) et tous les enregistrements sont accessibles au CR+D. Une courte sélection de pièces est aussi accessible pour écoute sur le site Web. La plupart des compositeurs représentés dans ces archives et dans cette dissertation sont nés dans des pays d'Amérique latine. Il y a aussi quelques compositeurs qui, bien que n'étant pas originaires de la région, ont poursuivi au moins une partie de leur carrière musicale en Amérique latine.\ud Cette thèse renferme de l'information sur des compositeurs liés à 18 pays d'Amérique latine: Argentine, Bolivie, Brésil, Chili, Colombie, Costa Rica, Cuba, République dominicaine, Équateur, El Salvador, Guatemala, Mexico, Panama, Paraguay, Pérou, Porto Rico, Uruguay et Venezuela. Les archives contiennent des enregistrements de compositeurs de tous les pays mentionnés. J'espère que ce texte incitera à explorer ce merveilleux univers musical plutôt inconnu, créé par des centaines de compositeurs latino-américains au cours des dernières décennies. ______________________________________________________________________________ MOTS-CLÉS DE L’AUTEUR : musique électroacoustique, Amérique latine, art et nouvelles technologies, éthique, mémoire, culture, contexte, pionniers, interdisciplinarité

An investigation of audio signal-driven sound synthesis with a focus on its use for bowed stringed synthesisers

This thesis proposes an alternative approach to sound synthesis. It seeks to offer traditional string players a synthesiser which will allow them to make use of their existing skills in performance. A theoretical apparatus reflecting on the constraints of formalisation is developed and used to shed light on construction-related shortcomings in the instrumental developments of related research. Historical aspects and methods of sound synthesis, and the act of musical performance, are addressed with the aim of drawing conclusions for the construction of algorithms and interfaces. The alternative approach creates an openness and responsiveness in the synthesis instrument by using implicit playing parameters without the necessity to define, specify or measure all of them. In order to investigate this approach, several synthesis algorithms are developed, sounds are designed and a selection of them empirically compared to conventionally synthesised sounds. The algorithms are used in collaborative projects with other musicians in order to examine their practical musical value. The results provide evidence that implementations using the approach presented can offer musically significant differences as compared to similarly complex conventional implementations, and that - depending on the disposition of the musician - they can form a valuable contribution to the sound repertoire of performers and composers

Mobile sound: media art in hybrid spaces

The thesis explores the relationships between sound and mobility through an examination of sound art. The research engages with the intersection of sound, mobility and art through original empirical work and theoretically through a critical engagement with sound studies. In dialogue with the work of De Certeau, Lefebvre, Huhtamo and Habermas in terms of the poetics of walking, rhythms, media archeology and questions of publicness, I understand sound art as an experimental mobile and public space. The thesis establishes and situates the emerging field of mobile sound art by mapping three key traditions of mobile sound art - locative art, sound art and public art - and creates a taxonomy of mobile sound art by defining four categories: 'placing sounds', 'sound platforms', 'sonifying mobility' and 'musical instruments' (each represented by one case study). In doing so it develops a methodology that is attentive to the specifics of the sonic and mobile of media experience. I demonstrate how sonic interactions and embodied mobility are designed and experienced in specific ways in each of the four case studies - 'Aura' by Symons (UK), 'Pophorns' by Torstensson and Sandelin (Sweden), 'SmSage' by Redfern and Borland (US) and 'Core Sample' by Rueb (US) (all 2007). In tracing the topos of the musical telephone, discussing the making and breaking of relevant micro publics, accounting for the polyphonies of footsteps and unwrapping bundles of rhythms, this thesis contributes to understanding complex media experiences in hybrid spaces. In doing so it critically sheds light on the quality of sonic artistic experiences, the audience engagement with urban, public and networked spaces and the relationship between sound art and everyday media experience. My thesis provides valuable insight into auditory ways of mobilising and making public spaces, non-verbal and embodied media practices, and rhythms and scales of mobile media experiences

Concerto for Laptop Ensemble and Orchestra: The Ship of Theseus and Problems of Performance for Electronics With Orchestra: Taxonomy and Nomenclature

This dissertation is an examination of the problems faced when staging a work for electronics and orchestra. Part I is an original composition and model for the exploration of those problems. Part II is a monograph reviewing those problems and concentrating on issues of taxonomy and nomenclature. Part I is a concerto for laptop ensemble and orchestra titled The Ship of Theseus. It is named after a philosophical paradox. If every component of an object (i.e. the boards of a ship) is replaced with newer parts, at what point does the original cease to exist? Likewise, if the music performed by an instrument or ensemble is sampled and played back on stage, is it still an orchestra, or is it a recording? The role of the soloists is also explored throughout the work. Similarly to the dialogue of a Classical concerto, at times the soloist enhances the orchestra; at other times it clashes. Part II is an exploration of the etymology and nomenclature of electroacoustic music. In chapter 1, I explore broad problems and concerns specific to electronics and orchestra. In chapter 2, I break down the etymologies of both the orchestra and electroacoustic music, focusing on general issues surrounding the latter specifically. A new taxonomy for electroacoustic music is presented. In chapter 3, I investigate the nomenclature of three well-known terms: live electronic, real time, and interactive. Each of these terms is problematic and often misused; as a result the new term transformational is introduced and defined. This term should not be associated with the general idea of a musical transformation (although such an idea is not unwarranted), but with the flow of musical information in and out of a system. It is my hope that with the introduction of a new classification based on musical information, I will not merely pad the decades-long discourse on nomenclature of electroacoustic music, but rather provide a starting point for composers and technicians to reconcile technology with the music itself. The terms presented in this dissertation should not be considered definitive, but rather the inception of a new dialogue