Notation Sequence Generation and Sound Synthesis in Interactive Spectral Music

Notation sequence generation and sound synthesis in interactive spectral music This thesis consists of a preliminary analysis of existing spectral music paradigms and proposes a methodology to address issues that arise in real-time spectral music composition and performance scenarios. This exploration involves an overview of meaning in spectral music with a particular focus on the ‘sonic object’ as a vehicle for expression. A framework for the production of ‘interactive spectral music’ was created. This framework takes form as a group of software based compositional tools called SpectraScore developed for the Max for Live platform. Primarily, these tools allow the user to analyse incoming audio and directly apply the collected data towards the generation of synthesised sound and notation sequences. Also presented is an extension of these tools, a novel system of correlation between emotional descriptors and spectrally derived harmonic morphemes. The final component is a portfolio of works created as examples of the techniques explored in scored and recorded form. As a companion to these works, an analysis component outlines the programmatic aspects of each piece and illustrates how they are executed within the music. Each scored piece corresponds with a recording of a live performance or performances of the work included in the attached DVD, which comprises individual realisations of the interactive works. Keywords: Spectralism, Music and Emotion, Electronic Music, Spectral Music, Algorithmic Music, Real-time Notatio

Notation Sequence Generation and Sound Synthesis in Interactive Spectral Music

Notation sequence generation and sound synthesis in interactive spectral music This thesis consists of a preliminary analysis of existing spectral music paradigms and proposes a methodology to address issues that arise in real-time spectral music composition and performance scenarios. This exploration involves an overview of meaning in spectral music with a particular focus on the ‘sonic object’ as a vehicle for expression. A framework for the production of ‘interactive spectral music’ was created. This framework takes form as a group of software based compositional tools called SpectraScore developed for the Max for Live platform. Primarily, these tools allow the user to analyse incoming audio and directly apply the collected data towards the generation of synthesised sound and notation sequences. Also presented is an extension of these tools, a novel system of correlation between emotional descriptors and spectrally derived harmonic morphemes. The final component is a portfolio of works created as examples of the techniques explored in scored and recorded form. As a companion to these works, an analysis component outlines the programmatic aspects of each piece and illustrates how they are executed within the music. Each scored piece corresponds with a recording of a live performance or performances of the work included in the attached DVD, which comprises individual realisations of the interactive works. Keywords: Spectralism, Music and Emotion, Electronic Music, Spectral Music, Algorithmic Music, Real-time Notatio

Perceptual Constancy

We perceive objects and events in a way that makes it possible to act, react, think, believe, etc. in reliable and predictable ways. To explain this perceptual stability, as well as its behavioral consequences, theorists invoke a set of capacities known as perceptual constancies. Thanks to constancies, perceivers latch onto what’s unchanging in the world even though sensory stimulation is in continuous flux. In this dissertation, I present and defend a new view of both perceptual constancy and perceptual objectivity, i.e. the capacity of perception to present the world as mind-independent. According to the traditional view, perceptual constancy is the capacity of perceptual systems to recover perceiver-independent properties of distal objects from a largely ambiguous proximal stimulus, ‘discounting’ contextual, perceiver-dependent information. I argue that the traditional view should be rejected because it is, on the one hand, too ‘visuo-centric’, and, on the other hand, unable to fully explain the roles that constancy plays in our lives. These roles include guiding action and enabling the stable conscious experiences that ground our perceptual judgments. The view I favor, which I call “Relational Invariance view”, holds that constancy is the capacity to track invariant relations within the perceptual scene or between some element in the scene and the perceiver. These invariant relations are specified by patterns of variation in the proximal stimulus over time, and perceivers can sometime directly control this variation through movement. This view explains the role that, intuitively, perceptual constancy plays in guiding motor action and in a wide variety of perceptual recognition tasks, where recovering perceiver-independent properties seems unnecessary. The Relational Invariance view is then tied to a new view of perceptual objectivity, whose core insight is that the ‘job’ of perception in enabling the experience of a mind-independent world is not to ‘abstract away’ from any sort of perspectival or contextual influence, but rather to ‘embrace’ these influences as intrinsic to the very idea of what it means to perceive the world for creatures like us

The Semantics of Timbre

Because humans lack a sensory vocabulary for auditory experiences, timbral qualities of sounds are often conceptualized and communicated through readily available sensory attributes from different modalities (e.g., bright, warm, sweet) but also through the use of onomatopoeic attributes (e.g., ringing, buzzing, shrill) or nonsensory attributes relating to abstract constructs (e.g., rich, complex, harsh). The analysis of the linguistic description of timbre, or timbre semantics, can be considered as one way to study its perceptual representation empirically. In the most commonly adopted approach, timbre is considered as a set of verbally defined perceptual attributes that represent the dimensions of a semantic timbre space. Previous studies have identified three salient semantic dimensions for timbre along with related acoustic properties. Comparisons with similarity-based multidimensional models confirm the strong link between perceiving timbre and talking about it. Still, the cognitive and neural mechanisms of timbre semantics remain largely unknown and underexplored, especially when one looks beyond the case of acoustic musical instruments

Listening-Mode-Centered Sonification Design for Data Exploration

Grond F. Listening-Mode-Centered Sonification Design for Data Exploration. Bielefeld: Bielefeld University; 2013.From the Introduction to this thesis: Through the ever growing amount of data and the desire to make them accessible to the user through the sense of listening, sonification, the representation of data by using sound has been subject of active research in the computer sciences and the field of HCI for the last 20 years. During this time, the field of sonification has diversified into different application areas: today, sound in auditory display informs the user about states and actions on the desktop and in mobile devices; sonification has been applied in monitoring applications, where sound can range from being informative to alarming; sonification has been used to give sensory feedback in order to close the action and perception loop; last but not least, sonifications have also been developed for exploratory data analysis, where sound is used to represent data with unknown structures for hypothesis building. Coming from the computer sciences and HCI, the conceptualization of sonification has been mostly driven by application areas. On the other hand, the sonic arts who have always contributed to the community of auditory display have a genuine focus on sound. Despite this close interdisciplinary relation of communities of sound practitioners, a rich and sound- (or listening)-centered concept about sonification is still missing as a point of departure for a more application and task overarching approach towards design guidelines. Complementary to the useful organization along fields of applications, a conceptual framework that is proper to sound needs to abstract from applications and also to some degree from tasks, as both are not directly related to sound. I hence propose in this thesis to conceptualize sonifications along two poles where sound serves either a normative or a descriptive purpose. In the beginning of auditory display research, a continuum between a symbolic and an analogic pole has been proposed by Kramer (1994a, page 21). In this continuum, symbolic stands for sounds that coincide with existing schemas and are more denotative, analogic stands for sounds that are informative through their connotative aspects. (compare Worrall (2009, page 315)). The notions of symbolic and analogic illustrate the struggle to find apt descriptions of how the intention of the listener subjects audible phenomena to a process of meaning making and interpretation. Complementing the analogic-symbolic continuum with descriptive and normative purposes of displays is proposed in the light of the recently increased research interest in listening modes and intentions. Similar to the terms symbolic and analogic, listening modes have been discussed in auditory display since the beginning usually in dichotomic terms which were either identified with the words listening and hearing or understood as musical listening and everyday listening as proposed by Gaver (1993a). More than 25 years earlier, four direct listening modes have been introduced by Schaeffer (1966) together with a 5th synthetic mode of reduced listening which leads to the well-known sound object. Interestingly, Schaeffer’s listening modes remained largely unnoticed by the auditory display community. Particularly the notion of reduced listening goes beyond the connotative and denotative poles of the continuum proposed by Kramer and justifies the new terms descriptive and normative. Recently, a new taxonomy of listening modes has been proposed by Tuuri and Eerola (2012) that is motivated through an embodied cognition approach. The main contribution of their taxonomy is that it convincingly diversifies the connotative and denotative aspects of listening modes. In the recently published sonification handbook, multimodal and interactive aspects in combination with sonification have been discussed as promising options to expand and advance the field by Hunt and Hermann (2011), who point out that there is a big need for a better theoretical foundation in order to systematically integrate these aspects. The main contribution of this thesis is to address this need by providing alternative and complementary design guidelines with respect to existing approaches, all of which have been conceived before the recently increased research interest in listening modes. None of the existing contributions to design frameworks integrates multimodality, and listening modes with a focus on exploratory data analysis, where sonification is conceived to support the understanding of complex data potentially helping to identify new structures therein. In order to structure this field the following questions are addressed in this thesis: • How do natural listening modes and reduced listening relate to the proposed normative and descriptive display purposes? • What is the relationship of multimodality and interaction with listening modes and display purposes? • How can the potential of embodied cognition based listening modes be put to use for exploratory data sonification? • How can listening modes and display purposes be connected to questions of aesthetics in the display? • How do data complexity and Parameter-mapping sonification relate to exploratory data analysis and listening modes

Towards music-driven procedural animation

We present our approach towards the development of a framework for the creation of music-driven procedural animations. We intend to explore the potential that elementary musical features hold for driving engaging audio-visual animations. To do so, we bring forward an integrated environment where real-time musical information is available and may be flexibly used for manipulating different aspects of a dynamic animation. In general terms, our approach consists of developing a virtual scene, populated by controllable entities, termed actors, and using scripting to define how these actors' behaviour or appearance change in response to musical information. Scripting operates by establishing associations, or mappings, between musical events, such as the ringing of notes or chords, or sound information, such as the frequency spectrum, and changes in the animation. The scenario we chose to explore is comprised of two main actors: trees and wind. Trees grow in an iterative process, and may develop leaves, while swaying in response to the wind field. The wind is represented as a vector field whose configuration and strength can be altered in real-time. Scripting then allows for synchronising these changes with musical events, providing a natural sense of harmony with the accompanying music. By having real-time access to musical information, as well as control over a reactive animation we believe to have taken a first step towards exploring a novel interdisciplinary concept with vast expressive potential.This work has been supported by national funds through FCT – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2019

Idealized computational models for auditory receptive fields

This paper presents a theory by which idealized models of auditory receptive fields can be derived in a principled axiomatic manner, from a set of structural properties to enable invariance of receptive field responses under natural sound transformations and ensure internal consistency between spectro-temporal receptive fields at different temporal and spectral scales. For defining a time-frequency transformation of a purely temporal sound signal, it is shown that the framework allows for a new way of deriving the Gabor and Gammatone filters as well as a novel family of generalized Gammatone filters, with additional degrees of freedom to obtain different trade-offs between the spectral selectivity and the temporal delay of time-causal temporal window functions. When applied to the definition of a second-layer of receptive fields from a spectrogram, it is shown that the framework leads to two canonical families of spectro-temporal receptive fields, in terms of spectro-temporal derivatives of either spectro-temporal Gaussian kernels for non-causal time or the combination of a time-causal generalized Gammatone filter over the temporal domain and a Gaussian filter over the logspectral domain. For each filter family, the spectro-temporal receptive fields can be either separable over the time-frequency domain or be adapted to local glissando transformations that represent variations in logarithmic frequencies over time. Within each domain of either non-causal or time-causal time, these receptive field families are derived by uniqueness from the assumptions. It is demonstrated how the presented framework allows for computation of basic auditory features for audio processing and that it leads to predictions about auditory receptive fields with good qualitative similarity to biological receptive fields measured in the inferior colliculus (ICC) and primary auditory cortex (A1) of mammals.Comment: 55 pages, 22 figures, 3 table

Objectifying the Subjective: Fundaments and Applications of Soft Metrology

The aim of the interdisciplinary research was to facilitate the understanding of a specific topic passing by different disciplinary perspectives. Soft metrology is the perfect example of a scientific field that needs that sort of approach. Seeking to provide a reproducible basis for qualifying and quantifying what are essentially ‘soft’ measurements (subject to human perception and interpretation) is a particularly challenging scientific endeavour. This chapter presents a theoretical overview of main concepts around soft metrology and, in the second instance, proposes a mathematical model for the measurement of a soft measurand through a dedicated index (IPER—influence on performance index)