Semantic Description of Timbral Transformations in Music Production

Abstract. In music production, descriptive terminology is used to define perceived sound transformations. By understanding the underlying statistical features associated with these descriptions, we can aid the retrieval of contextually relevant processing parameters using natural language, and create intelligent systems capable of assisting in audio engineering. In this study, we present an analysis of a dataset containing descriptive terms gathered using a series of processing modules, embedded within a Digital Audio Workstation. By applying hierarchical clustering to the audio feature space, we show that similarity in term representations exists within and between transformation classes. Furthermore, the organisation of terms in low-dimensional timbre space can be explained using perceptual concepts such as size and dissonance. We conclude by performing Latent Semantic Indexing to show that similar groupings exist based on term frequency

A Subsumption Agent for Collaborative Free Improvisation

This paper discusses the design and evaluation of an artificial agent for collaborative musical free improvisation. The agent provides a means to investigate the underpinnings of improvisational interaction. In connection with this general goal, the system is also used here to explore the implementation of a collaborative musical agent using a specific robotics architecture, Subsumption. The architecture of the system is explained, and its evaluation in an empirical study with expert improvisors is discussed. A follow-up study using a second iteration of the system is also presented. The system design and connected studies bring together Subsumption robotics, ecological psychology, and musical improvisation, and contribute to an empirical grounding of an ecological theory of improvisation

Generative rhythmic models

A system for generative rhythmic modeling is presented. The work aims to explore computational models of creativity, realizing them in a system designed for realtime generation of semi-improvisational music. This is envisioned as an attempt to develop musical intelligence in the context of structured improvisation, and by doing so to enable and encourage new forms of musical control and performance; the systems described in this work, already capable of realtime creation, have been designed with the explicit intention of embedding them in a variety of performance-based systems. A model of qaida, a solo tabla form, is presented, along with the results of an online survey comparing it to a professional tabla player's recording on dimensions of musicality, creativity, and novelty. The qaida model generates a bank of rhythmic variations by reordering subphrases. Selections from this bank are sequenced using a feature-based approach. An experimental extension into modeling layer- and loop-based forms of electronic music is presented, in which the initial modeling approach is generalized. Starting from a seed track, the layer-based model utilizes audio analysis techniques such as blind source separation and onset-based segmentation to generate layers which are shuffled and recombined to generate novel music in a manner analogous to the qaida model.M.S.Committee Chair: Chordia, Parag; Committee Member: Freeman, Jason; Committee Member: Weinberg, Gi

Ontology of music performance variation

Performance variation in rhythm determines the extent that humans perceive and feel the effect of rhythmic pulsation and music in general. In many cases, these rhythmic variations can be linked to percussive performance. Such percussive performance variations are often absent in current percussive rhythmic models. The purpose of this thesis is to present an interactive computer model, called the PD-103, that simulates the micro-variations in human percussive performance. This thesis makes three main contributions to existing knowledge: firstly, by formalising a new method for modelling percussive performance; secondly, by developing a new compositional software tool called the PD-103 that models human percussive performance, and finally, by creating a portfolio of different musical styles to demonstrate the capabilities of the software. A large database of recorded samples are classified into zones based upon the vibrational characteristics of the instruments, to model timbral variation in human percussive performance. The degree of timbral variation is governed by principles of biomechanics and human percussive performance. A fuzzy logic algorithm is applied to analyse current and first-order sample selection in order to formulate an ontological description of music performance variation. Asynchrony values were extracted from recorded performances of three different performance skill levels to create \timing fingerprints" which characterise unique features to each percussionist. The PD-103 uses real performance timing data to determine asynchrony values for each synthesised note. The spectral content of the sample database forms a three-dimensional loudness/timbre space, intersecting instrumental behaviour with music composition. The reparameterisation of the sample database, following the analysis of loudness, spectral flatness, and spectral centroid, provides an opportunity to explore the timbral variations inherent in percussion instruments, to creatively explore dimensions of timbre. The PD-103 was used to create a music portfolio exploring different rhythmic possibilities with a focus on meso-periodic rhythms common to parts of West Africa, jazz drumming, and electroacoustic music. The portfolio also includes new timbral percussive works based on spectral features and demonstrates the central aim of this thesis, which is the creation of a new compositional software tool that integrates human percussive performance and subsequently extends this model to different genres of music

Silent chill: A spectral analysis of Akira Yamaoka’s Silent Hill 2 original soundtrack

Silent Hill 2 (2001) is a psychological-survival horror game for the Sony Playstation 2 console, described as one of the greatest video games of all time. The game, as well as the original soundtrack by Akira Yamaoka, continue to have an active and dedicated cult following, with the soundtrack garnering millions of plays across streaming platforms. In particular, the ambient pieces in the soundtrack are very popular, colloquially described altogether as Silent Chill. Despite its popularity, few have systematically described the soundtrack’s unique characteristics or its ongoing influence and relevance to soundtrack composition today. It is suggested that timbral analysis can clarify some aspects of its mysterious appeal. A timbral-analytical framework adapting Lavengood’s (2017) spectrogram-based method and Blake’s (2012) culturally informed method is proposed and undertaken. The analyses find that the Silent Chill pieces are characterised by predominantly ‘dark’ timbres, inharmonicity, beating harmonics, and a spectral and auditory fullness due to overlapping and clashing frequencies between instruments. This framework, despite some limitations which are discussed, is found to be comprehensive and adequate for the timbral analysis of pieces in the style of Silent Chill, and can be adapted for other styles of soundtrack and ambient composition

Inferring rules from sound: The role of domain-specific knowledge in speech and music perception

Speech and music are two forms of complex auditory structure that play fundamental roles in everyday human experience and require certain basic perceptual and cognitive abilities. Nevertheless, when attempting to infer patterns from sequential auditory input, human listeners may use the same information differently depending on whether a sound is heard in a linguistic vs. musical context. The goal of these studies was to examine the role of domain-specific knowledge in auditory pattern perception. Specifically, the study examined the inference of rules in novel sound sequences that contained patterns of spectral structure (speech or instrument timbre) and fundamental frequency (pitch). Across all experiments, participants were first familiarized to a sequence containing pitch or syllable patterns that followed a particular rule (e.g., ABA), and they were subsequently asked to rate the similarity of novel sequences that were consistent or inconsistent with that rule. In two experiments participants were familiarized to either a pitch or syllable rule, and in a third experiment they were familiarized to simultaneous conflicting rules (e.g. pitch following ABA but syllables following ABB). Although participants readily detected inconsistent stimuli after familiarization to a single rule, in the conflicting two-rule condition they gave high similarity ratings to any stimulus that obeyed the syllable rule, regardless of whether or not the music dimension was consistent or inconsistent with familiarization. Three additional experiments took the same approach but tested rule-learning on the basis of pitch or timbre (instrument) sequences. In these experiments, participants gave the highest similarity ratings when the pitch rule was preserved, regardless of variation in the timbre dimension. These results support the notion that adults filter information according to domain-specific knowledge and expectations, presumably due to perceptual learning processes that take place during early development

Automatic Music Transcription with Convolutional Neural Networks using Intuitive Filter Shapes

This thesis explores the challenge of automatic music transcription with a combination of digital signal processing and machine learning methods. Automatic music transcription is important for musicians who can\u27t do it themselves or find it tedious. We start with an existing model, designed by Sigtia, Benetos and Dixon, and develop it in a number of original ways. We find that by using convolutional neural networks with filter shapes more tailored for spectrogram data, we see better and faster transcription results when evaluating the new model on a dataset of classical piano music. We also find that employing better practices shows improved results. Finally, we open-source our test bed for pre-processing, training, and testing the models to assist in future research