PnP Maxtools: Autonomous Parameter Control in MaxMSP Utilizing MIR Algorithms

This research presents a new approach to computer automation through the implementation of novel real-time music information retrieval algorithms developed for this project. It documents the development of the PnP.Maxtools package, a set of open source objects designed within the popular programming environment MaxMSP. The package is a set of pre/post processing filters, objective and subjective timbral descriptors, audio effects, and other objects that are designed to be used together to compose music or improvise without the use of external controllers or hardware. The PnP.Maxtools package objects are designed to be used quickly and easily using a `plug and play\u27 style with as few initial arguments needed as possible. The PnP.Maxtools package is designed to take incoming audio from a microphone, analyze it, and use the analysis to control an audio effect on the incoming signal in real-time. In this way, the audio content has a real musical and analogous relationship with the resulting musical transformations while the control parameters become more multifaceted and better able to serve the needs of artists. The term Reflexive Automation is presented that describes this unsupervised relationship between the content of the sound being analyzed and the analogous and automatic control over a specific musical parameter. A set of compositions are also presented that demonstrate ideal usage of the object categories for creating reflexive systems and achieving fully autonomous control over musical parameters

Timbra: An online tool for feature extraction, comparative analysis and visualization of timbre

Dariah.lab is a research infrastructure created for digital humanities, consisting of state-of-the-art hardware and dedicated software tools. One of the tools developed for digital musicology is Timbra, a web-based application for conducting research on sound timbre. The aim was to create an easy-touse online tool for non-programmers. The tool can be used to calculate, visualise, and compare different timbre characteristics of uploaded audio files and to export the extracted parameters in CSV format for further processing, e.g. by classification tools. The application offers extraction and visualisation of scalar features such as zero crossing rate, fundamental frequency, spectralcentroid, spectral roll-off, spectral flatness, band energy ratio, as well as feature vectors (e.g. chromagram, spectral contrast, spectrogram, and MFCCs). An interested user can compare selected sound characteristics using various types of plots and run dissimilarity analysis of timbre parameters by means of 2D or 3D multidimensional scaling (MDS). The paper showcases potentialapplications of the tool based on presented case studies. In terms of implementation, the calculations are performed at the backend Django server using Librosa and standard Python libraries. Dash library is used for the frontend. By offering an easy-to-use tool accessible anytime and anywhere through the Internet, we want to facilitate timbre analysis for a broader group of researchers, e.g. sound engineers, luthiers, phoneticians, or musicologists.

Large scale musical instrument identification

In this paper, automatic musical instrument identification using a variety of classifiers is addressed. Experiments are performed on a large set of recordings that stem from 20 instrument classes. Several features from general audio data classification applications as well as MPEG-7 descriptors are measured for 1000 recordings. Branch-and-bound feature selection is applied in order to select the most discriminating features for instrument classification. The first classifier is based on non-negative matrix factorization (NMF) techniques, where training is performed for each audio class individually. A novel NMF testing method is proposed, where each recording is projected onto several training matrices, which have been Gram-Schmidt orthogonalized. Several NMF variants are utilized besides the standard NMF method, such as the local NMF and the sparse NMF. In addition, 3-layered multilayer perceptrons, normalized Gaussian radial basis function networks, and support vector machines employing a polynomial kernel have also been tested as classifiers. The classification accuracy is high, ranging between 88.7% to 95.3%, outperforming the state-of-the-art techniques tested in the aforementioned experiment

Auditory Spectrum-Based Pitched Instrument Onset Detection

In this paper, a method for onset detection of music signals using auditory spectra is proposed. The auditory spectrogram provides a time-frequency representation that employs a sound processing model resembling the human auditory system. Recent work on onset detection employs DFT-based features describing spectral energy and phase differences, as well as pitch-based features. These features are often combined for maximizing detection performance. Here, the spectral flux and phase slope features are derived in the auditory framework and a novel fundamental frequency estimation algorithm based on auditory spectra is introduced. An onset detection algorithm is proposed, which processes and combines the aforementioned features at the decision level. Experiments are conducted on a dataset covering 11 pitched instrument types, consisting of 1829 onsets in total. Results indicate that auditory representations outperform various state-of-the-art approaches, with the onset detection algorithm reaching an F-measure of 82.6%