Towards the automated analysis of simple polyphonic music : a knowledge-based approach

PhDMusic understanding is a process closely related to the knowledge and experience of the listener. The amount of knowledge required is relative to the complexity of the task in hand. This dissertation is concerned with the problem of automatically decomposing musical signals into a score-like representation. It proposes that, as with humans, an automatic system requires knowledge about the signal and its expected behaviour to correctly analyse music. The proposed system uses the blackboard architecture to combine the use of knowledge with data provided by the bottom-up processing of the signal's information. Methods are proposed for the estimation of pitches, onset times and durations of notes in simple polyphonic music. A method for onset detection is presented. It provides an alternative to conventional energy-based algorithms by using phase information. Statistical analysis is used to create a detection function that evaluates the expected behaviour of the signal regarding onsets. Two methods for multi-pitch estimation are introduced. The first concentrates on the grouping of harmonic information in the frequency-domain. Its performance and limitations emphasise the case for the use of high-level knowledge. This knowledge, in the form of the individual waveforms of a single instrument, is used in the second proposed approach. The method is based on a time-domain linear additive model and it presents an alternative to common frequency-domain approaches. Results are presented and discussed for all methods, showing that, if reliably generated, the use of knowledge can significantly improve the quality of the analysis.Joint Information Systems Committee (JISC) in the UK National Science Foundation (N.S.F.) in the United states. Fundacion Gran Mariscal Ayacucho in Venezuela

Upbeat and Quirky, With a Bit of a Build: Communicating Meaning and Meeting Information Needs in the Music Industry

Music is widely used to accompany moving images, in films, advertising, television programmes and computer games. The process of choosing and using a piece of pre-existing commercial music for this purpose is known as synchronisation. The addition of music to a piece of film enhances the final work with cultural meaning, and generates additional income for the rights holders. This research examines the information needs of professionals involved in the selection of music, including Users from the advertising and film communities and Owners from the recording and publishing industries. A tentative communications model is developed and proposed from musicological, semiotic and communications literature. Interviews, knowledge organisation systems, queries and observations are identified as rich potential sources of textual data relating to the communications process around satisfying the Users’ information needs. The content of these texts is analysed to identify key musical facets. Mood is found to be an important factor when searching for unknown musical items. Using a Discourse Analytic approach to the interview texts, four discourses, or interpretive repertoires, are identified. These repertoires carry conflicting meanings of music and are employed throughout the community, although relative emphases vary according to the viewpoint of the stakeholder. This is supported by an analysis of the written texts of both the Owners (music search engines) and the Users (written queries, or briefs). A comparison is drawn between the emphasis of the repertoires and the precision of the search engines. The repertoires are applied to the theoretical communications model, which is revised to reflect the findings of the analyses. This is used to make recommendations on how to improve the disintermediated communications process, by emphasising the repertoires employed by the Users rather than those of the Owners

A Content-Aware Interactive Explorer of Digital Music Collections: The Phonos Music Explorer

La tesi si propone di utilizzare le più recenti tecnologie del Music Information Retrieval (MIR) al fine di creare un esploratore interattivo di cataloghi musicali. Il software utilizza tecniche avanzate quali riduzione di dimensionalità  mediante FastMap, generazione e streaming over-the-network di contenuto audio, segmentazione e estrazione di descrittori da segnali audio. Inoltre, il software è in grado di adattare in real-time il proprio output sulla base di interazioni dell'utent

Staff-line removal with selectional auto-encoders

Staff-line removal is an important preprocessing stage as regards most Optical Music Recognition systems. The common procedures employed to carry out this task involve image processing techniques. In contrast to these traditional methods, which are based on hand-engineered transformations, the problem can also be approached from a machine learning point of view if representative examples of the task are provided. We propose doing this through the use of a new approach involving auto-encoders, which select the appropriate features of an input feature set (Selectional Auto-Encoders). Within the context of the problem at hand, the model is trained to select those pixels of a given image that belong to a musical symbol, thus removing the lines of the staves. Our results show that the proposed technique is quite competitive and significantly outperforms the other state-of-art strategies considered, particularly when dealing with grayscale input images.This work was partially supported by the Spanish Ministerio de Educación, Cultura y Deporte through a FPU fellowship (AP2012- 0939) and the Spanish Ministerio de Economía y Competitividad through Project TIMuL (No. TIN2013-48152-C2-1-R, supported by UE FEDER funds)

Proceedings of the 7th Sound and Music Computing Conference

Proceedings of the SMC2010 - 7th Sound and Music Computing Conference, July 21st - July 24th 2010

Digital Watermarking for Verification of Perception-based Integrity of Audio Data

In certain application fields digital audio recordings contain sensitive content. Examples are historical archival material in public archives that preserve our cultural heritage, or digital evidence in the context of law enforcement and civil proceedings. Because of the powerful capabilities of modern editing tools for multimedia such material is vulnerable to doctoring of the content and forgery of its origin with malicious intent. Also inadvertent data modification and mistaken origin can be caused by human error. Hence, the credibility and provenience in terms of an unadulterated and genuine state of such audio content and the confidence about its origin are critical factors. To address this issue, this PhD thesis proposes a mechanism for verifying the integrity and authenticity of digital sound recordings. It is designed and implemented to be insensitive to common post-processing operations of the audio data that influence the subjective acoustic perception only marginally (if at all). Examples of such operations include lossy compression that maintains a high sound quality of the audio media, or lossless format conversions. It is the objective to avoid de facto false alarms that would be expectedly observable in standard crypto-based authentication protocols in the presence of these legitimate post-processing. For achieving this, a feasible combination of the techniques of digital watermarking and audio-specific hashing is investigated. At first, a suitable secret-key dependent audio hashing algorithm is developed. It incorporates and enhances so-called audio fingerprinting technology from the state of the art in contentbased audio identification. The presented algorithm (denoted as ”rMAC” message authentication code) allows ”perception-based” verification of integrity. This means classifying integrity breaches as such not before they become audible. As another objective, this rMAC is embedded and stored silently inside the audio media by means of audio watermarking technology. This approach allows maintaining the authentication code across the above-mentioned admissible post-processing operations and making it available for integrity verification at a later date. For this, an existent secret-key ependent audio watermarking algorithm is used and enhanced in this thesis work. To some extent, the dependency of the rMAC and of the watermarking processing from a secret key also allows authenticating the origin of a protected audio. To elaborate on this security aspect, this work also estimates the brute-force efforts of an adversary attacking this combined rMAC-watermarking approach. The experimental results show that the proposed method provides a good distinction and classification performance of authentic versus doctored audio content. It also allows the temporal localization of audible data modification within a protected audio file. The experimental evaluation finally provides recommendations about technical configuration settings of the combined watermarking-hashing approach. Beyond the main topic of perception-based data integrity and data authenticity for audio, this PhD work provides new general findings in the fields of audio fingerprinting and digital watermarking. The main contributions of this PhD were published and presented mainly at conferences about multimedia security. These publications were cited by a number of other authors and hence had some impact on their works