Automatic annotation of musical audio for interactive applications

PhDAs machines become more and more portable, and part of our everyday life, it becomes apparent that developing interactive and ubiquitous systems is an important aspect of new music applications created by the research community. We are interested in developing a robust layer for the automatic annotation of audio signals, to be used in various applications, from music search engines to interactive installations, and in various contexts, from embedded devices to audio content servers. We propose adaptations of existing signal processing techniques to a real time context. Amongst these annotation techniques, we concentrate on low and mid-level tasks such as onset detection, pitch tracking, tempo extraction and note modelling. We present a framework to extract these annotations and evaluate the performances of different algorithms. The first task is to detect onsets and offsets in audio streams within short latencies. The segmentation of audio streams into temporal objects enables various manipulation and analysis of metrical structure. Evaluation of different algorithms and their adaptation to real time are described. We then tackle the problem of fundamental frequency estimation, again trying to reduce both the delay and the computational cost. Different algorithms are implemented for real time and experimented on monophonic recordings and complex signals. Spectral analysis can be used to label the temporal segments; the estimation of higher level descriptions is approached. Techniques for modelling of note objects and localisation of beats are implemented and discussed. Applications of our framework include live and interactive music installations, and more generally tools for the composers and sound engineers. Speed optimisations may bring a significant improvement to various automated tasks, such as automatic classification and recommendation systems. We describe the design of our software solution, for our research purposes and in view of its integration within other systems.EU-FP6-IST-507142 project SIMAC (Semantic Interaction with Music Audio Contents); EPSRC grants GR/R54620; GR/S75802/01

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc

Measuring Expressive Music Performances: a Performance Science Model using Symbolic Approximation

Music Performance Science (MPS), sometimes termed systematic musicology in Northern Europe, is concerned with designing, testing and applying quantitative measurements to music performances. It has applications in art musics, jazz and other genres. It is least concerned with aesthetic judgements or with ontological considerations of artworks that stand alone from their instantiations in performances. Musicians deliver expressive performances by manipulating multiple, simultaneous variables including, but not limited to: tempo, acceleration and deceleration, dynamics, rates of change of dynamic levels, intonation and articulation. There are significant complexities when handling multivariate music datasets of significant scale. A critical issue in analyzing any types of large datasets is the likelihood of detecting meaningless relationships the more dimensions are included. One possible choice is to create algorithms that address both volume and complexity. Another, and the approach chosen here, is to apply techniques that reduce both the dimensionality and numerosity of the music datasets while assuring the statistical significance of results. This dissertation describes a flexible computational model, based on symbolic approximation of timeseries, that can extract time-related characteristics of music performances to generate performance fingerprints (dissimilarities from an ‘average performance’) to be used for comparative purposes. The model is applied to recordings of Arnold Schoenberg’s Phantasy for Violin with Piano Accompaniment, Opus 47 (1949), having initially been validated on Chopin Mazurkas.1 The results are subsequently used to test hypotheses about evolution in performance styles of the Phantasy since its composition. It is hoped that further research will examine other works and types of music in order to improve this model and make it useful to other music researchers. In addition to its benefits for performance analysis, it is suggested that the model has clear applications at least in music fraud detection, Music Information Retrieval (MIR) and in pedagogical applications for music education

Musical Haptics

Haptic Musical Instruments; Haptic Psychophysics; Interface Design and Evaluation; User Experience; Musical Performanc

Reflexion

Die Musik des 20. und bisherigen 21. Jahrhunderts ist geprägt von einer nie dagewesenen Komplexität kompositorischer Ansätze, die mit dem Aufbrechen der herkömmlichen Dur-Moll-Tonalität begann und mit den multimedialen Möglichkeiten des neuen Jahrhunderts noch lange nicht zu Ende ist. Auch die vor allem nach 1950 eingetretene fundamentale Wandlung ästhetischer Positionen aufgrund gesellschaftlicher Umbrüche hat ihre Spuren in der Musik hinterlassen. Der Begriff der Musik entwickelte sich im Laufe des 20. Jahrhunderts zu einem vielschichtigen Topos pluraler Musiken, basierend u.a. auf neuen kompositorischen Mitteln, interkulturellen Begegnungen, unterschiedlichen medialen Zugängen und nicht zuletzt einem Wandel ästhetischer Rezeptionsmechanismen. Im vorliegenden Band werden Referate zu diesem Themenkomplex aus der Rubrik „Freie Referate“ des 15. Internationalen Kongresses der Gesellschaft für Musikforschung, der vom 4.–8. September 2012 in Göttingen unter dem Motto „Musik | Musiken. Strukturen und Prozesse“ stattfand, zusammengefasst. Das Panorama reicht dabei von Fragen nach einer kompositorischen Spätstilistik, kulturellen Austauschprozessen, musikalischen Grenzerfahrungen und intermedialen Exotismen bis hin zu multimedialen Klanginstallationen und Musiktheaterkompositionen und dem damit verbundenen Aufbrechen tradierter Gattungsgefüge.Die Musik des 20. und bisherigen 21. Jahrhunderts ist geprägt von einer nie dagewesenen Komplexität kompositorischer Ansätze, die mit dem Aufbrechen der herkömmlichen Dur-Moll-Tonalität begann und mit den multimedialen Möglichkeiten des neuen Jahrhunderts noch lange nicht zu Ende ist. Auch die vor allem nach 1950 eingetretene fundamentale Wandlung ästhetischer Positionen aufgrund gesellschaftlicher Umbrüche hat ihre Spuren in der Musik hinterlassen. Der Begriff der Musik entwickelte sich im Laufe des 20. Jahrhunderts zu einem vielschichtigen Topos pluraler Musiken, basierend u.a. auf neuen kompositorischen Mitteln, interkulturellen Begegnungen, unterschiedlichen medialen Zugängen und nicht zuletzt einem Wandel ästhetischer Rezeptionsmechanismen. Im vorliegenden Band werden Referate zu diesem Themenkomplex aus der Rubrik „Freie Referate“ des 15. Internationalen Kongresses der Gesellschaft für Musikforschung, der vom 4.–8. September 2012 in Göttingen unter dem Motto „Musik | Musiken. Strukturen und Prozesse“ stattfand, zusammengefasst. Das Panorama reicht dabei von Fragen nach einer kompositorischen Spätstilistik, kulturellen Austauschprozessen, musikalischen Grenzerfahrungen und intermedialen Exotismen bis hin zu multimedialen Klanginstallationen und Musiktheaterkompositionen und dem damit verbundenen Aufbrechen tradierter Gattungsgefüge

Musical Haptics

This Open Access book offers an original interdisciplinary overview of the role of haptic feedback in musical interaction. Divided into two parts, part I examines the tactile aspects of music performance and perception, discussing how they affect user experience and performance in terms of usability, functionality and perceived quality of musical instruments. Part II presents engineering, computational, and design approaches and guidelines that have been applied to render and exploit haptic feedback in digital musical interfaces. Musical Haptics introduces an emerging field that brings together engineering, human-computer interaction, applied psychology, musical aesthetics, and music performance. The latter, defined as the complex system of sensory-motor interactions between musicians and their instruments, presents a well-defined framework in which to study basic psychophysical, perceptual, and biomechanical aspects of touch, all of which will inform the design of haptic musical interfaces. Tactile and proprioceptive cues enable embodied interaction and inform sophisticated control strategies that allow skilled musicians to achieve high performance and expressivity. The use of haptic feedback in digital musical interfaces is expected to enhance user experience and performance, improve accessibility for disabled persons, and provide an effective means for musical tuition and guidance

Proceedings of the 7th Sound and Music Computing Conference

Proceedings of the SMC2010 - 7th Sound and Music Computing Conference, July 21st - July 24th 2010

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

Interactive Tango Milonga: An Interactive Dance System for Argentine Tango Social Dance

abstract: When dancers are granted agency over music, as in interactive dance systems, the actors are most often concerned with the problem of creating a staged performance for an audience. However, as is reflected by the above quote, the practice of Argentine tango social dance is most concerned with participants internal experience and their relationship to the broader tango community. In this dissertation I explore creative approaches to enrich the sense of connection, that is, the experience of oneness with a partner and complete immersion in music and dance for Argentine tango dancers by providing agency over musical activities through the use of interactive technology. Specifically, I create an interactive dance system that allows tango dancers to affect and create music via their movements in the context of social dance. The motivations for this work are multifold: 1) to intensify embodied experience of the interplay between dance and music, individual and partner, couple and community, 2) to create shared experience of the conventions of tango dance, and 3) to innovate Argentine tango social dance practice for the purposes of education and increasing musicality in dancers.Dissertation/ThesisDoctoral Dissertation Music 201