45 research outputs found
Automatic annotation of musical audio for interactive applications
PhDAs machines become more and more portable, and part of our everyday life, it becomes
apparent that developing interactive and ubiquitous systems is an important
aspect of new music applications created by the research community. We are interested
in developing a robust layer for the automatic annotation of audio signals, to
be used in various applications, from music search engines to interactive installations,
and in various contexts, from embedded devices to audio content servers. We
propose adaptations of existing signal processing techniques to a real time context.
Amongst these annotation techniques, we concentrate on low and mid-level tasks
such as onset detection, pitch tracking, tempo extraction and note modelling. We
present a framework to extract these annotations and evaluate the performances of
different algorithms.
The first task is to detect onsets and offsets in audio streams within short latencies.
The segmentation of audio streams into temporal objects enables various
manipulation and analysis of metrical structure. Evaluation of different algorithms
and their adaptation to real time are described. We then tackle the problem of
fundamental frequency estimation, again trying to reduce both the delay and the
computational cost. Different algorithms are implemented for real time and experimented
on monophonic recordings and complex signals. Spectral analysis can be
used to label the temporal segments; the estimation of higher level descriptions is
approached. Techniques for modelling of note objects and localisation of beats are
implemented and discussed.
Applications of our framework include live and interactive music installations,
and more generally tools for the composers and sound engineers. Speed optimisations
may bring a significant improvement to various automated tasks, such as
automatic classification and recommendation systems. We describe the design of
our software solution, for our research purposes and in view of its integration within
other systems.EU-FP6-IST-507142 project SIMAC (Semantic Interaction with Music
Audio Contents);
EPSRC grants GR/R54620; GR/S75802/01
Design Strategies for Adaptive Social Composition: Collaborative Sound Environments
In order to develop successful collaborative music systems a variety
of subtle interactions need to be identified and integrated. Gesture
capture, motion tracking, real-time synthesis, environmental
parameters and ubiquitous technologies can each be effectively used
for developing innovative approaches to instrument design, sound
installations, interactive music and generative systems. Current
solutions tend to prioritise one or more of these approaches, refining
a particular interface technology, software design or compositional
approach developed for a specific composition, performer or
installation environment. Within this diverse field a group of novel
controllers, described as âTangible Interfacesâ have been developed.
These are intended for use by novices and in many cases follow a
simple model of interaction controlling synthesis parameters through
simple user actions. Other approaches offer sophisticated
compositional frameworks, but many of these are idiosyncratic and
highly personalised. As such they are difficult to engage with and
ineffective for groups of novices. The objective of this research is to
develop effective design strategies for implementing collaborative
sound environments using key terms and vocabulary drawn from the
available literature. This is articulated by combining an empathic
design process with controlled sound perception and interaction
experiments. The identified design strategies have been applied to
the development of a new collaborative digital instrument. A range
of technical and compositional approaches was considered to define
this process, which can be described as Adaptive Social Composition.
Dan Livingston
Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings
Structure perception is a fundamental aspect of music cognition in humans.
Historically, the hierarchical organization of music into structures served as
a narrative device for conveying meaning, creating expectancy, and evoking
emotions in the listener. Thereby, musical structures play an essential role in
music composition, as they shape the musical discourse through which the
composer organises his ideas. In this paper, we present a novel music
segmentation method, pitchclass2vec, based on symbolic chord annotations, which
are embedded into continuous vector representations using both natural language
processing techniques and custom-made encodings. Our algorithm is based on
long-short term memory (LSTM) neural network and outperforms the
state-of-the-art techniques based on symbolic chord annotations in the field
DEEP-RHYTHM FOR TEMPO ESTIMATION AND RHYTHM PATTERN RECOGNITION
International audienceIt has been shown that the harmonic series at the tempo frequency of the onset-strength-function of an audio signal accurately describes its rhythm pattern and can be used to perform tempo or rhythm pattern estimation. Recently, in the case of multi-pitch estimation, the depth of the input layer of a convolutional network has been used to represent the harmonic series of pitch candidates. We use a similar idea here to represent the harmonic series of tempo candidates. We propose the Harmonic-Constant-Q-Modulation which represents, using a 4D-tensors, the harmonic series of modulation frequencies (considered as tempo frequencies) in several acoustic frequency bands over time. This representation is used as input to a convolutional network which is trained to estimate tempo or rhythm pattern classes. Using a large number of datasets, we evaluate the performance of our approach and compare it with previous approaches. We show that it slightly increases Accuracy-1 for tempo estimation but not the average-mean-Recall for rhythm pattern recognition
Defragmenting Beethoven: Sound appropriation as bridge between classical tradition and electroacoustic music
This thesis serves as a written companion for two artistic-based research products built upon the concept of appropriation as connecting bridge between music technology and the classical tradition of music composition. The first artistic work is a set of 9 pieces called "Collages Vol 2", a continuation of a first release meant to be an exploratory work for the present research. The second work is the sound interaction design, and the creation of two compositions for the Network of Intelligent Sound Agents, or "NOISA", built at the Sound and Physical Interaction Research Group from the Department of Media, Aalto University. After providing context and a short survey of influences on music appropriation, I made a comprehensive documentation of each of the pieces created for this thesis, describing form, content, compositional approach and sound processing in a systematic way.
I investigated on the diverse forms of appropriation as a technique for electroacoustic music composition. The most influential references for my work are documented in this written work: From the historical approach of appropriation to borrowing in music of the XX century and recent times; including a description of the first volume of my original Collages. Later on, I described my second collection of Collages and the utilisation of appropriation theories in the context of NOISA, a music interface for live performance. Finally, there is a section dedicated to a discussion featuring a commentary of a number of reviews of "Collagesâ preceding a closing segment with conclusions and further plans to expand this research in the future
Recommended from our members
Este Futuro es Otro Futuro: The role of social discourse on the [under]development of contemporary academic electronic music in PerĂş
This dissertation explores the history of Peruvian electroacoustic and electronic musical experimentation since its arrival to the country, while confronting the particular issues that have kept their practice from being academically embraced or implemented into the official channels for musical learning. It reviews the failures and successes of the Peruvian technologically based musical arts in front of the social structures and ideologies that have historically permeated all the cultural activities of the country. By moving away from a traditional technical and formalist approach that confronts academically based electroacoustic and electronic music from the perspective of the musical object, this work unveils the highly politicized history of responses towards these arts in PerĂş and the direct role of the particular nationalistic postcolonial models on their development or lack of it. It is within a plural and ambiguous set of nationalists, nativist, socialist, anti-foreign and indigenist ideologies, analyzed in this work, that we can identify a constant: the suspicion towards, and ultimately, the rejection of technologically based musical traditions as an element in the construction of a history of Peruvian music. Most actors in this work, whether composers, performers or music researchers, that have attempted to dedicate their efforts towards the implementation of technology beyond the logic of the recording studio and tech-support, have had to negotiate their participation in musical academic and popular cultures according to the way they were perceived by the political and cultural surroundings of their time. This situation would force many of them to construct the cultural identities that could allow them to participate in national institutionalized discourses and practices, moving away from technology or maintaining it as a peripheral activity. In the majority of cases, those musicians interested in technology would either migrate in search of training and professional opportunities, or shift direction towards other musical practices. By analyzing, in this work, the national discourses of/about our actors and the silence of the institutions (public and private) we reveal an agent-structure relational behavioral pattern toward musical technology and related sound arts that is at the heart of its historical undermining
Measuring Expressive Music Performances: a Performance Science Model using Symbolic Approximation
Music Performance Science (MPS), sometimes termed systematic musicology in Northern Europe, is concerned with designing, testing and applying quantitative measurements to music performances. It has applications in art musics, jazz and other genres. It is least concerned with aesthetic judgements or with ontological considerations of artworks that stand alone from their instantiations in performances. Musicians deliver expressive performances by manipulating multiple, simultaneous variables including, but not limited to: tempo, acceleration and deceleration, dynamics, rates of change of dynamic levels, intonation and articulation. There are significant complexities when handling multivariate music datasets of significant scale. A critical issue in analyzing any types of large datasets is the likelihood of detecting meaningless relationships the more dimensions are included. One possible choice is to create algorithms that address both volume and complexity. Another, and the approach chosen here, is to apply techniques that reduce both the dimensionality and numerosity of the music datasets while assuring the statistical significance of results. This dissertation describes a flexible computational model, based on symbolic approximation of timeseries, that can extract time-related characteristics of music performances to generate performance fingerprints (dissimilarities from an âaverage performanceâ) to be used for comparative purposes. The model is applied to recordings of Arnold Schoenbergâs Phantasy for Violin with Piano Accompaniment, Opus 47 (1949), having initially been validated on Chopin Mazurkas.1 The results are subsequently used to test hypotheses about evolution in performance styles of the Phantasy since its composition. It is hoped that further research will examine other works and types of music in order to improve this model and make it useful to other music researchers. In addition to its benefits for performance analysis, it is suggested that the model has clear applications at least in music fraud detection, Music Information Retrieval (MIR) and in pedagogical applications for music education
X Reality Networked Performance: Message Based Distributed Systems For Controlling And Presenting Multiple Realities
X reality networked performances connect physical, fictional and computer generated
realities in a new world of performance, one that is without geographical bounds and that
can include many physical locationsâwith their own performers and audience membersâ
within a single event. They explore a unique medium while drawing on historical and
contemporary performing arts practices that normally occur within the confines of a single
physical location.
Such performances present a special set of requirements on the system that supports them.
They need to access and integrate all the systems that are typically found in the physical
place of the performance (such as theatre lighting) with those that are unique to the
medium, such as network technologies and environments for the delivery of computer
generated realities. Yet, no suitable systems or frameworks have been developed to support
them. Technologies are available (such as LoLA and UltraGrid) that support individual
aspectsâlike audio/video streamingâbut which do not address the wider requirements of
controlling and synchronising, of integrating all these technologies into a system of systems
for X reality networked performance.
Therefore, this research investigates the creation of a systems framework whereby existing
hardware and software components can be continuously integrated with bespoke
components to provide a platform for the delivery of X reality networked performances. The
methodological approach to this investigation is through the lens of the authorâs previous
experience in other fields of complex systems integration, including, approaches employed
in the design and integration of avionics systems.
Specifically, it tests if a systems integration approach to providing a technical platform for X
reality networked performances, one that employs stronglyâdefined interfaces and
communication protocols, and that is based on open and industry standards, delivers an
elegant platform that can be characterised as: deterministic, reliable, extendable, scalable,
reconfigurable, testable and cost effective.
iii
The platform for X reality networked performance has been developed iterativelyâusing
the results of a framework investigationâand tested in four different performance projects
over a period of 24âmonths, in ten different venues, across five countries. The research
concludes that the enabling framework is well suited to the delivery of X reality networked
performances. Also, that the approaches employed could equally be applied to the needs of
other arts practitioners who rely on complex technical systems for the creation and delivery
of their work
An Abstraction Framework for Tangible Interactive Surfaces
This cumulative dissertation discusses - by the example of four subsequent publications - the various layers of a tangible interaction framework, which has been developed in conjunction with an electronic musical instrument with a tabletop tangible user interface. Based on the experiences that have been collected during the design and implementation of that particular musical application, this research mainly concentrates on the definition of a general-purpose abstraction model for the encapsulation of physical interface components that are commonly employed in the context of an interactive surface environment. Along with a detailed description of the underlying abstraction model, this dissertation also describes an actual implementation in the form of a detailed protocol syntax, which constitutes the common element of a distributed architecture for the construction of surface-based tangible user interfaces. The initial implementation of the presented abstraction model within an actual application toolkit is comprised of the TUIO protocol and the related computer-vision based object and multi-touch tracking software reacTIVision, along with its principal application within the Reactable synthesizer. The dissertation concludes with an evaluation and extension of the initial TUIO model, by presenting TUIO2 - a next generation abstraction model designed for a more comprehensive range of tangible interaction platforms and related application scenarios