Corpus-Based Transcription as an Approach to the Compositional Control of Timbre

Timbre space is a cognitive model useful to address the problem of structuring timbre in electronic music. The recent concept of corpus-based concatenative sound synthesis is proposed as an approach to timbral control in both real- and deferred-time applications. Using CataRT and related tools in the FTM and Gabor libraries for Max/MSP we describe a technique for real-time analysis of a live signal to pilot corpus-based synthesis, along with examples of compositional realizations in works for instruments, electronics, and sound installation. To extend this technique to computer-assisted composition for acoustic instruments, we develop tools using the Sound Description Interchange Format (SDIF) to export sonic descriptors to OpenMusic where they may be further manipulated and transcribed into an instrumental score. This presents a flexible technique for the compositional organization of noise-based instrumental sounds

Affective Music Information Retrieval

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called \emph{acoustic emotion Gaussians} (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a larger-scale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.Comment: 40 pages, 18 figures, 5 tables, author versio

Learning to detect video events from zero or very few video examples

In this work we deal with the problem of high-level event detection in video. Specifically, we study the challenging problems of i) learning to detect video events from solely a textual description of the event, without using any positive video examples, and ii) additionally exploiting very few positive training samples together with a small number of ``related'' videos. For learning only from an event's textual description, we first identify a general learning framework and then study the impact of different design choices for various stages of this framework. For additionally learning from example videos, when true positive training samples are scarce, we employ an extension of the Support Vector Machine that allows us to exploit ``related'' event videos by automatically introducing different weights for subsets of the videos in the overall training set. Experimental evaluations performed on the large-scale TRECVID MED 2014 video dataset provide insight on the effectiveness of the proposed methods.Comment: Image and Vision Computing Journal, Elsevier, 2015, accepted for publicatio

MIDI-based music score editor

[Abstract] Music has always played a fundamental role in society. It is a basic human function and, as a consequence people have always found music significant whether it is for enjoyment in listening, its emotional response, performing or creating. Music is represented using music notation. Music notation is the transcription of music into music scores. Music scores have been a way of teaching and sharing music with others. In addition, it was a key contributor to evolving music language and allowing the works of composers to be preserved over time. The continuous advancements in technology have influenced both the creation and recording of music. One of the milestones for music creation and recording was Musical Instrument Digital Interface (MIDI). MIDI is a protocol that provides a standardized and efficient means of conveying musical performance information as electronic data. Then, it is possible to obtain musical notation information by just processing the electronic data inside a MIDI file. The objective of this project is to develop and implement a MIDI based music score creator. Users will be able to load MIDI files, process them and generate the correspondent music score. To achieve this, a combination of musical knowledge and information obtained by the MIDI protocol will be used to create the accurate scores.[Resumo] A música sempre xogou un papel fundamental na sociedade. A música é unha función básica nos humanos e polo tanto, as música sempre foi importante para as persoas, xa fora para disfrutar escoitándoa, pola resposta emocional, para interpretala ou para creala. A música é representada pola notación musical. A notación musical consiste na transcipción de música en partituras. As partituras músicales sempre foron unha forma de enseñar e compartir música entre as persoas. Ademais, as partituras foron un contribuinte clave na evolución da linguaxe musical e permitiron preservar as obras creadas por diferentes compositores a través do tempo. Os avances continuos en tecnoloxía influenciaron tanto a creación como a grabación da música. Un dos hitos máis importantes para a creación e grabación de musica foi a creación de MIDI. O protocolo MIDI proporciona un medio estandarizado e eficiente para transmitir información musical como datos electrónicos. Polo tanto, é posible obtener información sobre notación musical simplemente procesando os datos electrónicos dentro dun archivo MIDI. O obxectivo deste proxecto e desarrollar e implementar un creador de partituras musicales baseado en MIDI. Os usuarios poderán cargar archivos MIDI, procesales e xerar a partitura correspondiente. Para logralo, utilizarase unha combination de conocimientos musicales e de información obtida polo protocolo MIDI para crear partituras precisas.Traballo fin de grao (UDC.FIC). Enxeñaría informática. Curso 2019/202

Notation Sequence Generation and Sound Synthesis in Interactive Spectral Music

Notation sequence generation and sound synthesis in interactive spectral music This thesis consists of a preliminary analysis of existing spectral music paradigms and proposes a methodology to address issues that arise in real-time spectral music composition and performance scenarios. This exploration involves an overview of meaning in spectral music with a particular focus on the ‘sonic object’ as a vehicle for expression. A framework for the production of ‘interactive spectral music’ was created. This framework takes form as a group of software based compositional tools called SpectraScore developed for the Max for Live platform. Primarily, these tools allow the user to analyse incoming audio and directly apply the collected data towards the generation of synthesised sound and notation sequences. Also presented is an extension of these tools, a novel system of correlation between emotional descriptors and spectrally derived harmonic morphemes. The final component is a portfolio of works created as examples of the techniques explored in scored and recorded form. As a companion to these works, an analysis component outlines the programmatic aspects of each piece and illustrates how they are executed within the music. Each scored piece corresponds with a recording of a live performance or performances of the work included in the attached DVD, which comprises individual realisations of the interactive works. Keywords: Spectralism, Music and Emotion, Electronic Music, Spectral Music, Algorithmic Music, Real-time Notatio

Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

This paper demonstrates that word sense disambiguation (WSD) can improve neural machine translation (NMT) by widening the source context considered when modeling the senses of potentially ambiguous words. We first introduce three adaptive clustering algorithms for WSD, based on k-means, Chinese restaurant processes, and random walks, which are then applied to large word contexts represented in a low-rank space and evaluated on SemEval shared-task data. We then learn word vectors jointly with sense vectors defined by our best WSD method, within a state-of-the-art NMT system. We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones. This is demonstrated by translation on five language pairs. The improvements are above one BLEU point over strong NMT baselines, +4% accuracy over all ambiguous nouns and verbs, or +20% when scored manually over several challenging words.Comment: To appear in TAC

Notation Sequence Generation and Sound Synthesis in Interactive Spectral Music

Notation sequence generation and sound synthesis in interactive spectral music This thesis consists of a preliminary analysis of existing spectral music paradigms and proposes a methodology to address issues that arise in real-time spectral music composition and performance scenarios. This exploration involves an overview of meaning in spectral music with a particular focus on the ‘sonic object’ as a vehicle for expression. A framework for the production of ‘interactive spectral music’ was created. This framework takes form as a group of software based compositional tools called SpectraScore developed for the Max for Live platform. Primarily, these tools allow the user to analyse incoming audio and directly apply the collected data towards the generation of synthesised sound and notation sequences. Also presented is an extension of these tools, a novel system of correlation between emotional descriptors and spectrally derived harmonic morphemes. The final component is a portfolio of works created as examples of the techniques explored in scored and recorded form. As a companion to these works, an analysis component outlines the programmatic aspects of each piece and illustrates how they are executed within the music. Each scored piece corresponds with a recording of a live performance or performances of the work included in the attached DVD, which comprises individual realisations of the interactive works. Keywords: Spectralism, Music and Emotion, Electronic Music, Spectral Music, Algorithmic Music, Real-time Notatio

Mobile Music Development Tools for Creative Coders

This project is a body of work that facilitates the creation of musical mobile artworks. The project includes a code toolkit that enhances and simplifies the development of mobile music iOS applications, a flexible notation system designed for mobile musical interactions, and example apps and scored compositions to demonstrate the toolkit and notation system. The code library is designed to simplify the technical aspect of user-centered design and development with a more direct connection between concept and deliverable. This sim- plification addresses learning problems (such as motivation, self-efficacy, and self-perceived understanding) by bridging the gap between idea and functional prototype and improving the ability to contextualize the development process for musicians and other creatives. The toolkit helps to circumvent the need to learn complex iOS development patterns and affords more readable code. CSPD (color, shape, pattern, density) notation is a pseudo-tablature that describes performance interactions. The system leverages visual density and patterns of both color and shape to describe types of gestures (physical or musical) and their relationships rather than focusing on strict rhythmic or pitch/frequency content. The primary design goal is to visualize macro musical concepts that create middleground structure

Gesture cutting through textual complexity: Towards a tool for online gestural analysis and control of complex piano notation processing

International audienceThis project introduces a recently developed prototype for real-time processing and control of complex piano notation through the pianist’s gesture. The tool materializes an embodied cognition-influenced paradigm of interaction of pianists with complex notation (embodied or corporeal navigation), drawing from latest developments in the computer music fields of musical representation (augmented and interactive musical scores via INScore) and of multimodal interaction (Gesture Follower). Gestural, video, audio and MIDI data are appropriately mapped on the musical score, turning it into a personalized, dynamic, multimodal tablature. This tablature may be used for efficient learning, performance and archiving, with potential applications in pedagogy, composition, improvisation and score following. The underlying metaphor for such a tool is that instrumentalists touch or cut through notational complexity using performative gestures, as much as they touch their own keyboards. Their action on the instrument forms integral part of their understanding, which can be represented as a gestural processing of the notation. Next to the already mentioned applications, new perspectives in piano performance of post-1945 complex notation and in musicology (‘performative turn’), as well as the emerging field of ‘embodied and extended cognition’, are indispensable for this project