Pitch-Informed Solo and Accompaniment Separation

Das Thema dieser Dissertation ist die Entwicklung eines Systems zur Tonhöhen-informierten Quellentrennung von Musiksignalen in Soloinstrument und Begleitung. Dieses ist geeignet, die dominanten Instrumente aus einem Musikstück zu isolieren, unabhängig von der Art des Instruments, der Begleitung und Stilrichtung. Dabei werden nur einstimmige Melodieinstrumente in Betracht gezogen. Die Musikaufnahmen liegen monaural vor, es kann also keine zusätzliche Information aus der Verteilung der Instrumente im Stereo-Panorama gewonnen werden. Die entwickelte Methode nutzt Tonhöhen-Information als Basis für eine sinusoidale Modellierung der spektralen Eigenschaften des Soloinstruments aus dem Musikmischsignal. Anstatt die spektralen Informationen pro Frame zu bestimmen, werden in der vorgeschlagenen Methode Tonobjekte für die Separation genutzt. Tonobjekt-basierte Verarbeitung ermöglicht es, zusätzlich die Notenanfänge zu verfeinern, transiente Artefakte zu reduzieren, gemeinsame Amplitudenmodulation (Common Amplitude Modulation CAM) einzubeziehen und besser nichtharmonische Elemente der Töne abzuschätzen. Der vorgestellte Algorithmus zur Quellentrennung von Soloinstrument und Begleitung ermöglicht eine Echtzeitverarbeitung und ist somit relevant für den praktischen Einsatz. Ein Experiment zur besseren Modellierung der Zusammenhänge zwischen Magnitude, Phase und Feinfrequenz von isolierten Instrumententönen wurde durchgeführt. Als Ergebnis konnte die Kontinuität der zeitlichen Einhüllenden, die Inharmonizität bestimmter Musikinstrumente und die Auswertung des Phasenfortschritts für die vorgestellte Methode ausgenutzt werden. Zusätzlich wurde ein Algorithmus für die Quellentrennung in perkussive und harmonische Signalanteile auf Basis des Phasenfortschritts entwickelt. Dieser erreicht ein verbesserte perzeptuelle Qualität der harmonischen und perkussiven Signale gegenüber vergleichbaren Methoden nach dem Stand der Technik. Die vorgestellte Methode zur Klangquellentrennung in Soloinstrument und Begleitung wurde zu den Evaluationskampagnen SiSEC 2011 und SiSEC 2013 eingereicht. Dort konnten vergleichbare Ergebnisse im Hinblick auf perzeptuelle Bewertungsmaße erzielt werden. Die Qualität eines Referenzalgorithmus im Hinblick auf den in dieser Dissertation beschriebenen Instrumentaldatensatz übertroffen werden. Als ein Anwendungsszenario für die Klangquellentrennung in Solo und Begleitung wurde ein Hörtest durchgeführt, der die Qualitätsanforderungen an Quellentrennung im Kontext von Musiklernsoftware bewerten sollte. Die Ergebnisse dieses Hörtests zeigen, dass die Solo- und Begleitspur gemäß unterschiedlicher Qualitätskriterien getrennt werden sollten. Die Musiklernsoftware Songs2See integriert die vorgestellte Klangquellentrennung bereits in einer kommerziell erhältlichen Anwendung.This thesis addresses the development of a system for pitch-informed solo and accompaniment separation capable of separating main instruments from music accompaniment regardless of the musical genre of the track, or type of music accompaniment. For the solo instrument, only pitched monophonic instruments were considered in a single-channel scenario where no panning or spatial location information is available. In the proposed method, pitch information is used as an initial stage of a sinusoidal modeling approach that attempts to estimate the spectral information of the solo instrument from a given audio mixture. Instead of estimating the solo instrument on a frame by frame basis, the proposed method gathers information of tone objects to perform separation. Tone-based processing allowed the inclusion of novel processing stages for attack refinement, transient interference reduction, common amplitude modulation (CAM) of tone objects, and for better estimation of non-harmonic elements that can occur in musical instrument tones. The proposed solo and accompaniment algorithm is an efficient method suitable for real-world applications. A study was conducted to better model magnitude, frequency, and phase of isolated musical instrument tones. As a result of this study, temporal envelope smoothness, inharmonicty of musical instruments, and phase expectation were exploited in the proposed separation method. Additionally, an algorithm for harmonic/percussive separation based on phase expectation was proposed. The algorithm shows improved perceptual quality with respect to state-of-the-art methods for harmonic/percussive separation. The proposed solo and accompaniment method obtained perceptual quality scores comparable to other state-of-the-art algorithms under the SiSEC 2011 and SiSEC 2013 campaigns, and outperformed the comparison algorithm on the instrumental dataset described in this thesis.As a use-case of solo and accompaniment separation, a listening test procedure was conducted to assess separation quality requirements in the context of music education. Results from the listening test showed that solo and accompaniment tracks should be optimized differently to suit quality requirements of music education. The Songs2See application was presented as commercial music learning software which includes the proposed solo and accompaniment separation method

A microtonal wind controller building on Yamaha’s technology to facilitate the performance of music based on the “19-EDO” scale

We describe a project in which several collaborators adapted an existing instrument to make it capable of playing expressively in music based on the microtonal scale characterised by equal divsion of the octave into 19 tones (“19-EDO”). Our objective was not just to build this instrument, however, but also to produce a well-formed piece of music which would exploit it idiomatically, in a performance which would provide listeners with a pleasurable and satisfying musical experience. Hence, consideration of the extent and limits of the playing-techniques of the resulting instrument (a “Wind-Controller”) and of appropriate approaches to the composition of music for it were an integral part of the project from the start. Moreover, the intention was also that the piece, though grounded in the musical characteristics of the 19-EDO scale, would nevertheless have a recognisable relationship with what Dimitri Tymoczko (2010) has called the “Extended Common Practice” of the last millennium. So the article goes on to consider these matters, and to present a score of the resulting new piece, annotated with comments documenting some of the performance issues which it raises. Thus, bringing the project to fruition involved elements of composition, performance, engineering and computing, and the article describes how such an inter-disciplinary, multi-disciplinary and cross-disciplinary collaboration was co-ordinated in a unified manner to achieve the envisaged outcome. Finally, we consider why the building of microtonal instruments is such a problematic issue in a contemporary (“high-tech”) society like ours

Proceedings of the 6th International Workshop on Folk Music Analysis, 15-17 June, 2016

The Folk Music Analysis Workshop brings together computational music analysis and ethnomusicology. Both symbolic and audio representations of music are considered, with a broad range of scientific approaches being applied (signal processing, graph theory, deep learning). The workshop features a range of interesting talks from international researchers in areas such as Indian classical music, Iranian singing, Ottoman-Turkish Makam music scores, Flamenco singing, Irish traditional music, Georgian traditional music and Dutch folk songs. Invited guest speakers were Anja Volk, Utrecht University and Peter Browne, Technological University Dublin

Technological Support for Highland Piping Tuition and Practice

This thesis presents a complete hardware and software system to support the learning process associated with the Great Highland Bagpipe (GHB). A digital bagpipe chanter interface has been developed to enable accurate measurement of the player's nger movements and bag pressure technique, allowing detailed performance data to be captured and analysed using the software components of the system. To address the challenge of learning the diverse array of ornamentation techniques that are a central aspect of Highland piping, a novel algorithm is presented for the recognition and evaluation of a wide range of embellishments performed using the digital chanter. This allows feedback on the player's execution of the ornaments to be generated. The ornament detection facility is also shown to be e ective for automatic transcription of bagpipe notation, and for performance scoring against a ground truth recording in a game interface, Bagpipe Hero. A graphical user interface (GUI) program provides facilities for visualisation, playback and comparison of multiple performances, and for automatic detection and description of piping-speci c ngering and ornamentation errors. The development of the GUI was informed by feedback from expert pipers and a small-scale user study with students. The complete system was tested in a series of studies examining both lesson and solo practice situations. A detailed analysis of these sessions was conducted, and a range of usage patterns was observed in terms of how the system contributed to the di erent learning environments. This work is an example of a digital interface designed to connect to a long established and highly formalised musical style. Through careful consideration of the speci c challenges faced in teaching and learning the bagpipes, this thesis demonstrates how digital technologies can provide a meaningful contribution to even the most conservative cultural traditions.This work was funded by the Engineering and Physical Sciences Research Council (EPSRC) as part of the Doctoral Training Centre in Media and Arts Technology at Queen Mary University of London (ref: EP/G03723X/1)

A User-assisted Approach to Multiple Instrument Music Transcription

PhDThe task of automatic music transcription has been studied for several decades and is regarded as an enabling technology for a multitude of applications such as music retrieval and discovery, intelligent music processing and large-scale musicological analyses. It refers to the process of identifying the musical content of a performance and representing it in a symbolic format. Despite its long research history, fully automatic music transcription systems are still error prone and often fail when more complex polyphonic music is analysed. This gives rise to the question in what ways human knowledge can be incorporated in the transcription process. This thesis investigates ways to involve a human user in the transcription process. More specifically, it is investigated how user input can be employed to derive timbre models for the instruments in a music recording, which are employed to obtain instrument-specific (parts-based) transcriptions. A first investigation studies different types of user input in order to derive instrument models by means of a non-negative matrix factorisation framework. The transcription accuracy of the different models is evaluated and a method is proposed that refines the models by allowing each pitch of each instrument to be represented by multiple basis functions. A second study aims at limiting the amount of user input to make the method more applicable in practice. Different methods are considered to estimate missing non-negative basis functions when only a subset of basis functions can be extracted based on the user information. A method is proposed to track the pitches of individual instruments over time by means of a Viterbi framework in which the states at each time frame contain several candidate instrument-pitch combinations. A transition probability is employed that combines three different criteria: the frame-wise reconstruction error of each combination, a pitch continuity measure that favours similar pitches in consecutive frames, and an explicit activity model for each instrument. The method is shown to outperform other state-of-the-art multi-instrument tracking methods. Finally, the extraction of instrument models that include phase information is investigated as a step towards complex matrix decomposition. The phase relations between the partials of harmonic sounds are explored as a time-invariant property that can be employed to form complex-valued basis functions. The application of the model for a user-assisted transcription task is illustrated with a saxophone example.QMU

Gesture and listening: towards a social and eco-systemic hyperinstrument composition

The research implements interactive music processes involving sound synthesis and symbolic treatments within a single environment. The algorithms are driven by classical instrumental performance through hybrid systems called hyperinstruments, in which the sensing of the performance gestures leads to open and goal-oriented generative music forms. The interactions are composed with MAX/Msp, designing contexts and relationships between real-time instrumental timbre analysis (sometimes with added inertial motion tracking) with a gesture-based idea of form shaping. Physical classical instruments are treated as interfaces, giving rise to the need to develop unconventional mapping strategies on account of the multi-dimensional and interconnecting quality of timbre. Performance and sound gestures are viewed as salient energies, phrasings and articulations carrying information about human intentions, in this way becoming able to change the musical behaviour of a composition inside a coded dramaturgy. The interactive networks are designed in order to integrate traditional music practices and “languages” with computational systems designed to be self-regulating, through the mediation of timbre space and performance gestural descriptions. Following its classic definition, technology aims to be mainly related not to mechanical practices but rather to rhetorical approaches: for this reason the software often foresees interactive scores, and must be performed in accordance with a set of external verbal (and video) explanations, whose technical detail should nevertheless not impair the most intuitive approach to music making

Reworking musical strategies in the digital age

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University, 2/2/2011.This thesis comprises a portfolio of creative musical work and a written commentary. The creative work seeks to rework musical strategies through technology by challenging aspects of how music is traditionally performed, composed and presented. The portfolio of submitted work is divided into five main projects. The first project is E-tudes, a set of four compositions for live electronics and six keyboard players. The second project is a composition called On Violence, for piano, live electronics, sensors and computer display. The third project is Zizek!?, a computer-mediated-performance for three improvisers that serves as an alternative soundtrack to a documentary about Slovenian philosopher Slavoj Zizek. The fourth project is a collection of small experimental pieces for fixed media called FreuPinta. The fifth project consists of a selection of different improvisations that I devised or participated in using a computer environment I developed for live improvisation. Throughout the portfolio recent technological advancements are considered not for their use in implementing pre-existing models of music-making but rather for their potential to challenge preconceived notions about music. The written commentary gives the theoretical tools necessary to understand the underlying reasoning, preoccupations and concerns behind the submitted work as well as providing supplementary information about the musical results and the computer programmes developed as part of this research.Financial support from the Overseas Research Scheme (ORS), Brunel University School of Arts and the PRS for Music Foundation was used for this study

Composing Music for Acoustic Instruments and Electronics Mediated Through the Application of Microsound

This project seeks to extend, through a portfolio of compositions, the use of microsound to mixed works incorporating acoustic instrument and electronics. Issues relating to the notation of microsound when used with acoustic instruments are explored and the adoption of a clear and intuitive system of graphical notation is proposed. The design of the performance environment for the electroacoustic part is discussed and different models for the control of the electronics are considered. Issues relating to structure and form when applied to compositions that mix note-based material with texture-based material are also considered. A framework based on a pure sound/noise continuum, used in conjunction with a hierarchy of gestural archetypes, is adopted as a possible solution to the challenges of structuring mixed compositions. Gestural and textural relationships between different parts of the compositions are also explored and the use of extended instrumental techniques to create continua between the acoustic and the electroacoustic is adopted. The role of aleatoric techniques and improvisation in both the acoustic and the electroacoustic parts are explored through adoption of an interactive performance environment incorporating a pitch-tracking algorithm. Finally, the advantages and disadvantages of real time recording and processing of the electronic part when compared with live processing of pre-existing sound-files are discussed

Automatic transcription of polyphonic music exploiting temporal evolution

PhDAutomatic music transcription is the process of converting an audio recording into a symbolic representation using musical notation. It has numerous applications in music information retrieval, computational musicology, and the creation of interactive systems. Even for expert musicians, transcribing polyphonic pieces of music is not a trivial task, and while the problem of automatic pitch estimation for monophonic signals is considered to be solved, the creation of an automated system able to transcribe polyphonic music without setting restrictions on the degree of polyphony and the instrument type still remains open. In this thesis, research on automatic transcription is performed by explicitly incorporating information on the temporal evolution of sounds. First efforts address the problem by focusing on signal processing techniques and by proposing audio features utilising temporal characteristics. Techniques for note onset and offset detection are also utilised for improving transcription performance. Subsequent approaches propose transcription models based on shift-invariant probabilistic latent component analysis (SI-PLCA), modeling the temporal evolution of notes in a multiple-instrument case and supporting frequency modulations in produced notes. Datasets and annotations for transcription research have also been created during this work. Proposed systems have been privately as well as publicly evaluated within the Music Information Retrieval Evaluation eXchange (MIREX) framework. Proposed systems have been shown to outperform several state-of-the-art transcription approaches. Developed techniques have also been employed for other tasks related to music technology, such as for key modulation detection, temperament estimation, and automatic piano tutoring. Finally, proposed music transcription models have also been utilized in a wider context, namely for modeling acoustic scenes