Optimal spectral transportation with application to music transcription

International audienceMany spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timbre can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data. 1 Context Many of nowadays spectral unmixing techniques rely on non-negative matrix decompositions. This concerns for example hyperspectral remote sensing (with applications in Earth observation, astronomy, chemistry, etc.) or audio signal processing. The spectral sample v n (the spectrum of light observed at a given pixel n, or the audio spectrum in a given time frame n) is decomposed onto a dictionary W of elementary spectral templates, characteristic of pure materials or sound objects, such that v n ≈ Wh n. The composition of sample n can be inferred from the non-negative expansion coefficients h n. This paradigm has led to state-of-the-art results for various tasks (recognition, classification, denoising, separation) in the aforementioned areas, and in particular in music transcription, the central application of this paper. In state-of-the-art music transcription systems, the spectrogram V (with columns v n) of a musical signal is decomposed onto a dictionary of pure notes (in so-called multi-pitch estimation) or chords. V typically consists of (power-)magnitude values of a regular short-time Fourier transform (Smaragdis and Brown, 2003). It may also consists of an audio-specific spectral transform such as the Mel-frequency transform, like in (Vincent et al., 2010), or the Q-constant based transform, like in (Oudre et al., 2011). The success of the transcription system depends of course on the adequacy of the time-frequency transform & the dictionary to represent the data V

Machine Annotation of Traditional Irish Dance Music

The work presented in this thesis is validated in experiments using 130 realworld field recordings of traditional music from sessions, classes, concerts and commercial recordings. Test audio includes solo and ensemble playing on a variety of instruments recorded in real-world settings such as noisy public sessions. Results are reported using standard measures from the field of information retrieval (IR) including accuracy, error, precision and recall and the system is compared to alternative approaches for CBMIR common in the literature

Fast Retrieval Algorithm Using EMD Lower and Upper Bounds and a Search Algorithm in multidimensional index

Comparison of images requires a distance metric that is sensitive to the spatial location of objects and features. The Earth Mover’s Distance was introduced in Computer Vision to better approach human perceptual similarities. Its computation, however, is too complex for usage in interactive multimedia database scenarios. We develop new upper bounding approximation techniques for the Earth Mover’s Distance which satisfy high quality criteria and fast computation. In order to enable efficient query processing in large databases, we propose an index structure LUBMTree (Lower and Upper Bounds MTree), based of using the lower and upper bounds for the EMD to improve the search time. Experiments show the performance of research in the  LUBMTree compared with those obtained by  the research in the MTree. Keywords : indexing, similarity, search, signature, metric EMD, MTree, MAM

Cognitively-motivated geometric methods of pattern discovery and models of similarity in music

This thesis is concerned with cognitively-motivated representations of musical structure. Three problems are addressed, each related in terms of their focus on music as an object of perception, and in the application of geometrical methods of knowledge representation. The problem of pattern discovery in discrete representations of polyphonic music is first considered, and a heuristic proposed which seeks to assist musicological analysis by identifying patterns that may be salient in perception, from a large number of potential patterns. This work is based on geometric principles that are far removed from plausible psychological models of pattern induction, but the method is motivated by psychological evidence for the importance of invariance and repetition in perception. The second and third problems explicitly adopt a cognitive theory of representation, namely the conceptual space framework developed by Gärdenfors (2000). Within this framework, concepts can be represented geometrically within perceptually grounded quality dimensions, and where distance in the space corresponds to similarity. The second problem concerns the prediction of melodic similarity, and the theory of conceptual spaces is investigated in the novel context of point set representations of melodic structure, employing the Earth Mover's Distance metric (Rubner 2000). This work builds on the work of Typke (2007) concerning the application of Earth Mover's Distance to melodic similarity. Evaluation is performed with respect to published psychological data (Müllensiefen 2004), and the MIREX 2005 symbolic melodic similarity evaluation. The third problem concerns the conceptual representation of metrical structure, informed by the psychological theory of metre developed by London (2004). A symbolic formalisation of this theory is developed, alongside two geometrical models of metrical-rhythmic structure, which are evaluated within a genre classification task

Quest: querying music databases by acoustic and textual features

ABSTRACT With continued growth of music content available on the Internet, music information retrieval has attracted increasing attention. An important challenge for music searching is its ability to support both keyword and content based queries efficiently and with high precision. In this paper, we present a music query system − QueST (Query by acouStic and T extual features) to support both keyword and content based retrieval in large music databases. QueST has two distinct features. First, it provides new index schemes that can efficiently handle various queries within a uniform architecture. Concretely, we propose a hybrid structure consisting of Inverted file and Signature file to support keyword search. For content based query, we introduce the notion of similarity to capture various music semantics like melody and genre. We extract acoustic features from a music object, and map it to multiple high-dimension spaces with respect to the similarity notion using PCA and RBF neural network. Second, we design a result fusion scheme, called the Quick Threshold Algorithm, to speed up the processing of complex queries involving both textual and multiple acoustic features. Our experimental results show that QueST offers higher accuracy and efficiency compared to existing algorithms

Music Encoding Conference Proceedings

UIDB/00693/2020 UIDP/00693/2020publishersversionpublishe

Chinese Zheng and Identity Politics in Taiwan

Ph.D. Thesis. University of Hawaiʻi at Mānoa 2018

Pursuing Taiwanese-ness: the contemporary music practices of Taitung indigenous people

This research is an ethnography exploring the contemporary music practices of the indigenous people of Taitung, a southeastern county in Taiwan. Indigenous people make up a large proportion of the population of Taitung, and their music has in recent years been used in international and local events to potray a unique Taiwanese identity. I discuss how indigenous and other Taiwanese have collaborated to create this identity - the Taiwanese-ness - and how they have done so with tangled webs of concerns for authenticity, hybridisation and Otherness. I examine two opposite approaches in heredity and maintenance of the tradition: first, sticking to locality, and therefore passing down the tradition in a functional way; second, endorsing and appropriating transnational pop practices in order to garner commercial success. I argue that living experience - the familiarity to a musical culture which Mantle Hood (1982) considered the way that enabled ethnic groups to understand and evaluate their own musical traditions - is essential and irreplaceable. Hence, affiliation to a homeland, as depicted through notions of mountain and sea, becomes a key element in the self-identity of musicians as 'indigenous' (yuanzhumin in Mandarin, meaning 'original inhabitants'), and that the homeland, as the place of ancestors, allows indigenous groups to safeguard their traditions. However, indigenous Taiwanese are comfortable with and uphold a shared culture that was brought to the island by Han migrants, and this is evident in the influences of trans-cultural commercial and global Mandopop. Musicians tend to apply elements of their traditions such as indigenous languages, pentatonicism, ancient songs, specific rhythms and the incorporation of non-lexical vocables, wherever they can, using a bricolage approach. At the same time, musicians enrich the music culture, keeping tradition alive by adding to it in reciprocal ways elements from the outside, but also introducing the po­tential for cultural 'grey-out' as elements of traditional music are altered. Keywords: Taitung, indigenous people, music practices, mountain and sea, Taiwanese-ness