In this paper we present an approach to the transcription of musical queries based on a HMM. The HMM is used to model the audio features related to the singing voice, and the transcription is obtained through Viterbi decoding. We report our preliminary work on evaluation of the system

Matteo Sisti Sette

Nicola Orio

In this paper we present a first step towards a new approach to the transcription of musical queries, based on a HMM.  The HMM is used to model the audio features related to the singing voice, and the transcription is obtained through Viterbi decoding.  We report a preliminary evaluation of the proposed approach on a small number of sung queries

ORIO, NICOLA

SISTI SETTE M.

Archivio istituzionale della ricerca - Università di Padova

A HMM-Based Pitch Tracker for Audio Queries

JScholarship

A HMM-Based Pitch Tracker for Audio QueriesNicola Orio and Matteo Sisti SetteDepartment of Information Engineering, University of PadovaVia Gradenigo, 6/B35131 Padova, Italy{orio, msistis }@dei.unipd.itAbstractIn this paper we present an approach to the transcrip-tion of musical queries based on a hidden Markovmodel (HMM). The HMM is used to model the audiofeatures related to the singing voice, and the transcrip-tion is obtained through Viterbi decoding. We reportour preliminary work on evaluation of the system.1 IntroductionPotential users of a Music Information Retrieval (MIR) systemare likely to express their information needs through thequ ry-by-hummingparadigm. The user’s query usually needs to betranscribed for segmenting the audio signal in a sequence ofevents and describing each event with its features, normally thepitch. The effectiveness of a MIR system depends also on thequality of the query transcription. Research on automatic pitchtracking has been extensively carried out for many years, ap-plying different methods, including autocorrelation (Cheveign´and Kawahara, 2002), auditory models (Clarisse et al., 2002),and classical frequency or mel-cepstrum analyses; moreover, anumber of commercial pitch trackers is already available. Yet,there are some peculiarities of queries sung by non expert usersthat require more research on pitch tracking . The query canbe systematically or locally out of tune and notes can start withor be connected by glissandi and be recorded in a noisy en-vironment. Moreover, queries can be whistled, hummed withdifferent syllables, or sung with the lyrics. All these problemsneed to be taken into account during the development of a MIRfront-end.The presented approach has been developed as a part of an ex-isting MIR system (Melucci and Orio, 1999), which is based onthe extraction of relevant musical phrases as content descriptorsof music documents. Different levels of normalizations are usedto deal with errors in the sung queries. The output of the HMM-based pitch tracker is a sequence of MIDI-like music events,which is used as the input for the musical phrase extraction andnormalization of musical queries.Permission to make digital or hard copies of all or part of this workfor personal or classroom use is granted without fee provided thatcopies are not made or distributed for profit or commercial advan-tage and that copies bear this notice and the full citation on the firstpage. c©2003 Johns Hopkins University.2 Segmentation and pitch tracking with aHidden Markov ModelHidden Markov models (HMMs) have been applied in manydifferent areas, from speech recognition and biological se-quence analysis, to text and music information retrieval. Anextensive description of HMMs is given in (Rabiner and Juang,1993). HMMs have been applied to the pitch tracking ofspeech by Wu et al. (2001), while an application of HMMs tothe alignment of performances with scores was presented byRaphael (1999).From a statistical point of view, a sung query can be consideredas the observation of an unknown process, which is the melodythe user has in mind. What is observed is a set of audio features,which has to be chosen. The process can be modeled with aHMM, where transition probabilities concern both the evolutionof a given note (e.g., attack, sustain, release, and pauses) andthe possible sequences of notes (e.g., the probability that theuser sings a given interval), and emission probabilities refer tothe observed audio features. The transcription reduces then tothe discovery of the evolution along states in the HMM thatmost likely generated the query, which can be obtained throughViterbi decoding.We propose the use of a two-level HMM. Theevent levelis a setof states, each one labeled with a different pitch in the chromaticscale inside a given range. At this level a melody is modeledas a path among the states. The network is fully connected, asshown in Figure 1(a), but the probability to go from a state to an-other one depends on the melodic interval between their labels.Transition probabilities can be set to the a priori probability ofa musical interval in a given repertoire. Theaudio levelis aset of states connected with a left-to-right topology, with labelsrelated to the evolution of a musical event. Figure 1(b) showsa simple topology of a note event at the audio level, with threestates representing attack, sustain, and a final rest which may beskipped. This is the audio level topology we have used in ourpreliminary tests, but we believe that the great potential of theHMM-based approach lies in the possibility to implement morecomplex topologies. In Figure 2 we propose an extension to themodel of Figure 1(b), with multiple parallel attack states (withdifferent emission probability densities) representing differentkinds of attacks, e.g. vowel and consonant attacks, from silenceand from the steady state of the previous note. Multiple consec-utive sustain states (possibly with identical emission densities)allow to impose a lower bound to the note duration. Additionalstates, marked with ‘+’ and ‘-’ in the figure, are added in orderto model slight detunings during the sustain of a note. Otherevents, such as glissandi, vibrati, or thrills, can be representedby introducing other topologies and states at both levels.¹¸º· ¹¸º·¹¸º·±NNAAAKAAAU©©©*©©©¼PPPPPqPPPPPi. . .Do2Mi5Do#2l l l- - - -N N N3a s r(a) (b)Figure 1: The HMM topology. (a) The event level. (b) Theaudio level used in our experiments:a, s andr stand forattack,sustain, andrest respectively; squares represent null states.XzXz»:»:lalalala»:¡µXz@R lsi+± °i−NM- . . . - lsi+± °i−NM- lr--Figure 2: A topology for the audio level modeling differentkinds of attacks, durations, and slight detunings; self transitionarcs are not shown.The complete HMM is given by the definition of the emissionprobabilities. These probabilities need to be set and computedonly for states of the audio level, because states at the event levelare simply formed of states at the audio level. The features wehave considered and tested so far are: the log-energy of the sig-nal, for discriminating between silences and sounds; the sumof the energy in the bands of the first harmonics for each notein the melodic range, for distinguishing different pitches; thefirst derivatives of both these features, for a more precise detec-tion of note attacks; the maximum difference between energiesin adjacent harmonic bands, to reduce the common problem ofdoubling and halving octaves.The chosen representation of the spectrum, simply based on theoverall energy on the harmonic bands, helps dealing with prob-lems like missing harmonics or distortions due to possible user’slow-quality audio equipment; moreover the size of each band ischosen to model temporary detunings of at most a quarter-tone.Since users may not have absolute pitch, a preprocessing step iscarried out to estimate the most probable distance between theuser’s reference and the typical440 Hz to set accordingly theharmonic bands. We are working on adding other features, suchas the difference between the energies in the bands of contigu-ous pitches in the chromatic scale, in order to model glissandiand small variations of pitch inside a note, and to improve themodelling of attacks.All the features are modeled with unilateral exponential proba-bility density functions (pdfs); preliminary tests with Gaussiansgave slightly worse results. The exponential pdfs are trainedusing a statistical analysis on a set of labeled audio examples.States of the same kind (e.g., all sustain states) belonging to dif-ferent notes share the same parameters; they differ in the waythe harmonic bands are computed.The transcription can be obtained through decoding, consider-ing that the HMM has a transition each time a new audio frameis analyzed and a new set of features is observed. The pathalong states at the event level gives both the segmentation andthe pitch tracking, because each state is associated to an eventthrough its label.3 EvaluationWe carried out a preliminary evaluation of our approach, usingthe HMM shown in Figure 1. We tested the system with18audio queries sung by untrained users, using a low quality mi-crophone and a common digital audio interface. Queries had anoverall number of300 notes.Results are summarized in Table 1. Errors are mainly due toglissandi, which were not taken into account as possible expres-sive gestures and so were tracked by the system as chromaticscales preceding notes. Other common errors are the insertionor the wrong recognition of notes a halfstep from the correctone. These results highlight some improvements that can beapplied to the simple model, which will be part of our futurework.omitted total 4%notes repetition of same note 3%pitch errors total 10%(whole notes) ±1 semitone 8%> 1fifth 2%notes inserted total 13%during notes repetition of same note 3%1-2 semitones from correct note8%> 1fifth from correct note 2%notes inserted during silence 1%multiple notes detected during attacks 27%Table 1: Preliminary test results. All percentages refer to thetotal number of notes.ReferencesCheveigńe, A. & Kawahara, H. (2002). YIN, a fundamental fre-quency estimator for speech and music.Journal of the AcousticSociety of America, 111(4), 1917–1930.Clarisse, L. P. et al. (2002). An auditory model based transcriberof singing sequences. InProceedings of the International Con-ference on Music Information Retrieval, (pp. 116–123).Melucci, M. & Orio, N. (1999). Musical Information RetrievalUsing Melodic Surface. InProceedings of IV ACM Conferenceon Digital Libraries, (pp. 152–160).Rabiner, L. & Juang, B.H. (1993).Fundamentals of speechrecognition. (pp. 321–389). Prentice Hall, Englewood Cliffs.Raphael, C. (1999). Automatic segmentation of acoustic musi-cal signals using Hidden Markov Models.IEEE Transactionson Pattern Analysis and Machine Intelligence, 21(4), 360–370.Wu, M. et al. (2001). Pitch Tracking Based on Statistical An-ticipation. InProceedings of the Internetional Joint Conferenceon Neural Networks, Volume 2, (pp. 866–871). DC.

https://jscholarship.library.jhu.edu/bitstream/1774.2/46/1/paper.pdf

A HMM-Based Pitch Tracker for Audio Queries

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Padova

JScholarship