Marchini, Marco

Marxer, Richard

Purwins, Hendrik

The same piece of music maybe be experienced differently by different subjects, depending on various factors: The perceptual processing of the piece (e.g. 3500 inner hair cells vs. a few electrodes in a cochlear implant) may allow for a representation with various degrees of frequency and temporal resolution. The listener’s cognitive capabilities, e.g. memory, voluntary attention direction, abstraction and inference (normal listener vs. a person with dementia) will cause different cognitive processing and musical experience. The familiarity with a particular musical style, instrument, piece may affect the emotional impact and intellectual understanding of a piece.It could be desirable for a hearing device (hearing aid or cochlear implant) to perform musical signal processing to transfer the musical experience of a piece for a normal listener to a similar musical experience of the same piece in a listener with severe hearing limitations. Such a transformation would require a formal representation of the music piece that is scalable in complexity. Such a scalable representation is provided in [1], where the number of different sounds (e.g. considering open and closed hi-hats as one or two distinct sound categories) as well as the temporal context horizon (e.g. storing up to 2-note sequences or up to 10-note sequences) is adaptable. The framework in [1] is based on two cognitively plausible principles: unsupervised learning and statistical learning. Opposed to supervised learning in primary school children, where the school teacher points at a written letter and articulates its phonetic pronunciation, infants perceptually organise the phonemes they are exposed to into groups, based on the phoneme’s similarity and context (unsupervised learning). 8-month-old infants are able to learn statistical relationships between neighboring speech sounds [3] (statistical learning). In [1], Figure 1, unsupervised learning is implemented as agglomerative clustering, informed by the Gestalt principle of regularity. The model [1] performs statistical learning, applying variable length Markov chains. In [2], grouping of sounds into phonetic/instrument categories and learning of instrument event sequences is performed jointly using a Hierarchical Dirichlet Process Hidden Markov Model.Whereas machines often learn by processing a large data base and subsequently updating parameters of the algorithm, humans learn instantaneously, i.e. the mental representation is continuously changed after every exposure to small batches of sound events (online learning). In [2] online learning is implemented via the interplay of Cobweb clustering and a hierarchical n-gram instantaneously updating the number of timbre groups and their respective transition counts. We propose to use online learning for the co-evolution of both CI user and machine in (re-)learning musical language.[1] Marco Marchini and Hendrik Purwins. Unsupervised analysis and generation of audio percussion sequences. In International Symposium on Computer Music Modeling and Retrieval, pages 205–218. Springer, 2011.[2] R. Marxer and H. Purwins. Unsupervised incremental online learning and prediction of musical audio signals. IEEE Transactions on Audio, Speech, and Language Processing, 24(5):863–74, 2016.[3] J. Saffran, R. Aslin, and E. Newport. Statistical learning by 8-month-old infants. Science, 274(5294):1926–1928, 1996

VBN (Videnbasen)  Aalborg Universitets forskningsportal

    Aalborg UniversitetTowards Statistical Unsupervised Online Learning for Music Listening with HearingDevicesPurwins, Hendrik; Marchini, Marco; Marxer, RichardPublished in:First International Music &amp; CI SymposiumPublication date:2017Link to publication from Aalborg UniversityCitation for published version (APA):Purwins, H., Marchini, M., & Marxer, R. (2017). Towards Statistical Unsupervised Online Learning for MusicListening with Hearing Devices. In First International Music & CI Symposium (pp. 41)General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: May 01, 20171Towards Statistical Unsupervised Online Learningfor Music Listening with Hearing DevicesHendrik Purwins , Marco Marchini , and Ricard MarxerThe same piece of music maybe be experienced differentlyby different subjects, depending on various factors: The per-ceptual processing of the piece (e.g. 3500 inner hair cellsvs. a few electrodes in a cochlear implant) may allow for arepresentation with various degrees of frequency and temporalresolution. The listener’s cognitive capabilities, e.g. memory,voluntary attention direction, abstraction and inference (nor-mal listener vs. a person with dementia) will cause differentcognitive processing and musical experience. The familiaritywith a particular musical style, instrument, piece may affectthe emotional impact and intellectual understanding of a piece.It could be desirable for a hearing device (hearing aidor cochlear implant) to perform musical signal processingto transfer the musical experience of a piece for a normallistener to a similar musical experience of the same piece in alistener with severe hearing limitations. Such a transformationwould require a formal representation of the music piecethat is scalable in complexity. Such a scalable representationis provided in [1], where the number of different sounds(e.g. considering open and closed hi-hats as one or twodistinct sound categories) as well as the temporal contexthorizon (e.g. storing up to 2-note sequences or up to 10-H. Purwins is with Audio Analysis Lab and Sound and Music ComputingGroup, Department of Architecture, Design & Media Technology, AalborgUniversitet København. hpu@create.aau.dkM. Marchini is with Sony CSL, 6, Rue Amyot, 75005 Paris, France.marco.marchini3@gmail.comR. Marxer is with the Speech and Hearing Group, Department of Com-puter Science, University of Sheffield, Sheffield S1 4DP, U.K. (e-mail:r.marxer@sheffield.ac.uk.Figure 1. Percussive sound events are grouped into timbre categories (left),using agglomerative clustering (unsupervised learning), yielding a clusteringtree (middle) that can be cut at different resolutions, e.g. following a Gestaltprinciple of rhythmical regularity (dashed line), resulting in correspondingdiscrete sequences (right) that are further analyzed using variable lengthMarkov chains. Such a model is scalable in complexity parameterised bythe number of clusters and the sequence length. [1]-26 -25 -24 −23 −22 −21 −20 −19−3−2−101212345678 9 101112MFCC1MFCC24 7 10 11 3 2 12 6 1 5 8 9Cluster Distance Threshold11.5233.52.5Onset Indexnote sequences) is adaptable. The framework in [1] is basedon two cognitively plausible principles: unsupervised learningand statistical learning. Opposed to supervised learning inprimary school children, where the school teacher points ata written letter and articulates its phonetic pronunciation,infants perceptually organise the phonemes they are exposedto into groups, based on the phoneme’s similarity and context(unsupervised learning). 8-month-old infants are able to learnstatistical relationships between neighboring speech sounds [3](statistical learning). In [1], Figure 1, unsupervised learningis implemented as agglomerative clustering, informed by theGestalt principle of regularity. The model [1] performs statis-tical learning, applying variable length Markov chains. In [2],grouping of sounds into phonetic/instrument categories andlearning of instrument event sequences is performed jointlyusing a Hierarchical Dirichlet Process Hidden Markov Model.Whereas machines often learn by processing a large database and subsequently updating parameters of the algorithm,humans learn instantaneously, i.e. the mental representationis continously changed after every exposure to small batchesof sound events (online learning). In [2] online learningis implemented via the interplay of Cobweb clustering anda hierarchical n-gram instantaneously updating the numberof timbre groups and their respective transition counts. Wepropose to use online learning for the co-evolution of both CIuser and machine in (re-)learning musical language.REFERENCES[1] Marco Marchini and Hendrik Purwins. Unsupervised analysis andgeneration of audio percussion sequences. In International Symposiumon Computer Music Modeling and Retrieval, pages 205–218. Springer,2011.[2] R. Marxer and H. Purwins. Unsupervised incremental online learningand prediction of musical audio signals. IEEE Transactions on Audio,Speech, and Language Processing, 24(5):863–74, 2016.[3] J. Saffran, R. Aslin, and E. Newport. Statistical learning by 8-month-oldinfants. Science, 274(5294):1926–1928, 1996.

Towards Statistical Unsupervised Online Learning for Music Listening with Hearing Devices

http://vbn.aau.dk/files/242982190/stat_music.pdf

Towards Statistical Unsupervised Online Learning for Music Listening with Hearing Devices

Abstract

Similar works

Full text

Available Versions

VBN (Videnbasen) Aalborg Universitets forskningsportal