Modeling the auditory scene: predictive regularity representations and perceptual objects

Predictive processing of information is essential for goal directed behavior. We offer an account of auditory perception suggesting that representations of predictable patterns, or ‘regularities’, extracted from the incoming sounds serve as auditory perceptual objects. The auditory system continuously searches for regularities within the acoustic signal. Primitive regularities may be encoded by neurons adapting their response to specific sounds. Such neurons have been observed in many parts of the auditory system. Representations of the detected regularities produce predictions of upcoming sounds as well as alternative solutions for parsing the composite input into coherent sequences potentially emitted by putative sound sources. Accuracy of the predictions can be utilized for selecting the most likely interpretation of the auditory input. Thus in our view, perception generates hypotheses about the causal structure of the world

Sensory auditory processing and intuitive sound detection : an investigation of musical experts and nonexperts

The auditory system can detect occasional changes (deviants) in acoustic regularities without the need for subjects to focus their attention on the sound material. Deviant detection is reflected in the elicitation of the mismatch negativity component (MMN) of the event-related potentials. In the studies presented in this thesis, the MMN is used to investigate the auditory abilities for detecting similarities and regularities in sound streams. To investigate the limits of these processes, professional musicians have been tested in some of the studies. The results show that auditory grouping is already more advanced in musicians than in nonmusicians and that the auditory system of musicians can, unlike that of nonmusicians, detect a numerical regularity of always four tones in a series. These results suggest that sensory auditory processing in musicians is not only a fine tuning of universal abilities, but is also qualitatively more advanced than in nonmusicians. In addition, the relationship between the auditory change-detection function and perception is examined. It is shown that, contrary to the generally accepted view, MMN elicitation does not necessarily correlate with perception. The outcome of the auditory change-detection function can be implicit and the implicit knowledge of the sound structure can, after training, be utilized for behaviorally correct intuitive sound detection. These results illustrate the automatic character of the sensory change detection function.Kuulojärjestelmä analysoi akustisia tapahtumia jäsentääkseen meitä ympäröiviä ääniä. Kuulohavaintoon liittyvät kyvyt eivät ole staattisia, vaan niitä voidaan muokata harjoittelulla. Esimerkiksi viulisti pystyy erottamaan paljon pienempiä sävelkorkeuseroja kuin joku, jonka korva ei ole samalla tavalla kehittynyt harjoittelussa. Tästä syystä muusikot ovat mielenkiintoinen kohderyhmä kuulohavaintoon liittyvien kykyjen tutkimuksessa. Osa kuuloprosesseista tapahtuu automaattisesti, vaikka emme ole edes aina tietoisia meitä ympäröivistä äänistä. Joskus emme tarkkaile ääntä kuinnes se yhtäkkiä muuttuu. Toisinaan taas emme ole täysin tietoisia äänen muutostakaan, mutta tarkoittaako tämä sitä, että se ei voi vaikuttaa toimintaamme? Tämän väitöskirjan tutkimuksissa aivosähkökäyrää eli EEG:tä mitattiin pään pinnalle kiinnitetyistä elektrodeista. EEG:llä mitatuista sähkökentistä voidaan päätellä, milloin kuulojärjestelmä on löytänyt satunnaisen muutoksen äänisarjassa, joka on muuten säännöllinen. Tämä ilmenee negatiivisena heilahduksena, jota kutsutaan MMN-vasteeksi (engl. mismatch negativity). MMN-vaste syntyy, vaikka koehenkilöt eivät tarkkailisi ääniärsykkeitä, ja tästä syystä sitä voidaan käyttää kuulojärjestelmässä automaattisesti tapahtuvan äänten käsittelyn tutkimiseen. Havaitsemme merkitystä yhdistämällä samankaltaisia ääniä ja erottamalla niitä erilaisista äänistä. Tätä kutsutaan ryhmittelyksi. Tämän väitöskirjan tulosten mukaan kuuloaineiston ryhmittelytoiminto on kehittyneempi muusikoilla kuin muilla. Väitöskirjassa myös näytetään, että muusikoiden löytää neljän äänen säännönmukaisu uden muille merkityksettöm ästä äänisarjasta. Tämä viittaa siihen, että osa kuuloprosesseista, joita käytetään musiikin jäsentämiseen, on automatisoitunut muusikoilla. Tämä on hyödyllistä prosessointiresurssien kannalta, sillä näin niitä jää korkeampien toimintojen käyttöön. Tämän väitöskirjan viimeisessä tutkimuksessa tarkasteltiin automaattisen poikkeavuuden käsittelyn ja tietoisen havaitsemisen suhdetta. Tutkimus osoitti, että MMN-vasteen syntyminen ei välttämättä satu yhteen tietoisen havainnon kanssa. Vaikka aivot löysivät satunnaiset muutokset ääniärsykkeissä, koehenkilöt eivät tietoisesti havainneet muutoksia. Ero sen välillä, mitä aivot käsittelivät ja mitä ihmiset tietoisella tasolla havaitsivat, osoittaa, että koehenkilöillä oli implisiittistä tietoa äänten rakenteesta. Kun koehenkilöitä sitten harjoitettiin kuulemaan muutoksia esittämällä yhtäaikaisesti visuaalinen vihje, puolet heistä oppi löytämään satunnaiset muutokset, pystymättä kuitenkaan kertomaan, miten ne erosivat muista äänistä. Toisin sanoen, he pystyivät löytämään poikkeamat intuitiivisesti. Tulokset osoittavat, että implisiittistä tietoa voidaan hyödyntää oikeiden intuitiivisten päätösten tekemisessä

Looking for a pattern: An MEG study on the abstract mismatch negativity in musicians and nonmusicians

<p>Abstract</p> <p>Background</p> <p>The mismatch negativity (MMN) is an early component of event-related potentials/fields, which can be observed in response to violations of regularities in sound sequences. The MMN can be elicited by simple feature (e.g. pitch) deviations in standard oddball paradigms as well as by violations of more complex sequential patterns. By means of magnetoencephalography (MEG) we investigated if a pattern MMN could be elicited based on global rather than local probabilities and if the underlying ability to integrate long sequences of tones is enhanced in musicians compared to nonmusicians.</p> <p>Results</p> <p>A pattern MMN was observed in response to violations of a predominant sequential pattern (AAAB) within a standard oddball tone sequence consisting of only two different tones. This pattern MMN was elicited even though the probability of pattern deviants in the sequence was as high as 0.5. Musicians showed more leftward-lateralized pattern MMN responses, which might be due to a stronger specialization of the ability to integrate information in a sequence of tones over a long time range.</p> <p>Conclusion</p> <p>The results indicate that auditory grouping and the probability distribution of possible patterns within a sequence influence the expectations about upcoming tones, and that the MMN might also be based on global statistical knowledge instead of a local memory trace. The results also show that auditory grouping based on sequential regularities can occur at a much slower presentation rate than previously presumed, and that probability distributions of possible patterns should be taken into account even for the construction of simple oddball sequences.</p

Processing of Complex Auditory Patterns in Musicians and Nonmusicians

In the present study we investigated the capacity of the memory store underlying the mismatch negativity (MMN) response in musicians and nonmusicians for complex tone patterns. While previous studies have focused either on the kind of information that can be encoded or on the decay of the memory trace over time, we studied capacity in terms of the length of tone sequences, i.e., the number of individual tones that can be fully encoded and maintained. By means of magnetoencephalography (MEG) we recorded MMN responses to deviant tones that could occur at any position of standard tone patterns composed of four, six or eight tones during passive, distracted listening. Whereas there was a reliable MMN response to deviant tones in the four-tone pattern in both musicians and nonmusicians, only some individuals showed MMN responses to the longer patterns. This finding of a reliable capacity of the short-term auditory store underlying the MMN response is in line with estimates of a three to five item capacity of the short-term memory trace from behavioural studies, although pitch and contour complexity covaried with sequence length, which might have led to an understatement of the reported capacity. Whereas there was a tendency for an enhancement of the pattern MMN in musicians compared to nonmusicians, a strong advantage for musicians could be shown in an accompanying behavioural task of detecting the deviants while attending to the stimuli for all pattern lengths, indicating that long-term musical training differentially affects the memory capacity of auditory short-term memory for complex tone patterns with and without attention. Also, a left-hemispheric lateralization of MMN responses in the six-tone pattern suggests that additional networks that help structuring the patterns in the temporal domain might be recruited for demanding auditory processing in the pitch domain

Predictability effects in auditory scene analysis: a review

Many sound sources emit signals in a predictable manner. The idea that predictability can be exploited to support the segregation of one source's signal emissions from the overlapping signals of other sources has been expressed for a long time. Yet experimental evidence for a strong role of predictability within auditory scene analysis (ASA) has been scarce. Recently, there has been an upsurge in experimental and theoretical work on this topic resulting from fundamental changes in our perspective on how the brain extracts predictability from series of sensory events. Based on effortless predictive processing in the auditory system, it becomes more plausible that predictability would be available as a cue for sound source decomposition. In the present contribution, empirical evidence for such a role of predictability in ASA will be reviewed. It will be shown that predictability affects ASA both when it is present in the sound source of interest (perceptual foreground) and when it is present in other sound sources that the listener wishes to ignore (perceptual background). First evidence pointing toward age-related impairments in the latter capacity will be addressed. Moreover, it will be illustrated how effects of predictability can be shown by means of objective listening tests as well as by subjective report procedures, with the latter approach typically exploiting the multi-stable nature of auditory perception. Critical aspects of study design will be delineated to ensure that predictability effects can be unambiguously interpreted. Possible mechanisms for a functional role of predictability within ASA will be discussed, and an analogy with the old-plus-new heuristic for grouping simultaneous acoustic signals will be suggested

Regularity extraction from non-adjacent sounds

The regular behavior of sound sources helps us to make sense of the auditory environment. Regular patterns may, for instance, convey information on the identity of a sound source (such as the acoustic signature of a train moving on the rails). Yet typically, this signature overlaps in time with signals emitted from other sound sources. It is generally assumed that auditory regularity extraction cannot operate upon this mixture of signals because it only finds regularities between adjacent sounds. In this view, the auditory environment would be grouped into separate entities by means of readily available acoustic cues such as separation in frequency and location. Regularity extraction processes would then operate upon the resulting groups. Our new experimental evidence challenges this view. We presented two interleaved sound sequences which overlapped in frequency range and shared all acoustic parameters. The sequences only differed in their underlying regular patterns. We inserted deviants into one of the sequences to probe whether the regularity was extracted. In the first experiment, we found that these deviants elicited the mismatch negativity (MMN) component. Thus the auditory system was able to find the regularity between the non-adjacent sounds. Regularity extraction was not influenced by sequence cohesiveness as manipulated by the relative duration of tones and silent inter-tone-intervals. In the second experiment, we showed that a regularity connecting non-adjacent sounds was discovered only when the intervening sequence also contained a regular pattern, but not when the intervening sounds were randomly varying. This suggests that separate regular patterns are available to the auditory system as a cue for identifying signals coming from distinct sound sources. Thus auditory regularity extraction is not necessarily confined to a processing stage after initial sound grouping, but may precede grouping when other acoustic cues are unavailable

Predictive coding in auditory perception: challenges and unresolved questions.

Predictive coding is arguably the currently dominant theoretical framework for the study of perception. It has been employed to explain important auditory perceptual phenomena, and it has inspired theoretical, experimental and computational modelling efforts aimed at describing how the auditory system parses the complex sound input into meaningful units (auditory scene analysis). These efforts have uncovered some vital questions, addressing which could help to further specify predictive coding and clarify some of its basic assumptions. The goal of the current review is to motivate these questions and show how unresolved issues in explaining some auditory phenomena lead to general questions of the theoretical framework. We focus on experimental and computational modelling issues related to sequential grouping in auditory scene analysis (auditory pattern detection and bistable perception), as we believe that this is the research topic where predictive coding has the highest potential for advancing our understanding. In addition to specific questions, our analysis led us to identify three more general questions that require further clarification: (1) What exactly is meant by prediction in predictive coding? (2) What governs which generative models make the predictions? and (3) What (if it exists) is the correlate of perceptual experience within the predictive coding framework

Electromagnetic Correlates of Musical Expertise in Processing of Tone Patterns

Using magnetoencephalography (MEG), we investigated the influence of long term musical training on the processing of partly imagined tone patterns (imagery condition) compared to the same perceived patterns (perceptual condition). The magnetic counterpart of the mismatch negativity (MMNm) was recorded and compared between musicians and non-musicians in order to assess the effect of musical training on the detection of deviants to tone patterns. The results indicated a clear MMNm in the perceptual condition as well as in a simple pitch oddball (control) condition in both groups. However, there was no significant mismatch response in either group in the imagery condition despite above chance behavioral performance in the task of detecting deviant tones. The latency and the laterality of the MMNm in the perceptual condition differed significantly between groups, with an earlier MMNm in musicians, especially in the left hemisphere. In contrast the MMNm amplitudes did not differ significantly between groups. The behavioral results revealed a clear effect of long-term musical training in both experimental conditions. The obtained results represent new evidence that the processing of tone patterns is faster and more strongly lateralized in musically trained subjects, which is consistent with other findings in different paradigms of enhanced auditory neural system functioning due to long-term musical training

Is neural and behavioral sound processing affected by practice strategies in musicians?

Previous exploratory studies suggest that pre-attentive auditory processing of musicians differ depending on the strategies they use in music practicing and performance. This study aimed at systematically determining whether there are differences in neural sound processing and behavioral measures between musicians preferring and not-preferring aural strategies including improvising, playing by ear and rehearsing by listening recordings. Participants were assigned into aural (n = 13) and non-aural (n = 11) groups according to how much they employ aural strategies, as determined by a questionnaire. The amplitude, latency, and scalp topography of the memory-related mismatch negativity (MMN) component of the event-related brain potentials were investigated with the so-called ‘optimal’ paradigm probing simple sound feature processing and with the ‘transposed-melody’ paradigm, probing complex sound pattern processing. Further, their behavioral accuracy in sound perception was tested with an attentive discrimination task in the transposed-melody paradigm and with the AMMA musicality test. Results showed that there were group differences both at the pre-attentive and behavioral levels of sound processing. First, in the optimal paradigm, the MMN morphology for the isolated sound features was similar between groups but its MMN amplitude, latency and topography for different sound features differed. Second, in the ‘transposed-melody’ paradigm, MMN was larger for the deviant that changed its contour as compared with the deviant that changed the last tone and thus the interval between the two last tones of the melody. The Contour-MMN amplitude as determined in the beginning of the recordings correlated with the subsequent behavioral discrimination accuracy in attentive condition. However, there were no group differences in the behavioral discrimination both deviants being detected equally well. The Interval-MMN amplitudes decreased especially in the aural group after the attentive condition. Moreover, the Interval-MMN latency in the non-aural group prolonged after the attentive condition as compared to the preceding condition whereas in the aural group the MMN latency shortened. No changes were seen in the Contour-MMN between conditions with either of the groups. Third, the non-aural group outperformed the aural group in the AMMA musicality test (Tonal subtest and Total scores). Additionally, AMMA scores (especially the Rhythm) correlated significantly with the Contour-MMN amplitudes after the attentive condition. Taken together, the present results suggest that practice strategies do not affect musicians' pre-attentive processing of simple sound features but might affect complex sound pattern processing. Complex sound pattern processing related also to the attentive behavioral performance in all musicians. While providing new insights into behavioral and neural differences between musicians preferring different practice strategies, results only partially support previous findings concerning discriminatory accuracy of violation within complex sound pattern learning.Aiempien tutkimusten mukaan muusikoiden käyttämät harjoittelustrategiat voivat vaikuttaa heidän esitietoiseen kuulotiedon käsittelyynsä. Tutkimuksen tavoitteena oli systemaattisesti selvittää, eroaako auraalisia ja ei-auraalisia (ts. korvakuuloon perustuvia) harjoittelustrategioita suosivien muusikoiden kuulotiedon käsittely aivotutkimusten ja behavioraalisten mittausten perusteella. Auraalisilla strategioilla tarkoitettiin harjoittelukäytäntöjä, jotka sisälsivät runsaasti improvisointia, korvakuulolta soittamista ja harjoittelua äänitteiden kuuntelun avulla. Osallistujat jaettiin auraaliseen (n=13) ja ei-auraaliseen (11) ryhmään sen perusteella kuinka paljon he kertoivat taustatietokyselylomakkeessa hyödyntävänsä auraalisia strategioita. Tutkimusmenetelminä käytettiin aivojen jännitevaste-rekisteröintejä (EEG:tä) poikkeavuusnegatiivisuusasetelmassa, AMMA-musikaalisuustestiä sekä tätä tutkimusta varten suunniteltua kyselylomaketta. EEG-kokeessa esitettiin yksinkertainen äänten piirteitä testaava optimiparadigma sekä monimutkaisten, transponoitujen, melodiakulkujen paradigma, jonka aikana testattiin myös kuulohavainnoinnin erottelutarkkuutta tarkkailutilanteessa. Erillisellä tutkimuskerralla tehtiin AMMA-musikaalisuustesti sekä täytettiin taustatietokyselylomake. Tulosten mukaan ryhmät erosivat toisistaan sekä esitietoisella että tietoisella kuulotiedonkäsittelyn tasolla. Optimiparadigmassa MMN ei eronnut amplitudin tai latenssin osalta ryhmien välillä, mutta sen amplitudi ja jakauma eri äänen poikkeamapiirteiden välillä vaihteli. Transponoitujen melodioiden paradigmassa MMN oli vahvempi kuviopoikkeavuudelle (jossa toiseksi viimeinen sävel vaihtui) verrattuna intervallipoikkeavuudelle (jossa viimeinen sävel vaihtui). Kuviopoikkeavuuden MMN ensimmäisessä ei-tarkkailutilanteessa korreloi merkitsevästi tarkkailutilanteen behavioraaliseen erottelutarkkuuteen. Sitä vastoin, behavioraalinen erottelutarkkuus ei eronnut ryhmien välillä ja molemmat poikkeavuudet eroteltiin yhtä tarkasti. Intervalli-MMN:n amplitudi pienentyi tarkkailutilanteen erityisesti auraalisella ryhmällä. Lisäksi intervalli-MMN:n latenssi kasvoi tarkkailutilanteen jälkeen ei-auraalisella ryhmällä, kun taas auraalisella ryhmällä se pieneni verrattuna tarkkailua edeltävään tilanteeseen. Kuvio-MMN ei eronnut tilanteiden välillä kummallakaan ryhmällä. AMMA-musikaalisuustestissä ei-auraalinen ryhmä suoriutui auraalista paremmin sävelkorkeuksien erottelukykyä testaavassa (’Tonal’) osiossa ja koko testissä. Lisäksi AMMA:n tulokset (erityisesti Rytmi-osio) korreloivat merkitsevästi tarkkailutilanteen jälkeiseen kuvio-MMN:n amplitudiin. Tulosten mukaan harjoittelustrategiat eivät näytä vaikuttavan muusikoiden esitietoiseen äänen piirteiden havaitsemiseen mutta saattavat vaikuttaa monimutkaisten melodiakulkujen havaitsemiseen. Monimutkaisten melodiakulkujen kuulotiedonkäsittely liittyi myös behavioraaliseen suoriutumiseen kaikilla muusikoilla. Tulokset tarjoavat uutta tietoa erilaisia harjoittelustrategioita suosivien muusikoiden välisistä eroista behavioraalisen ja neuraalisen tarkastelun tasoilla, mutta tukevat vain osittain aiempia aivotutkimuslöydöksiä poikkeavuuksien erottelutarkkuudesta monimutkaisten äänisarjojen oppimisen yhteydessä

Temporal boundary of auditory event formation : An electrophysiological marker

The formation of auditory events requires integration between successive sounds. There is a temporal limit below which a single sound event is perceived while above which a second perceptual event is formed. Behavioral studies applying the Temporal Order Judgment paradigm showed that this boundary is between 20 and 70 ms. Here we provide event-related potential (ERP) evidence from two experiments showing a qualitative change in the processing of tone pairs between 25 and 75 ms within-pair inter-stimulus intervals (ISI). We also show that this temporal boundary can be influenced by the immediate acoustical context, the statistical distribution of the ISIs within the sequence of tone-pairs.Peer reviewe