Andersen, Tobias

Dau, Torsten

Gil Carvajal, Juan Camilo

Humans can integrate auditory and visual information when perceiving speech. This isevident in the McGurk effect, in which a presentation of e.g. auditory /aba/ and visual /aga/leads to the audiovisually fused percept /ada/. With the pairing of auditory /aga/ and visual/aba/, however, the illusion takes the form of a combination percept of either /abga/ or /agba/.Here, we investigated how audiovisual timing influences the perceived order of theconsonants in the McGurk combination. Stimuli were recorded with the consonants /g/ and/b/ using vowel-consonant-vowel (VCV) utterances with two syllabic contexts. First, the“internal timing” was studied by articulating the consonant to either emphasize the closingphase (VC-V) or the opening phase (V-CV). This produced cross-modally asynchronousconsonants while maintaining synchrony of the vowels. Auditory /ag_a/ dubbed onto visual/a_ba/ was mostly heard as /agba/ whereas auditory /a_ga/ dubbed onto visual /ab_a/ wasmostly heard as /abga/. Hence, syllabic context largely determined the perceived consonantorder. Second, the effect of audiovisual stimulus onset asynchrony (SOA) was examined atfive different SOAs, ranging from 200 ms auditory lead to 200 ms visual lead. The resultsshowed no effect on the perceived consonant order but audiovisual SOAs influenced thestrength of the illusion. Furthermore, we found that the window of integration is highlyasymmetric for combination illusions and that the direction of the asymmetry depends onthe perceived consonant order. We interpret the results as indicative of feature basedaudiovisual integration where formant transitions and aspirations are integrated separately

Online Research Database In Technology

  General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.   Users may download and print one copy of any publication from the public portal for the purpose of private study or research.  You may not further distribute the material or use it for any profit-making activity or commercial gain  You may freely distribute the URL identifying the publication in the public portal  If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.         Downloaded from orbit.dtu.dk on: Mar 30, 2019Consonant-Order Reversals in the McGurk Combination IllusionGil Carvajal, Juan Camilo; Dau, Torsten; Andersen, TobiasPublication date:2018Document VersionPublisher's PDF, also known as Version of recordLink back to DTU OrbitCitation (APA):Gil Carvajal, J. C., Dau, T., & Andersen, T. (2018). Consonant-Order Reversals in the McGurk CombinationIllusion. Abstract from 19th Annual International Multisensory Research Forum, Toronto, Canada. P2.40 Consonant-Order Reversals in the McGurk Combination Illusion Gil-Carvajal, J. C., Dau, T. & Andersen, T. Cognitive Systems, Department of Applied Mathematics and Computer Science, Technical University of Denmark, Richard Petersens Plads, 2800, Kgs. Lyngby, Denmark  Humans can integrate auditory and visual information when perceiving speech. This is evident in the McGurk effect, in which a presentation of e.g. auditory /aba/ and visual /aga/ leads to the audiovisually fused percept /ada/. With the pairing of auditory /aga/ and visual /aba/, however, the illusion takes the form of a combination percept of either /abga/ or /agba/. Here, we investigated how audiovisual timing influences the perceived order of the consonants in the McGurk combination. Stimuli were recorded with the consonants /g/ and /b/ using vowel-consonant-vowel (VCV) utterances with two syllabic contexts. First, the “internal timing” was studied by articulating the consonant to either emphasize the closing phase (VC-V) or the opening phase (V-CV). This produced cross-modally asynchronous consonants while maintaining synchrony of the vowels. Auditory /ag_a/ dubbed onto visual /a_ba/ was mostly heard as /agba/ whereas auditory /a_ga/ dubbed onto visual /ab_a/ was mostly heard as /abga/. Hence, syllabic context largely determined the perceived consonant order. Second, the effect of audiovisual stimulus onset asynchrony (SOA) was examined at five different SOAs, ranging from 200 ms auditory lead to 200 ms visual lead. The results showed no effect on the perceived consonant order but audiovisual SOAs influenced the strength of the illusion. Furthermore, we found that the window of integration is highly asymmetric for combination illusions and that the direction of the asymmetry depends on the perceived consonant order. We interpret the results as indicative of feature based audiovisual integration where formant transitions and aspirations are integrated separately.  Acknowledgments: This work was supported by the Oticon Centre of Excellence for Hearing and Speech Sciences (CHeSS) and by the Technical University of Denmark.   	

Consonant-Order Reversals in the McGurk Combination Illusion

http://orbit.dtu.dk/ws/files/163086235/Poster2.40_Abstract_IMRF2018.pdf

Consonant-Order Reversals in the McGurk Combination Illusion

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology