McGurk Doesn\u27t Work: Using EEG to Investigate the Time Course of the McGurk Effect

Speech perception is an inherently multimodal process, where auditory signals provide concurrent information to visual information from the speaker\u27s mouth movements. The McGurk effect is an illusion often used to study audiovisual speech integration; it is created by presenting incongruent auditory and visual speech cues. In the original study, listening to the spoken syllable /ba/ while watching visual mouth movements for /ga/ resulted in a \u27fusion\u27 perception of /da/ in 98% of participants (McGurk & MacDonald, 1976). Following, research on audiovisual integration has largely looked at the effect in isolated syllables. Our goal was to enhance the ecological validity of the McGurk effect by creating word stimuli mimicking everyday conversations (e.g., pairing audio for /beer/ with lip movements for /gear/ to determine listeners\u27 interpretation). In this study, we varied task (forced-choice vs. open-ended) and stimuli (words vs. non-words) between participants. In the word condition, all three stimuli formed words (e.g., beer, deer, gear) whereas in the non-word condition, either the B, D, or G stimuli was a word and the other two were nonwords (e.g., besk, desk, gesk). We found that fusion responses were much lower than in previous studies, potentially due to the use of full words. Importantly, participants also showed the most fusion responses when the D choice was a word with B and G forming non-words. We conclude that the McGurk effect occurs at a decision level rather than a perceptual level because task and stimulus differences influenced McGurk effect likelihood