Sensory uncertainty governs the extent of audio-visual interaction

Abstract

NoAuditory signals have been shown to exert a marked influence on visual perception in a wide range of tasks. However, the mechanisms of these interactions are, at present, poorly understood. Here we present a series of experiments where a temporal cue within the auditory domain can significantly affect the localisation of a moving visual target. To investigate the mechanism of this interaction, we first modulated the spatial positional uncertainty of the visual target by varying its size. When visual positional uncertainty was low (small target size), auditory signals had little or no influence on perceived visual location. However, with increasing visual uncertainty (larger target sizes), auditory signals exerted a significantly greater influence on perceived visual location. We then altered the temporal profile of the auditory signal by modulating the spread of its Gaussian temporal envelope. Introducing this temporal uncertainty to the auditory signal greatly reduced its effect on visual localisation judgements. These findings support the view that the relative uncertainty in individual sensory domains governs the perceptual outcome of multisensory integration