The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect

The Colavita effect refers to the phenomenon that when confronted with an audiovisual stimulus, observers report more often to have perceived the visual than the auditory component. The Colavita effect depends on low-level stimulus factors such as spatial and temporal proximity between the unimodal signals. Here, we examined whether the Colavita effect is modulated by synesthetic congruency between visual size and auditory pitch. If the Colavita effect depends on synesthetic congruency, we expect a larger Colavita effect for synesthetically congruent size/pitch (large visual stimulus/low-pitched tone; small visual stimulus/high-pitched tone) than synesthetically incongruent (large visual stimulus/high-pitched tone; small visual stimulus/low-pitched tone) combinations. Participants had to identify stimulus type (visual, auditory or audiovisual). The study replicated the Colavita effect because participants reported more often the visual than auditory component of the audiovisual stimuli. Synesthetic congruency had, however, no effect on the magnitude of the Colavita effect. EEG recordings to congruent and incongruent audiovisual pairings showed a late frontal congruency effect at 400–550 ms and an occipitoparietal effect at 690–800 ms with neural sources in the anterior cingulate and premotor cortex for the 400- to 550-ms window and premotor cortex, inferior parietal lobule and the posterior middle temporal gyrus for the 690- to 800-ms window. The electrophysiological data show that synesthetic congruency was probably detected in a processing stage subsequent to the Colavita effect. We conclude that—in a modality detection task—the Colavita effect can be modulated by low-level structural factors but not by higher-order associations between auditory and visual inputs. Keywords Synesthetic congruency Audiovisual integration Colavita effect Event-related potential

Auditory dominance in motor-sensory temporal recalibration

Perception of synchrony between one’s own action (e.g. a finger tap) and the sensory feedback thereof (e.g. a flash or click) can be shifted after exposure to an induced delay (temporal recalibration effect, TRE). It remains elusive, however, whether the same mechanism underlies motor-visual (MV) and motor-auditory (MA) TRE. We examined this by measuring crosstalk between MV- and MA-delayed feedbacks. During an exposure phase, participants pressed a mouse at a constant pace while receiving visual or auditory feedback that was either delayed (+150 ms) or subjectively synchronous (+50 ms). During a post-test, participants then tried to tap in sync with visual or auditory pacers. TRE manifested itself as a compensatory shift in the tap–pacer asynchrony (a larger anticipation error after exposure to delayed feedback). In experiment 1, MA and MV feedback were either both synchronous (MV-sync and MA-sync) or both delayed (MV-delay and MA-delay), whereas in experiment 2, different delays were mixed across alternating trials (MV-sync and MA-delay or MV-delay and MA-sync). Exposure to consistent delays induced equally large TREs for auditory and visual pacers with similar build-up courses. However, with mixed delays, we found that synchronized sounds erased MV-TRE, but synchronized flashes did not erase MA-TRE. These results suggest that similar mechanisms underlie MA- and MV-TRE, but that auditory feedback is more potent than visual feedback to induce a rearrangement of motor-sensory timing

Measuring perception without introspection

Binocular rivalry, the perceptual alternation between incompatible monocular stimuli, is conventionally measured by asking the subject which percept is currently visible. This is problematic because the response is unverifiable, open to response bias, and falsely assumes that the perceptual experience is binary. We overcame these limitations in a new approach that does not require subjective reporting of perceptual state. A brief test stimulus was added to one eye's inducing stimulus at random times and contrasts. The test was presented at one of two spatial locations, the subject indicated which alternative had been shown, and the correctness of the response was recorded as a function of test contrast. Given the random timing of the test stimulus, it was sometimes delivered when the tested eye was dominant and, at other times, suppressed. Accordingly, the psychometric function recorded during rivalry should be a mixture of the dominance and suppression forms of the function. This was indeed the case: The probability of a correct response during rivalry was significantly less than that obtained with a binocularly congruent stimulus. The psychometric function during rivalry was well modeled as a weighted sum of the congruence curve with an assumed suppression curve. Optimal fitting provided estimates of both suppression depth and percept predominance that corresponded closely with estimates obtained with the conventional method. We have therefore characterized rivalry without the uncertainties introduced by the subject's perceptual report. This provides a model that may be applicable to the broader field of perceptual ambiguity

Audio-visual speech in noise perception in dyslexia

Individuals with developmental dyslexia (DD) may experience, besides reading problems, other speech-related processing deficits. Here, we examined the influence of visual articulatory information (lip-read speech) at various levels of background noise on auditory word recognition in children and adults with DD. We found that children with a documented history of DD have deficits in their ability to gain benefit from lip-read information that disambiguates noise-masked speech. We show with another group of adult individuals with DD that these deficits persist into adulthood. These deficits could not be attributed to impairments in unisensory auditory word recognition. Rather, the results indicate a specific deficit in audio-visual speech processing and suggest that impaired multisensory integration might be an important aspect of DD

Audio-visual speech in noise perception in dyslexia

FSW - Self-regulation models for health behavior and psychopathology - ou