Crossmodal prep and mod comp

Two experiments examined the influence of cue-based preparation on modality compatibility effects in task switching

New insights into modality-compatibility effects in task switching

Daily life quite frequently involves multitasking. We have to balance between different activities by shifting our attention between them or by simultaneously engaging in them according to our goal. In such a situation, we are exposed to a multimodal world. We have to perceive and process stimuli in different modalities and there are several possibilities for the modalities in which responses may be given. A large body of research is aimed at understanding the costs stemming from multitasking; however, in recent times, the fact that multitasking costs depend, at least partly, on sensory-motor modalities is receiving relatively more attention. The present dissertation is based on the previous findings of Stephan and Koch (2010, 2011, 2016), who implemented the task-switching paradigm wherein participants switched between an auditory-vocal (AV) and a visual-manual (VM) task in one condition and an auditory-manual (AM) and a visual-vocal (VV) task in another condition. Stephan and Koch (2010, 2011, 2016) observed larger switch costs with the AM-VV condition than with the AV-VM condition, and based on their results, they proposed the modality-compatibility account. Modality compatibility refers to the similarity between the stimulus modality and the modality of the anticipated sensory-response effects. For instance vocal responses produce auditory effects. It is assumed that there is a special linkage between sensory-motor modalities that comes from ideomotor (see Greenwald 1972) response-effect learning that leads to a tendency to bind the response modality to the most salient stimulus modality. Therefore, switching between modality-compatible tasks, the anticipated response-effect and the stimulus induced activation prime each other, whereas modality-incompatible tasks evoke between-task crosstalk due to a priming of the competing task. The aim of the present work is to investigate the modality-compatibility effects in task-switching from three research perspectives: structure, flexibility and plasticity. In the structural perspective, in Study I, we investigated the possible sensory-motor modality mapping combinations between stimuli in visual and auditory modalities, and responses in manual and vocal modalities in order to explore the structural boundaries of the modality-compatibility effects. We found that modality compatibility influences switch costs only when there is a variability between the stimulus and the response modality, but there are no relevant differences across modality compatibility when either the stimulus or the response modality is kept constant. Following the structural point of view, Study II indicated for the first time that modality compatibility also influences performance in a free choice situation by leading to a bias to produce modality compatible mappings. Importantly, even though the choices of response modalities were endogenous in Study II, modality compatibility still influenced switch costs as we have previously observed with forced-choice tasks. With regard to the flexibility perspective, in Study III, we conducted two experiments in order to explore the role of advance task preparation on modality-compatibility effects by introducing explicit cues. On the one hand, in both experiments, we found generally reduced modality-compatibility effects on switch costs, indicating that modality compatibility can be prepared with explicit cues and with sufficient time. On the other hand, the non-significant interaction between modality-compatibility effects on switch costs and the preparation effects suggests different underlying mechanisms. Finally, in the plasticity perspective, we introduced single-task practice in order to study the short-term practice effects on modality compatibility. In Study IV, we found a practice-related modulation of switch costs. After participants practiced modality compatible single-tasks, we observed that modality compatibility influenced switch costs; however, when participants practiced modality-incompatible single-tasks, modality compatibility had no relevant effect on switch costs. The findings of Study IV indicate that short-term associations by the practiced modality incompatible mappings could override the long-term modality-compatibility effects. In these four studies, we investigated modality-compatibility effects in task-switching based on the previous work of Stephan and Koch (2010, 2011, 2016). We explored the structural boundaries of modality-compatibility effects and found that modality compatibility represents compatibility across mappings and not within tasks at the level of stimulus or response modalities. The finding that using explicit cues reduces the modality-compatibility effects in the first place and the possibly different mechanisms underlying modality compatibility effects and active preparation facilitate to take into account processes that are needed to perform a task with several modalities. Finally, the reduced between-task crosstalk as a result of implementing short-term single-task practice is promising and provides an interesting perspective not only for research on modality compatibility but also for research investigating practice effects