Learning from multimedia: the locus of modality effects

Research in educational psychology has focused on facilitating learning by using two presentation modalities (auditory and visual) to convey information. Learning is theorized to improve through an increase in perceptual information flow. I hypothesized that presenting information in two modalities might also provide additional benefits that occur after information is perceived, and while it is being processed for learning. The present study explored whether perceptual effects and cognitive effects of multimedia presentation can be separated by presenting auditory and visual information sequentially or simultaneously. During simultaneous presentation, the typical multimedia effect (that is, facilitating learning by presenting information in two modalities) did not occur, suggesting that the multimedia effect might depend upon more than perceptual effects. Moreover, the manipulation showed significant effects of presentation type during sequential presentation, suggesting that effects previously thought to be a result of reducing perceptual overlap might actually occur after perception. Based on the results of this study, I recommend that information designers reconsider the sources assumed to influence the multimedia learning effect. This would have implications for determining the optimal presentation of information.M.S.Committee Chair: Catrambone, Richard; Committee Member: Rogers, Wendy; Committee Member: Walker, Bruc

Two stage process model of learning from multimedia: guidelines for design

Theories of learning from multimedia suggest that when media include two modal forms (e.g., visual and auditory), learning is improved by activating modally segregated working memory subsystems, thereby expanding the total cognitive resource available for learning (Mayer, 2001; Sweller, 1999). However, a recent meta-analysis suggests that the typical modality effect (use of narrations and diagrams [i.e., multimodal] leads to better learning than use of text and diagrams [i.e., unimodal]) might be limited to situations in which presentations are matched to the time it takes for the narration to play (Ginns, 2005). This caveat can be accounted for by the differences in ways that people process unimodal and multimodal information, but not by the expansion of working memory explanation for modality effects (Tabbers, 2002). In this paper, I propose a framework for conceptualizing how people interact with multimedia instructional materials. According to this approach, learning from multimedia requires (1) creating mental codes to represent to-be-learned information and (2) forming a network of associations among these mental codes to characterize how this information is related. The present research confirms, in two between-subjects experiments, predictions from this model when presentation pace and verbal presentation modality are manipulated to accompany static (Experiment 1) and animated (Experiment 2) diagrams. That is, the data suggest that learning from unimodal presentations improved as presentation pace was slowed, whereas learning from multimodal presentations did not change as presentation pace was slowed. A third experiment also confirmed predicted patterns of eye movement behavior, demonstrating patterns of increasing dwell time on pictures and switches between media as pace was slowed for unimodal presentations but not multimodal presentations. It is concluded that the parallel patterns of learning outcomes and eye-movement behavior support the proposed model and are not predicted by other models of learning from multimedia instructions. This improvement in predictions of the effects of manipulating design elements (e.g., presentation pace and verbal presentation modality) on learning can help designers as they consider what combination of resources (e.g., classroom time or equipment for multimodal presentation) to devote to instructional design.Ph.D.Committee Chair: Richard Catrambone; Committee Member: James Foley; Committee Member: Mark Guzdial; Committee Member: Paul Corballis; Committee Member: Wendy Roger