Reading a story. Different degrees of learning in different learning environments

he learning environment in which material is acquired may produce differences in delayed recall and in the elements that individuals focus on. These differences may appear even during development. In the present study, we compared three different learning environments in 450 normally developing 7-year-old children subdivided into three groups according to the type of learning environment. Specifically, children were asked to learn the same material shown in three different learning environments: reading illustrated books (TB); interacting with the same text displayed on a PC monitor and enriched with interactive activities (PC-IA); reading the same text on a PC monitor but not enriched with interactive narratives (PC-NoIA). Our results demonstrated that TB and PC-NoIA elicited better verbal memory recall. In contrast, PC-IA and PCNoIA produced higher scores for visuo-spatial memory, enhancing memory for spatial relations, positions and colors with respect to TB. Interestingly, only TB seemed to produce a deeper comprehension of the story's moral. Our results indicated that PC-IA offered a different type of learning that favored visual details. In this sense, interactive activities demonstrate certain limitations, probably due to information overabundance, emotional mobilization, emphasis on images and effort exerted in interactive activities. Thus, interactive activities, although entertaining, act as disruptive elements which interfere with verbal memory and deep moral comprehensio

The closer, the better? Processing relations between picture elements in historical paintings

The present eye-tracking study investigated how audio explanations influence perception and the cognitive processing of historical paintings. Spatially close and distant pairs of picture elements and their semantic relations were named in an audio text either immediately after each other or with descriptions of other elements in between. It was assumed that the number of backward fixation counts on the first of the two mentioned related picture elements should be higher if they are spatially close rather than spatially distant. There should also be more backward fixation counts if the elements are named temporally close rather than temporally distant. Similar predictions were made for the retention of these picture elements and their relations. A 2x2x2 within-subject design (n=36) with spatial distance (close vs. distant), temporal distance (close vs. distant) and painting (Leutze vs. West) revealed more background fixation counts for spatially close compared to spatially distant elements but just for the Leutze painting. Accordingly, the relations between the spatially close pairs were retained better than between the spatially distant pairs in the Leutze painting but vice versa for the West painting. The results are discussed with regard to the spatial contiguity principle of multimedia learning and research on text coherence

Processing and learning from multiple sources: a comparative case study of students with dyslexia working in a multiple source multimedia context

This study investigated how four 10th-grade students with dyslexia processed and integrated information across web pages and representations when learning in a multiple source multimedia context. Eye movement data showed that participants' processing of the materials varied with respect to their initial exploration of the web pages, their overall processing time, and the linearity of their processing patterns, with post-learning interviews indicating the deliberate, strategic considerations underlying each participant's processing pattern. Eye movement data in terms of fixation duration and percentage of regressions also corroborated the findings of formal, diagnostic assessments. Finally, it was found that participants differed with respect to how much factual information they learned from working with the materials and how well they were able to integrate information across the web pages and representations, with results suggesting particular problems with learning factual information and, at the same time,constructing a coherent mental representation of the issue, as well as with drawing on textual information in the integration process. This study brings together two research areas that essentially have been kept apart in theory and research, that is, dyslexia and multimedia learning, and it provides unique information about the role of individual differences in multiple source multimedia contexts

Integrative Processing of Verbal and Graphical Information During Re-reading Predicts Learning from Illustrated Text: An Eye-Movement Study

Printed or digital textbooks contain texts accompanied by various kinds of visualisation. Successful comprehension of these materials requires integrating verbal and graphical information. This study investigates the time course of processing an illustrated text through eye-tracking methodology in the school context. The aims were to identify patterns of first- and second-pass reading and to examine whether the integrative processing of text and picture during the less automatic and more purposeful second-pass reading predicts learning, after controlling for reading comprehension, prior knowledge, and self-concept. Forty-three 7th graders read an illustrated science text while their eye-movements were recorded. A cluster analysis revealed two processing patterns during the first-pass reading, which differed for the time spent on the main concepts in the text and picture. During re-reading, two patterns of stronger and weaker integrative processing emerged. Integration of verbal and graphical information was revealed by the frequency of second-pass transitions from text to picture and from picture to text, and the duration of picture re-inspecting while re-reading text information (look-from text to picture) and re-reading text information while re-inspecting the visualised information (look-from picture to text). A series of hierarchical regression analyses indicated that only the patterns of integrative processing during the second-pass reading uniquely predict verbal and graphical recalls, and the transfer of knowledge. The study provides evidence that the delayed processing which integrates text and graphics contributes to text retention and the application of newly learned knowledge, over and above individual characteristics. The educational significance is outlined

Pun processing in advertising posters: evidence from eye tracking

This study examines the process of reading polycode advertising posters, focusing in particular on the effect of a pun in the headline. The pun, or a sequence of lexical items that can be perceived as ambiguous, is contained in the headline and different meanings of this sequence are supported by the picture and text. The results of the preliminary experiment showed that advertisements with puns are rated as more attractive, original, effective and positive compared to advertisements without puns. We hypothesized that puns in the headlines increase cognitive effort in processing posters, leading to higher evaluations. The main experiment tested this and examined differences in eye movement when reading posters with and without puns. Fifty-five Russian participants viewed advertisements while their eye movements were recorded. Our results showed no fundamental differences in the general pattern of viewing advertisement posters with and without puns. We found that readers start to perceive polycode advertisements from the text and spend more time reading the text than looking at an image. These findings shed light on how attention is distributed between verbal and non-verbal components of polycode texts, and which type of poster is more effective for information retrieval at different processing levels

Deaf and Hard of Hearing Readers and Science Comics: A Mixed Methods Investigation on Process

Deaf and hard of hearing (DHH) students bring diverse language and literacy backgrounds to the task of academic reading, which becomes increasingly complex and abstract in the upper grades. Teachers often differentiate their instruction by providing multimedia resources, of which students interact with verbal and pictorial information. A growing body of research supports multimedia learning; however, most of the studies have focused exclusively on learning outcomes, leaving teachers in the dark about the cognitive processes underlying these effects. This mixed methods study addresses this gap by using a nonfiction comic to investigate the reading processes of DHH 7th -12th grade students. Eye tracking and cued retrospective protocol were employed in a concurrent nested design to answer the question, how do DHH students read and learn from multimedia science texts? This study was guided by the cognitive theory of multimedia which states that reading comprehension is better supported when learning from words and pictures rather than words alone, especially when readers cognitively integrate the two representations to form a coherent mental model. Temporal and sequential eye tracking results revealed that readers’ transitions between related words and pictures were a statistically significant variable in explaining factual knowledge learning outcomes. These strategic shifts in attention were further explained by readers’ retrospective verbal reports of their thinking. Students’ descriptions of their vii reading processes were interpreted into the following themes: repairing, connecting representations, passive transitions, and connecting to self. The integration of quantitative and qualitative methods at the interpretation stage revealed that although the theme of repairing was equally distributed across all student reports, the theme of connecting representations was largely present in the reports from students who made high counts of integrative transitions. The major findings of this study align with the cognitive theory of multimedia, that students’ learning outcomes were significantly predicted by the deliberate strategies to cognitively integrate words and pictures to form and maintain a coherent mental model. The discussion includes ways in which teachers can capitalize on explicit modeling of these behaviors and employ students’ “think alouds” to better understand and support the development of effective multimedia reading processes

Gaze transitions when learning with multimedia

Eye tracking methodology is used to examine the influence of interactive multimedia on the allocation of visual attention and its dynamics during learning. We hypothesized that an interactive simulation promotes more organized switching of attention between different elements of multimedia learning material, e.g., textual description and pictorial visualization. Participants studied a description of an algorithm accompanied either by an interactive simulation, self-paced animation, or static illustration. Using a novel framework for entropy-based comparison of gaze transition matrices, results showed that the interactive simulation elicited more careful visual investigation of the learning material as well as reading of the problem description through to its completion

Should learners use their hands for learning? Results from an eye‐tracking study

Given the widespread use of touch screen devices, the effect of the users' fingers on information processing and learning is of growing interest. The present study drew on cognitive load theory and embodied cognition perspectives to investigate the effects of pointing and tracing gestures on the surface of a multimedia learning instruction. Learning performance, cognitive load and visual attention were examined in a one‐factorial experimental design with the between‐subject factor pointing and tracing gestures. The pointing and tracing group were instructed to use their fingers during the learning phase to make connections between corresponding text and picture information, whereas the control group was instructed not to use their hands for learning. The results showed a beneficial effect of pointing and tracing gestures on learning performance, a significant shift in visual attention and deeper processing of information by the pointing and tracing group, but no effect on subjective ratings of cognitive load. Implications for future research and practice are discussed

Effects of Illustrations on Retention and Visual Attention Using Authentic Textbooks

Problem The purpose of this exploratory study was to validate a model of multimedia learning, incorporating learner-specific characteristics such as intelligence, memory, and measures of visual attention. Method The sample consisted of 62 Andrews University students (26 males, 36 females, and mean age 21.7). Data were gathered by means of standardized testing (RPM, WAIS- III) and eye-tracking. MLR was used to determine significant visual attention predictors for retention and SEM was used to test a hypothesized model of multimedia learning. Results Multiple Linear Regression was significant (p.05; GFI = .923; CFI = .986; RMSEA = .043; PGFI = .451) proving the hypothesized model’s excellent fit to the data. Conclusions This exploratory study indicates that there are different learning strategies in a college population that are not related to characteristics such as intelligence or working memory. These strategies are learned or acquired and have thus clear implications for practice

Multimodal reading and second language learning

Most of the texts that second language learners engage with include both text (written and/or spoken) and images. The use of images accompanying texts is believed to support reading comprehension and facilitate learning. Despite their widespread use, very little is known about how the presentation of multiple input sources affects the attentional demands and the underlying cognitive processes involved. This paper provides a review of research on multimodal reading, with a focus on attentional processing. It first introduces the relevant theoretical frameworks and empirical evidence provided in support of the use of pictures in reading. It then reviews studies that have looked at the processing of text and pictures in first and second language contexts. Based on this review, main gaps in research and future research directions are identified. The discussion provided in this paper aims at advancing research on multimodal reading in a second language. Achieving a better understanding of the underlying cognitive processes in multimodal reading is crucial to inform pedagogical practices and to develop theoretical accounts of second language multimodal reading