Multimedia: How to Combine Language and Visuals

In the last decade, advanced computer technology has allowed for development of information systems and learning environments that combine language with other forms of human communication in innovative ways. Language in the form of written texts, for example, can be combined not only with static pictures or graphs as in printed material, but also with animation or video

Regulating distance to the screen while engaging in difficult tasks

Regulation of distance to the screen (i.e., head-to-screen distance, fluctuation of head-to-screen distance) has been proved to reflect the cognitive engagement of the reader. However, it is still not clear (a) whether regulation of distance to the screen can be a potential parameter to infer high cognitive load and (b) whether it can predict the upcoming answer accuracy. Configuring tablets or other learning devices in a way that distance to the screen can be analyzed by the learning software is in close reach. The software might use the measure as a person-specific indicator of need for extra scaffolding. In order to better gauge this potential, we analyzed eye-tracking data of children (N = 144, Mage = 13 years, SD = 3.2 years) engaging in multimedia learning, as distance to the screen is estimated as a by-product of eye tracking. Children were told to maintain a still seated posture while reading and answering questions at three difficulty levels (i.e., easy vs. medium vs. difficult). Results yielded that task difficulty influences how well the distance to the screen can be regulated, supporting that regulation of distance to the screen is a promising measure. Closer head-to-screen distance and larger fluctuation of head-to-screen distance can reflect that participants are engaging in a challenging task. Only large fluctuation of head-to-screen distance can predict the future incorrect answers. The link between distance to the screen and processing of cognitive task can obtrusively embody reader’s cognitive states during system usage, which can support adaptive learning and testing

Multimedia: How to Combine Language and Visuals

In the last decade, advanced computer technology has allowed for development of information systems and learning environments that combine language with other forms of human communication in innovative ways. Language in the form of written texts, for example, can be combined not only with static pictures or graphs as in printed material, but also with animation or video

Multimedia: How to Combine Language and Visuals

In the last decade, advanced computer technology has allowed for development of information systems and learning environments that combine language with other forms of human communication in innovative ways. Language in the form of written texts, for example, can be combined not only with static pictures or graphs as in printed material, but also with animation or video

Eye Tracking Indicators of Reading Approaches in Text-Picture Comprehension

Despite numerous researches on reading as well as multimedia comprehension, reading approaches in text-picture comprehension have become a focus of research only rarely. The current experiment aims at exploring text-picture comprehension in different reading approaches with item solving. In a within subjects design using eye tracking, seventeen secondary school students processed our blended text and picture materials in three different ways. (1) Unguided processing with text and picture and without the question. (2) Information gathering to answer the question after the prior experience with text and pictures. (3) Comprehending text and pictures to solve the task with the prior information of the question. Eye tracking data showed that text and picture play different roles in comprehension in different reading approaches. The data suggest that (1) text plays the main role to construct the mental model in unguided spontaneous processing of text and picture. (2) Students seem to mainly rely on the picture as an external representation when trying to answer a question after the prior experience with the material. (3)Text and picture are both used heavily when students work out an answer with the prior experience with the question presented. The text likely plays a major part in guiding the processing of meaning, whereas the picture is used as an external representation for information retrieval. Our work provides a first step towards an Item Solving Model in Text-Picture Comprehension. It also provides pedagogical implications for learning in secondary school

The mechanisms of “incidental news consumption”: An eye tracking study of news interaction on Facebook

This exploratory study examines how participants incidentally consumed news on social media through an eye tracking analysis of their visual interaction with posts on Facebook. By interaction, we refer to the attention participants gave to news (measured through the time devoted to looking at the content); how they read these news items (measured through ocular movements on the screen); and the way they engaged with this content (measured through forms of participation such as liking, commenting, or sharing news). The data were triangulated through interviews with Facebook users and an analysis of the metrics of posts from Costa Rican news organizations on Facebook from 2017 to 2020. We draw on scholarship in communication studies and multimodal discourse analysis. We argue for a more nuanced approach to what study participants did when they incidentally encountered news on social media that focuses on mechanisms, that is, the specific procedures and operations that shape user interaction with news on Facebook (such as visual fixations on parts of news posts; the visual entry points through which they begin to interact with the news; the sequences that characterize how they navigate content; and the time they spend assessing various multimodal elements).Universidad de Costa Rica/[]/UCR/Costa RicaUCR::Vicerrectoría de Investigación::Unidades de Investigación::Ciencias Sociales::Centro de Investigación en Comunicación (CICOM)UCR::Vicerrectoría de Docencia::Artes y Letras::Facultad de Letras::Escuela de Filología, Lingüística y Literatur