Reviving Static Charts into Live Charts

Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience

Auditory Experiences in Game Transfer Phenomena:

This study investigated gamers’ auditory experiences as after effects of playing. This was done by classifying, quantifying, and analysing 192 experiences from 155 gamers collected from online videogame forums. The gamers’ experiences were classified as: (i) auditory imagery (e.g., constantly hearing the music from the game), (ii) inner speech (e.g., completing phrases in the mind), (iii) auditory misperceptions (e.g., confusing real life sounds with videogame sounds), and (iv) multisensorial auditory experiences (e.g., hearing music while involuntary moving the fingers). Gamers heard auditory cues from the game in their heads, in their ears, but also coming from external sources. Occasionally, the vividness of the sound evoked thoughts and emotions that resulted in behaviours and copying strategies. The psychosocial implications of the gamers’ auditory experiences are discussed. This study contributes to the understanding of the effects of auditory features in videogames, and to the phenomenology of non-volitional experiences (e.g., auditory imagery, auditory hallucinations)

Exploration of concise redundancy in online multimedia learning

With the rapid growth of multimedia in education, the importance of investigating the effect of redundancy, repeating instructional messages to enhance conceptualization in instructional material design, is becoming more important. Various studies have been conducted recently regarding the effects of different forms of redundancy. A multimedia lesson presenting concurrent on-screen text, still graphics or animations, and narration is a typical setting in redundancy research. Concise redundancy is the revision of the on-screen text into a concise form which is presented to the learners concurrently with visualizations and narration. The purpose of this study was to investigate, while controlling for spatial ability, the effects of concise redundancy on students\u27 retention and confidence when learning with highly complex multimedia materials. In addition, the effects of animation or still graphics along with text redundancy were examined. No significant differences were found between the graphic presentations (animation or series of stills) and text redundancy groups (full, concise, or none) on retention or levels of confidence. When examining the results taking into account high and low spatial abilities, no significant differences were found in terms of different graphic presentation (animation or series of stills) and different text redundancy groups (full, concise, or none). However, in one condition, low spatial ability learners exhibited significantly higher levels of confidence than high spatial ability learners when learning with narrated static graphics and concise redundancy. The current study should provide further guidance for researchers who are interested in examining narrated multimedia lessons containing concise redundancy when comparing static graphics to animated graphics

Effective Use of Multimedia Presentations to Maximize Learning Within High School Classrooms

This research used an evidenced-based experimental 2 x 2 factorial design General Linear Model with Repeated Measures Analysis of Covariance (RMANCOVA). For this analysis, time served as the within-subjects factor while treatment group (i.e., static and signaling, dynamic and signaling, static without signaling, and dynamic without signaling) served as the between-subject independent variable. Three dependent variables were used to assess learner outcomes: (a) a 14 multiple-choice pre and post-test to measure knowledge retention, (b) a pre and post-test concept map to measure synthesis and structure of knowledge, and (c) four questions based on a Likert scale asking students to rank the cognitive difficulty of understanding four aspects of the animation they engaged in. A mental rotations test was used in the pretest conditions to establish a control and used as a covariate. The treatment contained a four minute and 53 second animation that served as an introductory multimedia presentation explaining the gravitational effects of the moon and sun on the earth. These interactions occur at predictable times and are responsible for creating the tidal effects experienced on Earth. There were 99 volunteer high school participants enrolled in science classes randomly assigned to one of four treatment conditions. The research was conducted to determine how motion and the principle of signaling, established in The Cognitive Theory of Multimedia Learning affected precollege learners. The experiment controlled for modality, segmenting, temporal contiguity, redundancy, and navigational control. Results of the RMANCOVA indicated statistical significance for the within subjects effect: over time for all participants, with time and knowledge retention measured from the multiple-choice results, and in the category quality of concepts represented in the concept map analysis. However, there were no significant differences in the between groups analysis for knowledge retention based on the multiple-choice assessment, or among groups over time in the concept map variables number of concepts, levels, and quality of concepts. Additionally, when measuring cognitive difficulty when learning from the animations, no significant differences were measured

Theory-driven Visual Design to Support Reflective Dietary Practice via mHealth: A Design Science Approach

Design for reflection in human-computer interaction (HCI) has evolved from focusing on an abstract and outcome-driven design subject towards exposing procedural or structural reflection characteristics. Although HCI research has recognized that an individual\u27s reflection is a long-lasting, multi-layered process that can be supported by meaningful design, researchers have made few efforts to derive insights from a theoretical perspective about appropriate translation into end-user visual means. Therefore, we synthesize theoretical knowledge from reflective practice and learning and argue for a differentiation between time contexts of reflection that design needs to address differently. In an interdisciplinary design-science-research project in the mHealth nutrition promotion context, we developed theory-driven guidelines for “reflection-in-action” and “reflection-on-action”. Our final design guidelines emerged from prior demonstrations and a final utility evaluation with mockup artifacts in a laboratory experiment with 64 users. Our iterative design and the resulting design guidelines offer assistance for addressing reflection design by answering reflective practice’s respective contextual requirements. Based on our user study, we show that reflection in terms of “reflection- in-action” benefits from offering actionable choice criteria in an instant timeframe, while “reflection-on-action” profits from the structured classification of behavior-related criteria from a longer, still memorable timeframe

The Morphic Orator: Transmogrified Delivery on the Audio-Enabled Web

Audio is an effective but often overlooked component of World Wide Web delivery. Of the nearly twenty billion web pages estimated to exist, statistically few use sound. Those few using sound often use it poorly and with hardly any regard to theoretical and rhetorical issues. This thesis is an examination of the uses of audio on the World Wide Web, specifically focusing on how that use could be informed by current and historical rhetorical theory. A theoretical methodology is applied to suggest the concepts and disciplines required to make online audio more meaningful and useful. The thesis argues for the connection between the Web and the modern orator, its embodiment, its place in sound reproduction technology, and awareness of the limitations placed on it by design and convention

Interactive applications and rhetorical devices for guiding parent-clinician communication through data visualizations

Effective communication between clinicians and parents of young children can decrease parents' anxiety and discomfort, help them handle bad news and uncertainty, and improve their adherence to proposed interventions. Parent-clinician communication further has the potential to facilitate collaboration and increase parents' empowerment. However when communication involves a discussion of the child's developmental delay or challenging behaviors, parents experience an emotional strain as they discuss hopes and fears, developmental concerns, and feelings of distress. As a consequence, communication challenges may emerge such as denial and the parent's resistance against the information that the clinician presents. In addition to the emotional strain, parents also experience a cognitive burden due to medical jargon or presentation of data that is inaccessible to them. In fact, in most health care settings, parents reported their expectation of more accessible information than is currently provided. In order to address these challenges, I present data visualization as a method of facilitating parent-clinician communication. This dissertation covers the cognitive perception and the practical application of data visualization in parent-clinician communication through: (1) rhetorical devices that are used to guide people's understanding of data visualizations, and (2) interactive applications I have built that explore the role of data visualizations in clinical communication. Through exploring cognitive and practical aspects of visualizations in communication, this dissertation makes three contributions. First, I showcase three interactive webtools that involve visualizations, and demonstrate that visualizations can facilitate family-clinician communication through overcoming 1) the emotional barriers by presenting children's behaviors to parents in an objective manner and 2) the cognitive barriers by acting as an anchor for conversation and presenting important developmental concepts or patterns that are hard to convey through words or text. Next, I identify features that make behavioral visualizations useful for various communication based tasks, such as displaying microbehaviors and providing a balanced representation of child-adult interaction, instead of solely focusing on the child behavior. Finally, I present visual and textual cues as rhetorical devices for shaping the message in the visualization and guiding the viewers through visualizations. These devices help reduce confusion and prevent miscommunication in visual-based communication as thus contribute to a more effective parent-clinician communication

Successful learning with whiteboard animations – a question of their procedural character or narrative embedding?

Although whiteboard animations are increasingly used for educational purposes, there is little empirical evidence as to why such animations can enhance learning. To specify essential elements, their dynamic visual presentation, as well as their narrative embedding, were found to be theortically important. In a first Experiment (N = 133) with a 2 (presentation mode: static pictures vs. progressive drawing) x 2 (narrative context: with vs. without a narrative) between-subject factorial design, motivational, cognitive, affective variables, as well as learning outcomes, of secondary school students were measured. Results revealed that progressive drawing, as well as a narrative context, are mostly associated with an increase in learning-relevant variables. In a second experiment with the same sample and the same experimental design but a different whiteboard animation, results from Experiment 1 generalize to another learning content. Again, a progressive drawing, as well as a narrative context within whiteboard animation, fostered learning relevant variables as well as learning outcomes. Results are discussed considering the cognitive theory of multimedia learning, the contiguity effect as well as the instructional design theory of anchored instruction