How Do Viewers Synthesize Conflicting Information from Data Visualizations?

Scientific knowledge develops through cumulative discoveries that build on, contradict, contextualize, or correct prior findings. Scientists and journalists often communicate these incremental findings to lay people through visualizations and text (e.g., the positive and negative effects of caffeine intake). Consequently, readers need to integrate diverse and contrasting evidence from multiple sources to form opinions or make decisions. However, the underlying mechanism for synthesizing information from multiple visualizations remains underexplored. To address this knowledge gap, we conducted a series of four experiments (N = 1166) in which participants synthesized empirical evidence from a pair of line charts presented sequentially. In Experiment 1, we administered a baseline condition with charts depicting no specific context where participants held no strong belief. To test for the generalizability, we introduced real-world scenarios to our visualizations in Experiment 2, and added accompanying text descriptions similar to on-line news articles or blog posts in Experiment 3. In all three experiments, we varied the relative direction and magnitude of line slopes within the chart pairs. We found that participants tended to weigh the positive slope more when the two charts depicted relationships in the opposite direction (e.g., one positive slope and one negative slope). Participants tended to weigh the less steep slope when the two charts depicted relationships in the same direction (e.g., both positive). Through these experiments, we characterize participants' synthesis behaviors depending on the relationship between the information they viewed, contribute to theories describing underlying cognitive mechanisms in information synthesis, and describe design implications for data storytelling.Comment: 11 pages, 5 figures, To be published in The IEEE Transactions on Visualizations and Computer Graphic

Improving the Analyst and Decision-Maker’s Perspective through Uncertainty Visualization

This thesis constructs the Taxonomy of Uncertainty and an approach for enhancing the information in decision support systems. The hierarchical categorization of numerous causes for uncertainty defines the taxonomy, which fostered the development of a technique for visualizing uncertainty. This technique is fundamental to expressing the multi-dimensional uncertainty that can be associated with any object. By including and intuitively expressing uncertainty, the approach facilitates and enhances intuition and decision-making without undue information overload. The resulting approach for enhancing the information involves recording uncertainty, identifying the relevant items, computing and visualizing uncertainty, and providing interaction with the selection of uncertainty. A prototype embodying this approach to enhancing information by including uncertainty was used to validate these efforts. Evaluation responses of a small sample space support the thesis that the decision-maker\u27s knowledge is enhanced with enlightening information afforded by including and visualizing uncertainty, which can improve the decision-making process. Although the concept was initially conceived to help decision support system users deal with uncertainty, this methodology and these ideas can be applied to any problem where objects with many potential reasons for uncertainty are the focus of the decision-making

EFFECTS OF INFORMATION PRESENTATION MODALITIES ON ANTIBIOTIC REASSESSMENT DECISION-MAKING IN PICU: A COMPARISON STUDY

Antibiotic resistance due to unnecessarily prolonged treatment course in the pediatric intensive care unit (PICU) remains a major healthcare challenge worldwide. Yet, appropriate clinical decision to stop the empiric treatment is susceptible to subjective judgment and often affected by the availability, accessibility, and modality of clinical information. This mix of factors for success has given rise to the challenge of effectively translating data to clinical decision-making. Visually presented clinical information is generally favored by physicians. However, there has been limited work to identify the appropriate clinical context and information presentation modality for a given decision support tool. Moreover, physician’s cognitive processes of changing belief when interacting with visually presented clinical information remain unexplored. This comparison study sought to assess the impact of information-presentation modality in a simulated environment using 4 case-vignettes and employed a factorial design with the following factors: 2 (decision correctness) by 2 (information presentation modality) by 4 (complexity-decision pairs). We hypothesized that compared to text narration, an interactive visualization would increase the correctness in decision outcome and change in belief of ongoing bacterial infection. 22 physicians completed the study. Overall, the interactive visualization led to small, but statistically nonsignificant, improvements in decision accuracy over text narration (χ2 (16) = 17.92, p = 0.33; LR (16) = 20.33, p = 0.21). However, when patient’s medical history was complex and required stopping of antibiotics, visualization significantly outperformed text narration in supporting making the accurate decision (p = 0.03). This result suggests that a complex patient’s clinical information presented with an interactive trend graph may provide a better basis for clinical decision-making than a traditional clinical note. We conclude that interactive visualization may be helpful for physicians assessing their antibiotic strategy for patients with a complex medical history. Future studies should conduct clinical trials investigating the use of interactive visualization to appropriately stop treatments given a complex patient medical history

Reasoning about quantities and concepts: studies in social learning

We live and learn in a ‘society of mind’. This means that we form beliefs not just based on our own observations and prior expectations but also based on the communications from other people, such as our social network peers. Across seven experiments, I study how people combine their own private observations with other people’s communications to form and update beliefs about the environment. I will follow the tradition of rational analysis and benchmark human learning against optimal Bayesian inference at Marr’s computational level. To accommodate human resource constraints and cognitive biases, I will further contrast human learning with a variety of process level accounts. In Chapters 2–4, I examine how people reason about simple environmental quantities. I will focus on the effect of dependent information sources on the success of group and individual learning across a series of single-player and multi-player judgement tasks. Overall, the results from Chapters 2–4 highlight the nuances of real social network dynamics and provide insights into the conditions under which we can expect collective success versus failures such as the formation of inaccurate worldviews. In Chapter 5, I develop a more complex social learning task which goes beyond estimation of environmental quantities and focuses on inductive inference with symbolic concepts. Here, I investigate how people search compositional theory spaces to form and adapt their beliefs, and how symbolic belief adaptation interfaces with individual and social learning in a challenging active learning task. Results from Chapter 5 suggest that people might explore compositional theory spaces using local incremental search; and that it is difficult for people to use another person’s learning data to improve upon their hypothesis

Visualizing COVID-19 with data: the effects of individual differences on perception of data in news

Mass media and public health organizations' efforts play a significant role in disseminating information and reducing the morbidity and mortality of infectious disease outbreaks. The vast amount of data generated about the pandemic led to the enormous spread of various data visualizations and infographics. Visuals served as the main tools that helped experts and journalists explain the consequences of the pandemic, communicate the facts, and persuade people to follow safety measures. Current research investigates how various formats of news messages such as data visualization and textual content affect an individual's perception of the message, such as message acceptance (positive attitudes about the message, intentions to follow prevention measures, and self-efficacy measure for behavior change), message rejection measures (defensive avoidance, negative attitudes about the message, reactance, anger) as well as credibility and effectiveness of the message. Political partisanship, need for cognition, and graphicacy were used as moderators. Results have demonstrated that the format of the message does not affect acceptance or rejection of the message, while moderators were significant predictors for dependent variables. The computational textual analysis illustrates the differences in topics between partisan groups where Democrats expressed more hope, positive sentiment, and more trust in vaccination, government, media, and science than independents and Republicans who were more prone to conspiracy theory thinking.Includes bibliographical references

Engineering Adaptive Interfaces – Enhancement of Comprehension and Decision-Making

The role of information systems is growing steadily and permeating more and more all levels of our society. Meanwhile, information systems have to support different user groups in various decision situations simultaneously. Hence, the existing design approach to creat- ing a unified user interface is reaching its limits. This work examines adaptive information system design by investigating user-adaptive information visualization and situation-aware nudging. An exploratory eye-tracking study investigates participants’ perception and comprehension of different financial visualizations and shows that none of them can be preferred across the board. Moreover, it reveals expertise knowledge as the research direction for visualization recommendations. Afterward, two empirical studies are conducted to relate different visualizations to participants’ domain-specific knowledge. The first study, conducted with a broad sample of the population, shows that financial and graphical literacy increases participants’ financial decision-making competency with certain visualizations. The second study, conducted with a more specific sample and an additional visualization, underlines a large part of the first study’s results. Additionally, it identifies statistical literacy as an increasing factor in financial decision-making. Both studies are demonstrating that different visualizations cause different cognitive loads despite the same amount of information. After all, the results are used to derive visualization recommendations based on domain-specific knowledge and cognitive load. This work also investigates the situation-aware effectiveness of nudging with the example of decision inertia. In a preliminary study, an experimental task is systematically transferred to different situational contexts by observing situational user characteristics. The identified contexts are examined in a subsequent large-scale empirical study with different nudges to reduce decision inertia. The results show gender-specific differences in decision inertia across the context. Hence, information system design has to adapt to gender and situational user characteristics to support users in their decision-making. Moreover, the study delivers empirical evidence for the contextual effectiveness of nudg- ing. Future nudging research has to incorporate situational user characteristics to provide effective nudges in different situational contexts. Especially, further fundamental research is needed to understand the situational effectiveness of nudging. The study identifies in- dividual situational preferences as one promising research stream

Interactive animated visualizations of probabilistic models

Bayesian probabilistic models’ structure (determined by the mathematical relations of the model’s variables) and outputs (i.e., the posterior distributions inferred through Bayesian inference) are complex and difficult to grasp and interprete without specialized knowledge. Various visualizations of probabilistic models exist but it is very little known about whether and how they support users’ comprehension of the models. The aim of this thesis is to investigate whether adding interaction or animation to visual representations of probabilistic models help people better understand the structure of models and interprete the (causal and non-causal) relations of the variables. This research presents a generic pipeline to transform a probabilistic model expressed in a Probabilistic Programming Language (PPL) and associated inference results into a standardized format which can then be automatically translated into an interactive probabilistic models explorer (IPME). IPME provides at-a-glance communication of a model’s structure and uncertainty, and allows interactive exploration of the multi-dimensional prior or posterior MCMC sample space. A collapsible tree-like structure represents the structure of the model in IPME. Each variable is represented by a node that presents graphically the prior or posterior distribution of the variable. Slicing on indexing dimensions or forming conjunctive restrictions on variables by interacting with the distribution visualizations is supported. Each user interaction with the explorer triggers the reestimation and visualization of the model’s uncertainty. This closed-loop exchange of responses between the user and the explorer allows the user to gain a more intuitive comprehension of the model. IPME was designed to enhance informativeness, transparency and explainability and ultimately, the potential of increasing trust in models. This research investigates also whether adding interactive conditioning to classical scatter plot matrices that present samples from the prior distribution of probabilistic models helps users better understand the models, and if there are levels of structural detail and model designs for which it is beneficial. A user study was conducted. The analysis of the collected data showed that interactive conditioning is beneficial in cases of sophisticated model designs and the difference in response time between the interaction and static group becomes less important in higher levels of structural detail. Participants using interactive conditioning were more confident about their responses overall with the effect being stronger in tasks of lower level of detail. This research proposes a pipeline to generate simulated probabilistic data from interven tions applied on causal structures that are expressed in PPLs using probabilistic modeling and Bayesian inference. An automatic visualization tool for visualizing the simulated probabilistic data generated by this pipeline was developed. A user study to evaluate the proposed tool was conducted. How effectively and efficiently people identify the causal model of the presented data and make decisions on interventional experiments when the uncertainty in the simulated data of interventions was presented using static, animated, or interactive visualizations was investigated. The findings suggested that participants were able to identify the causal model of the presented data either given a single intervention or by exploring various interventions. Their performance in identifying sufficient interventions was poor. Participants did not rely on the sufficient interventions to identify the causal model in the case of multi-interventional tasks. They might have relied more on combining information from multiple interventions to draw their conclusions. There were three different visual exploration strategies of the information in the scatter plot matrices which participants followed; roughly 1/3 of them relied on both the scatter and KDE plots, another 1/3 of them relied more on the scatter plots, and the last 1/3 of them relied more on the KDE plots. Those who followed the last strategy had a better performance in identifying the causal model given a specific intervention. Most participants judged the design of the visualization positively with many having mentioned that “it was informative”

Learners' self-assessment and metacognition when using an open learner model with drill down

Metacognition is ‘thinking on thinking’. It is important to educational practices for learners/teachers, and in activities such as formative-assessment and self-directed learning. The ability to perform metacognition is not innate and requires fostering, and self-assessment contributes to this. Literature suggests proven practices for promoting metacognitive opportunities and ongoing enquiry about how technology best supports these. This thesis considers an open learner model (OLM) with a drill-down approach as a method to investigate support for metacognition and self-assessment. Measuring aspects of metacognition without unduly influencing it is challenging. Direct measures (e.g. learners ‘thinking-aloud’) could distort/disrupt/encourage/effect metacognition. The thesis develops methods to evaluate aspects of metacognition without directly affecting it, relevant to future learning-analytics research/OLM design. It proposes a technology specification/implementation for supporting metacognition research and highlights the relevance of using a drill-down approach. Using measures that correspond to post-hoc learner accounts, this thesis identifies a baseline of student activity that is consistent with important regulation of cognition tasks and students’ specific interest in problems. Whilst this does not always influence self-assessment accuracy, students indicating their self-assessment ability can be used as a proxy measure to identify those who will improve. Evidence supports claims that OLMs remain relevant in metacognition research