Teaching Tip: Evaluating Visualizations with a Compact Rubric

Students now have readily available and powerful tools to access, manipulate, combine, and visualize data. Acquiring data and visual literacy requires more than knowledge of how to use these tools. Students need to engage with assignments that challenge them to make relatively complex visualizations, interpret them, and explain why these interpretations matter for given problem situations. This paper describes how to structure feedback for these assignments. The few published visualization evaluation rubrics are mainly oriented toward how-to-do-it heuristics. This paper makes a contribution by presenting, defining, and giving examples of the use of an innovative compact rubric for evaluating visualizations (CRVE). This rubric eliminates some of the length and complexity of heuristic evaluation, focusing on interpretation and relevance. In a graduate business intelligence course, students showed definite improvement over the course of the semester in the construction of visualizations, telling a story with headlines, and striving for data exploration. However, higher levels of technical correctness of visualizations did not necessarily correspond to better interpretations. This finding underscores the importance of emphasizing interpretation through a feedback mechanism like the CRVE presented here

Macroprudential oversight, risk communication and visualization

This paper discusses the role of risk communication in macroprudential oversight and of visualization in risk communication. Beyond the soar in data availability and precision, the transition from firm-centric to system-wide supervision imposes vast data needs. Moreover, except for internal communication as in any organization, broad and effective external communication of timely information related to systemic risks is a key mandate of macroprudential supervisors, further stressing the importance of simple representations of complex data. This paper focuses on the background and theory of information visualization and visual analytics, as well as techniques within these fields, as potential means for risk communication. We define the task of visualization in risk communication, discuss the structure of macroprudential data, and review visualization techniques applied to systemic risk. We conclude that two essential, yet rare, features for supporting the analysis of big data and communication of risks are analytical visualizations and interactive interfaces. For visualizing the so-called macroprudential data cube, we provide the VisRisk platform with three modules: plots, maps and networks. While VisRisk is herein illustrated with five web-based interactive visualizations of systemic risk indicators and models, the platform enables and is open to the visualization of any data from the macroprudential data cube

Evaluation methodology for visual analytics software

O desafio do Visual Analytics (VA) é produzir visualizações que ajudem os utilizadores a concentrarem-se no aspecto mais relevante ou mais interessante dos dados apresentados. A sociedade actual enfrenta uma quantidade de dados que aumenta rapidamente. Assim, os utilizadores de informação em todos os domínios acabam por ter mais informação do que aquela com que podem lidar. O software VA deve suportar interacções intuitivas para que os analistas possam concentrar-se na informação que estão a manipular, e não na técnica de manipulação em si. Os ambientes de VA devem procurar minimizar a carga de trabalho cognitivo global dos seus utilizadores, porque se tivermos de pensar menos nas interacções em si, teremos mais tempo para pensar na análise propriamente dita. Tendo em conta os benefícios que as aplicações VA podem trazer e a confusão que ainda existe ao identificar tais aplicações no mercado, propomos neste trabalho uma nova metodologia de avaliação baseada em heurísticas. A nossa metodologia destina-se a avaliar aplicações através de testes de usabilidade considerando as funcionalidades e características desejáveis em sistemas de VA. No entanto, devido à sua natureza quatitativa, pode ser naturalmente utilizada para outros fins, tais como comparação para decisão entre aplicações de VA do mesmo contexto. Além disso, seus critérios poderão servir como fonte de informação para designers e programadores fazerem escolhas apropriadas durante a concepção e desenvolvimento de sistemas de VA

시각화 초심자에게 시각적 비교를 돕는 정보 시각화 기술의 디자인

학위논문(박사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2020. 2. 서진욱.The visual comparison is one of the fundamental tasks in information visualization (InfoVis) that enables people to organize, evaluate, and combine information fragmented in visualizations. For example, people perform visual comparison tasks to compare data over time, from different sources, or with different analytic models. While the InfoVis community has focused on understanding the effectiveness of different visualization designs for supporting visual comparison tasks, it is still unclear how to design effective comparative visualizations due to several limitations: (1) Empirical findings and practical implications from those studies are fragmented, and (2) we lack user studies that directly investigated the effectiveness of different visualization designs for visual comparison. In this dissertation, we present the results of three studies to build our knowledge on how to support effective visual comparison to InfoVis novices⁠—general people who are not familiar with visual representations and visual data exploration process. Identifying the major stages in the visualization construction process where novices confront challenges with visual comparison tasks, we explored two high-level comparison tasks with actual users: comparing visual mapping (encoding barrier) and comparing information (interpretation barrier) in visualizations. First, we conducted a systematical literature review on research papers (N = 104) that focused on supporting visual comparison tasks to gather and organize the practical insights that researchers gained in the wild. From this study, we offered implications for designing comparative visualizations, such as actionable guidelines, as well as the lucid categorization of comparative designs which can help researchers explore the design space. In the second study, we performed a qualitative user study (N = 24) to investigate how novices compare and understand visual mapping suggested in a visual-encoding recommendation interface. Based on the study, we present novices' main challenges in using visual encoding recommendations and design implications as remedies. In the third study, we conducted a design study in the area on bioinformatics to design and implement a visual analytics tool, XCluSim, that helps users to compare multiple clustering results. Case studies with a bioinformatician showed that our system enables analysts to easily evaluate the quality of a large number of clustering results. Based on the results of three studies in this dissertation, we suggest a future research agenda, such as designing recommendations for visual comparison and distinguishing InfoVis novices from experts.시각적 비교는 정보 시각화를 이용한 핵심적인 데이터 분석 과정 중 하나로써, 분산되어 있는 정보들을 사람들이 서로 정리, 평가, 병합할 수 있도록 돕는다. 예를 들어, 사람들은 시간의 흐름에 따른 데이터의 변화를 보거나, 서로 다른 출처의 데이터를 비교하거나, 같은 데이터를 여러 분석 모델들을 이용해 평가하기 위해 시각적 비교 과업을 흔히 수행하게 된다. 효과적인 시각화 디자인을 위한 여러 연구가 정보 시각화 분야에서 이루어지고 있는 반면, 어떤 디자인을 통해 효과적으로 시각적 비교를 지원할 수 있는지에 대한 이해는 다음의 제약들로 인해 아직까지 불분명하다. (1) 경험적 통찰들과 실용적 설계 지침들이 파편화되어 있으며 (2) 비교 시각화를 지원하는 방법을 이해하기 위한 사용자 실험의 수가 여전히 제한적이다. 본 논문에서는 시각화 초심자들에게 효과적으로 시각적 비교를 지원하기 위한 정보 시각화 디자인 방법을 더 깊이 이해하기 위해서 일련의 세 연구를 진행하고 이에 대한 결과를 제시한다. 특별히, 시각화 초심자들이 시각적 비교를 할 때 어려움을 경험할 수 있는 두 주요 시각화 단계를 확인함으로써, 본 연구에서는 시각적 인코딩 비교 (인코딩 장벽) 및 정보 비교 (해석 장벽) 과업들에 초점을 맞춘다. 첫째, 비교 시각화 디자인을 제시한 문헌들(N = 104)을 체계적으로 조사 및 분석함으로써 시각화 연구자들이 사용자 실험과 시각화 설계 과정을 통해 얻은 실용적 통찰들을 정리하였다. 이 문헌조사를 기반으로 비교 시각화 설계에 대한 지침들을 정립하고, 비교 시각화를 위한 디자인 공간을 더 깊이 이해하고 탐색하는 데 도움을 줄 수 있는 시각화 분류 및 예시들을 제공한다. 둘째, 초심자들이 시각화 추천 인터페이스에서 어떻게 새로운 시각적 인코딩들을 서로 비교하고 사용하는지에 대한 이해를 돕기 위해 사용자 실험(N = 24)을 수행하였다. 이 실험의 결과를 기반으로, 초심자들의 주요 어려움들과 이들을 해결하기 위한 디자인 지침들을 제시한다. 셋째, 생명정보학자가 시각적으로 다수 개의 클러스터링 결과들을 비교 및 분석할 수 있도록 도와주는 시각화 시스템, XCluSim을 디자인하고 구현하는 디자인 스터디를 수행하였다. 사례 연구를 통해 실제로 생명정보학자가 XCluSim을 이용하여 많은 클러스터링 결과들을 쉽게 비교 및 평가할 수 있다는 것을 보였다. 마지막으로, 이 세 연구 결과들을 기반으로 비교 시각화 분야에서 유망한 향후 연구들을 제시한다.CHAPTER 1. Introduction 1 1.1 Background and Motivation 1 1.2 Research Questions and Approaches 4 1.2.1 Revisiting Comparative Layouts: Design Space, Guidelines, and Future Directions 5 1.2.2 Understanding How InfoVis Novices Compare Visual Encoding Recommendation 6 1.2.3 Designing XCluSim: a Visual Analytics System for Comparing Multiple Clustering Results 7 1.3 Dissertation Outline 8 CHAPTER 2. Related Work 9 2.1 Visual Comparison Tasks 9 2.2 Visualization Designs for Comparison 10 2.2.1 Gleicher et al.s Comparative Layout 11 2.3 Understanding InfoVis Novices 12 2.4 Visualization Recommendation Interfaces 13 2.5 Comparative Visualizations for Cluster Analysis 14 CHAPTER 3. Comparative Layouts Revisited: Design Space, Guidelines, and Future Directions 19 3.1 Introduction 19 3.2 Literature Review 21 3.2.1 Method 22 3.3 Comparative Layouts in The Wild 23 3.3.1 Classifying Comparison Tasks in User Studies 25 3.3.2 Same LayoutIs Called Differently 26 3.3.3 Lucid Classification of Comparative Layouts 28 3.3.4 Advantages and Concerns of Using Each Layout 30 3.3.5 Trade-offs between Comparative Layouts 36 3.3.6 Approaches to Overcome the Concerns 38 3.3.7 Comparative Layout Explorer 42 3.4 Discussion 42 3.4.1 Guidelines for Comparative Layouts 44 3.4.2 Promising Directions for Future Research 48 3.5 Summary 49 CHAPTER 4. Understanding How InfoVis Novices Compare Visual Encoding Recommendation 51 4.1 Motivation 51 4.2 Interface 53 4.2.1 Visualization Goals 53 4.2.2 Recommendations 54 4.2.3 Representation Methods for Recommendations 54 4.2.4 Interface 58 4.2.5 Pilot Study 61 4.3 User Study 62 4.3.1 Participants 62 4.3.2 Interface 62 4.3.3 Tasks and Datasets 65 4.3.4 Procedure. 65 4.4 Findings 68 4.4.1 Poor Design Decisions 68 4.4.2 Role of Preview, Animated Transition, and Text 69 4.4.3 Challenges For Understanding Recommendations 70 4.4.4 Learning By Doing 71 4.4.5 Effects of Recommendation Order 71 4.4.6 Personal Criteria for Selecting Recommendations 72 4.5 Discussion 73 4.5.1 Design Implications 73 4.5.2 Limitations and FutureWork 75 4.6 Summary 77 CHAPTER 5. Designing XCluSim: a Visual Analytics System for Comparing Multiple Clustering Results 78 5.1 Motivation 78 5.2 Task Analysis and Design Goals 79 5.3 XCluSim 80 5.3.1 Color Encoding of Clusters Using Tree Colors 82 5.3.2 Overview of All Clustering Results 83 5.3.3 Visualization for Comparing Selected Clustering Results 86 5.3.4 Visualization for Individual Clustering Results 92 5.3.5 Implementation 100 5.4 CaseStudy 100 5.4.1 Elucidating the Role of Ferroxidase in Cryptococcus Neoformans Var. Grubii H99 (CaseStudy 1) 100 5.4.2 Finding a Clustering Result that Clearly Represents Biological Relations (CaseStudy 2) 103 5.5 Discussion 106 5.5.1 Limitations and FutureWork 108 5.6 Summary 108 CHAPTER 6. Future Research Agenda 110 6.0.1 Recommendation for Visual Comparison 110 6.0.2 Understanding the Perception of Subtle Difference 111 6.0.3 Distinguishing InfoVis Novices from Experts 112 CHAPTER 7. Conclusion. 113 Abstract (Korean) 129 Acknowledgments (Korean) 131Docto

Data-ink Ratio and Task Complexity in Graph Comprehension

Human processing of graphical information is a topic which has wide-reaching implications for decision-making in a variety of contexts. A deeper understanding of the processes of graphical perception can lead to the development of design guidelines which can enhance performance in graphical perception tasks. This study evaluates the data-ink ratio guideline, which recommends the removal of non-data graph elements, resulting in minimalist graph designs. In an experiment, participants answered graph comprehension questions using bar graphs and boxplots with varying data-ink ratios. Participants answered questions with similar levels of accuracy and mental effort. Some participants drew on graphs, reducing the data-ink ratio of high and medium data-ink stimuli. Additionally, expert interviews were conducted regarding graph use, graph creation, and opinions about the data-ink concept and example graphs. Interviewees had a variety of opinions and preferences with regard to graph design, many of which were dependent upon the specific circumstances of presentation. Most interviewees did not think that high data-ink graph designs were superior. These results suggest that data-ink maximization does not improve performance in graph comprehensions tasks, and that arguments regarding the data-ink ratio deal with the subjective issue of graph aesthetics