Toward Visualization-Specific Heuristic Evaluation

This position paper describes heuristic evaluation as it relates to visualization and visual analytics. We review heuristic evaluation in general, then comment on previous process-based, performance-based, and framework-based efforts to adapt the method to visualization-specific needs. We postulate that the framework-based approach holds the most promise for future progress in development of visualization-specific heuristics, and propose a specific framework as a starting point. We then recommend a method for community involvement and input into the further development of the heuristic framework and more detailed design and evaluation guidelines

Doctor of Philosophy

dissertationThis dissertation establishes a new visualization design process model devised to guide visualization designers in building more effective and useful visualization systems and tools. The novelty of this framework includes its flexibility for iteration, actionability for guiding visualization designers with concrete steps, concise yet methodical definitions, and connections to other visualization design models commonly used in the field of data visualization. In summary, the design activity framework breaks down the visualization design process into a series of four design activities: understand, ideate, make, and deploy. For each activity, the framework prescribes a descriptive motivation, list of design methods, and expected visualization artifacts. To elucidate the framework, two case studies for visualization design illustrate these concepts, methods, and artifacts in real-world projects in the field of cybersecurity. For example, these projects employ user-centered design methods, such as personas and data sketches, which emphasize our teams' motivations and visualization artifacts with respect to the design activity framework. These case studies also serve as examples for novice visualization designers, and we hypothesized that the framework could serve as a pedagogical tool for teaching and guiding novices through their own design process to create a visualization tool. To externally evaluate the efficacy of this framework, we created worksheets for each design activity, outlining a series of concrete, tangible steps for novices. In order to validate the design worksheets, we conducted 13 student observations over the course of two months, received 32 online survey responses, and performed a qualitative analysis of 11 in-depth interviews. Students found the worksheets both useful and effective for framing the visualization design process. Next, by applying the design activity framework to technique-driven and evaluation-based research projects, we brainstormed possible extensions to the design model. Lastly, we examined implications of the design activity framework and present future work in this space. The visualization community is challenged to consider how to more effectively describe, capture, and communicate the complex, iterative nature of data visualization design throughout research, design, development, and deployment of visualization systems and tools

A Pattern Approach to Examine the Design Space of Spatiotemporal Visualization

Pattern language has been widely used in the development of visualization systems. This dissertation applies a pattern language approach to explore the design space of spatiotemporal visualization. The study provides a framework for both designers and novices to communicate, develop, evaluate, and share spatiotemporal visualization design on an abstract level. The touchstone of the work is a pattern language consisting of fifteen design patterns and four categories. In order to validate the design patterns, the researcher created two visualization systems with this framework in mind. The first system displayed the daily routine of human beings via a polygon-based visualization. The second system showed the spatiotemporal patterns of co-occurring hashtags with a spiral map, sunburst diagram, and small multiples. The evaluation results demonstrated the effectiveness of the proposed design patterns to guide design thinking and create novel visualization practices

Designing a Situational Awareness Information Display: Adopting an Affordance-Based Framework to Amplify User Experience in Environmental Interaction Design

User experience remains a crucial consideration when assessing the successfulness of information visualization systems. The theory of affordances provides a robust framework for user experience design. In this article, we demonstrate a design case that employs an affordance-based framework and evaluate the information visualization display design. SolarWheels is an interactive information visualization designed for large display walls in computer network control rooms to help cybersecurity analysts become aware of network status and emerging issues. Given the critical nature of this context, the status and performance of a computer network must be precisely monitored and remedied in real time. In this study, we consider various aspects of affordances in order to amplify the user experience via visualization and interaction design. SolarWheels visualizes the multilayer multidimensional computer network issues with a series of integrated circular visualizations inspired by the metaphor of the solar system. To amplify user interaction and experience, the system provides a three-zone physical interaction that allows multiple users to interact with the system. Users can read details at different levels depending on their distance from the display. An expert evaluation study, based on a four-layer affordance framework, was conducted to assess and improve the interactive visualization design

Developing a Data Dashboard Framework for Population Health Surveillance: Widening Access to Clinical Trial Findings

Background: Population surveillance sites generate many datasets relevant to disease surveillance. However, there is a risk that these data are underutilized because of the volumes of data gathered and the lack of means to quickly disseminate analysis. Data visualization offers a means to quickly disseminate, understand, and interpret datasets, facilitating evidence-driven decision making through increased access to information. Objectives: This paper describes the development and evaluation of a framework for data dashboard design, to visualize datasets produced at a demographic health surveillance site. The aim of this research was to produce a comprehensive, reusable, and scalable dashboard design framework to fit the unique requirements of the context. Methods: The framework was developed and implemented at a demographic surveillance platform at the Africa Health Research Institute, in KwaZulu-Natal, South Africa. This context represents an exemplar implementation for the use of data dashboards within a population health-monitoring setting. Before the full launch, an evaluation study was undertaken to assess the effectiveness of the dashboard framework as a data communication and decision-making tool. The evaluation included a quantitative task evaluation to assess usability and a qualitative questionnaire exploring the attitudes to the use of dashboards. Results: The evaluation participants were drawn from a diverse group of users working at the site (n=20), comprising of community members, nurses, scientific and operational staff. Evaluation demonstrated high usability for the dashboard across user groups, with scientific and operational staff having minimal issues in completing tasks. There were notable differences in the efficiency of task completion among user groups, indicating varying familiarity with data visualization. The majority of users felt that the dashboards provided a clear understanding of the datasets presented and had a positive attitude to their increased use. Conclusions: Overall, this exploratory study indicates the viability of the data dashboard framework in communicating data trends within population surveillance setting. The usability differences among the user groups discovered during the evaluation demonstrate the need for the user-led design of dashboards in this context, addressing heterogeneous computer and visualization literacy present among the diverse potential users present in such settings. The questionnaire highlighted the enthusiasm for increased access to datasets from all stakeholders highlighting the potential of dashboards in this context

Conceptual design framework for information visualization to support multidimensional datasets in higher education institutions

Information Visualization (InfoVis) enjoys diverse adoption and applicability because of its strength in solving the problem of information overload inherent in institutional data. Policy and decision makers of higher education institutions (HEIs) are also experiencing information overload while interacting with students‟ data, because of its multidimensionality. This constraints decision making processes, and therefore requires a domain-specific InfoVis conceptual design framework which will birth the domain‟s InfoVis tool. This study therefore aims to design HEI Students‟ data-focused InfoVis (HSDI) conceptual design framework which addresses the content delivery techniques and the systematic processes in actualizing the domain specific InfoVis. The study involved four phases: 1) a users‟ study to investigate, elicit and prioritize the students‟ data-related explicit knowledge preferences of HEI domain policy. The corresponding students‟ data dimensions are then categorised, 2) exploratory study through content analysis of InfoVis design literatures, and subsequent mapping with findings from the users‟ study, to propose the appropriate visualization, interaction and distortion techniques for delivering the domain‟s explicit knowledge preferences, 3) conceptual development of the design framework which integrates the techniques‟ model with its design process–as identified from adaptation of software engineering and InfoVis design models, 4) evaluation of the proposed framework through expert review, prototyping, heuristics evaluation, and users‟ experience evaluation. For an InfoVis that will appropriately present and represent the domain explicit knowledge preferences, support the students‟ data multidimensionality and the decision making processes, the study found that: 1) mouse-on, mouse-on-click, mouse on-drag, drop down menu, push button, check boxes, and dynamics cursor hinting are the appropriate interaction techniques, 2) zooming, overview with details, scrolling, and exploration are the appropriate distortion techniques, and 3) line chart, scatter plot, map view, bar chart and pie chart are the appropriate visualization techniques. The theoretical support to the proposed framework suggests that dictates of preattentive processing theory, cognitive-fit theory, and normative and descriptive theories must be followed for InfoVis to aid perception, cognition and decision making respectively. This study contributes to the area of InfoVis, data-driven decision making process, and HEI students‟ data usage process

GraphUnit: Evaluating Interactive Graph Visualizations Using Crowdsourcing

We present GraphUnit, a framework and online service that automates the process of designing, running and analyzing results of controlled user studies of graph visualizations by leveraging crowdsourcing and a set of evaluation modules based on a graph task taxonomy. User studies play an important role in visualization research but conducting them requires expertise and is time consuming. GraphUnit simplifies the evaluation process by allowing visualization designers to easily configure user studies for their web-based graph visualizations, deploy them online, use Mechanical Turk to attract participants, collect user responses and store them in a database, and analyze incoming results automatically using appropriate statistical tools and graphs. We demonstrate the effectiveness of GraphUnit by replicating two published evaluation studies on network visualization, and showing that these studies could be configured in less than an hour. Finally, we discuss how GraphUnit can facilitate quick evaluations of alternative graph designs and thus encourage the frequent use of user studies to evaluate design decisions in iterative development processes