Experiences in involving analysts in visualisation design

Involving analysts in visualisation design has obvious benefits, but the knowledge-gap between domain experts ("analysts") and visualisation designers ("designers") often makes the degree of their involvement fall short of that aspired. By promoting a culture of mutual learning, understanding and contribution between both analysts and designers from the outset, participants can be raised to a level at which all can usefully contribute to both requirement definition and design. We describe the process we use to do this for tightly-scoped and short design exercises -- with meetings/workshops, iterative bursts of design/prototyping over relatively short periods of time, and workplace-based evaluation -- illustrating this with examples of our own experience from recent work with bird ecologists

On Regulatory and Organizational Constraints in Visualization Design and Evaluation

Problem-based visualization research provides explicit guidance toward identifying and designing for the needs of users, but absent is more concrete guidance toward factors external to a user's needs that also have implications for visualization design and evaluation. This lack of more explicit guidance can leave visualization researchers and practitioners vulnerable to unforeseen constraints beyond the user's needs that can affect the validity of evaluations, or even lead to the premature termination of a project. Here we explore two types of external constraints in depth, regulatory and organizational constraints, and describe how these constraints impact visualization design and evaluation. By borrowing from techniques in software development, project management, and visualization research we recommend strategies for identifying, mitigating, and evaluating these external constraints through a design study methodology. Finally, we present an application of those recommendations in a healthcare case study. We argue that by explicitly incorporating external constraints into visualization design and evaluation, researchers and practitioners can improve the utility and validity of their visualization solution and improve the likelihood of successful collaborations with industries where external constraints are more present.Comment: 9 pages, 2 figures, presented at BELIV workshop associated with IEEE VIS 201

Evaluating the relationship between user interaction and financial visual analysis

It has been widely accepted that interactive visualization techniques enable users to more effectively form hypotheses and identify areas for more detailed investigation. There have been numerous empirical user studies testing the effectiveness of specific visual analytical tools. However, there has been limited effort in connecting a user’s interaction with his reasoning for the purpose of extracting the relationship between the two. In this paper, we present an approach for capturing and analyzing user interactions in a financial visual analytical tool and describe an exploratory user study that examines these interaction strategies. To achieve this goal, we created two visual tools to analyze raw interaction data captured during the user session. The results of this study demonstrate one possible strategy for understanding the relationship between interaction and reasoning both operationally and strategically. Index Terms: H.5.2 [Information Interfaces And Presentatio

The Validity, Generalizability and Feasibility of Summative Evaluation Methods in Visual Analytics

Many evaluation methods have been used to assess the usefulness of Visual Analytics (VA) solutions. These methods stem from a variety of origins with different assumptions and goals, which cause confusion about their proofing capabilities. Moreover, the lack of discussion about the evaluation processes may limit our potential to develop new evaluation methods specialized for VA. In this paper, we present an analysis of evaluation methods that have been used to summatively evaluate VA solutions. We provide a survey and taxonomy of the evaluation methods that have appeared in the VAST literature in the past two years. We then analyze these methods in terms of validity and generalizability of their findings, as well as the feasibility of using them. We propose a new metric called summative quality to compare evaluation methods according to their ability to prove usefulness, and make recommendations for selecting evaluation methods based on their summative quality in the VA domain.Comment: IEEE VIS (VAST) 201

Design by immersion: A transdisciplinary approach to problem-driven visualizations

While previous work exists on how to conduct and disseminate insights from problem-driven visualization work and design studies, the literature does not address how to accomplish these goals in transdisciplinary teams in ways that advance all disciplines involved. In this paper we introduce and define a new methodological paradigm we call design by immersion, which provides an alternative perspective on problem-driven visualization work. Design by immersion embeds transdisciplinary experiences at the center of the visualization process by having visualization researchers participate in the work of the target domain (or domain experts participate in visualization research). Based on our own combined experiences of working on cross-disciplinary, problem-driven visualization projects, we present six case studies that expose the opportunities that design by immersion enables, including (1) exploring new domain-inspired visualization design spaces, (2) enriching domain understanding through personal experiences, and (3) building strong transdisciplinary relationships. Furthermore, we illustrate how the process of design by immersion opens up a diverse set of design activities that can be combined in different ways depending on the type of collaboration, project, and goals. Finally, we discuss the challenges and potential pitfalls of design by immersion

Visualization Psychology for Eye Tracking Evaluation

Technical progress in hardware and software enables us to record gaze data in everyday situations and over long time spans. Among a multitude of research opportunities, this technology enables visualization researchers to catch a glimpse behind performance measures and into the perceptual and cognitive processes of people using visualization techniques. The majority of eye tracking studies performed for visualization research is limited to the analysis of gaze distributions and aggregated statistics, thus only covering a small portion of insights that can be derived from gaze data. We argue that incorporating theories and methodology from psychology and cognitive science will benefit the design and evaluation of eye tracking experiments for visualization. This book chapter provides an overview of how eye tracking can be used in a variety of study designs. Further, we discuss the potential merits of cognitive models for the evaluation of visualizations. We exemplify these concepts on two scenarios, each focusing on a different eye tracking study. Lastly, we identify several call for actions