4 research outputs found
The Validity, Generalizability and Feasibility of Summative Evaluation Methods in Visual Analytics
Many evaluation methods have been used to assess the usefulness of Visual
Analytics (VA) solutions. These methods stem from a variety of origins with
different assumptions and goals, which cause confusion about their proofing
capabilities. Moreover, the lack of discussion about the evaluation processes
may limit our potential to develop new evaluation methods specialized for VA.
In this paper, we present an analysis of evaluation methods that have been used
to summatively evaluate VA solutions. We provide a survey and taxonomy of the
evaluation methods that have appeared in the VAST literature in the past two
years. We then analyze these methods in terms of validity and generalizability
of their findings, as well as the feasibility of using them. We propose a new
metric called summative quality to compare evaluation methods according to
their ability to prove usefulness, and make recommendations for selecting
evaluation methods based on their summative quality in the VA domain.Comment: IEEE VIS (VAST) 201