ConfusionFlow: A Model-Agnostic Visualization for Temporal Analysis of Classifier Confusion

Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to assess classifiers' performances, evaluate their learning behavior over time, and compare different models. Typically, this analysis is based on single-number performance measures such as accuracy. A more detailed evaluation of classifiers is possible by inspecting class errors. The confusion matrix is an established way for visualizing these class errors, but it was not designed with temporal or comparative analysis in mind. More generally, established performance analysis systems do not allow a combined temporal and comparative analysis of class-level information. To address this issue, we propose ConfusionFlow, an interactive, comparative visualization tool that combines the benefits of class confusion matrices with the visualization of performance characteristics over time. ConfusionFlow is model-agnostic and can be used to compare performances for different model types, model architectures, and/or training and test datasets. We demonstrate the usefulness of ConfusionFlow in a case study on instance selection strategies in active learning. We further assess the scalability of ConfusionFlow and present a use case in the context of neural network pruning

ProjectionPathExplorer: Exploring Visual Patterns in Projected Decision-Making Paths

In problem-solving, a path towards solutions can be viewed as a sequence of decisions. The decisions, made by humans or computers, describe a trajectory through a high-dimensional representation space of the problem. By means of dimensionality reduction, these trajectories can be visualized in lower-dimensional space. Such embedded trajectories have previously been applied to a wide variety of data, but analysis has focused almost exclusively on the self-similarity of single trajectories. In contrast, we describe patterns emerging from drawing many trajectories---for different initial conditions, end states, and solution strategies---in the same embedding space. We argue that general statements about the problem-solving tasks and solving strategies can be made by interpreting these patterns. We explore and characterize such patterns in trajectories resulting from human and machine-made decisions in a variety of application domains: logic puzzles (Rubik's cube), strategy games (chess), and optimization problems (neural network training). We also discuss the importance of suitably chosen representation spaces and similarity metrics for the embedding.Comment: Final version; accepted for publication in the ACM TiiS Special Issue on "Interactive Visual Analytics for Making Explainable and Accountable Decisions

VisAhoi: Towards a Library to Generate and Integrate Visualization Onboarding Using High-level Visualization Grammars

Visualization onboarding supports users in reading, interpreting, and extracting information from visual data representations. General-purpose onboarding tools and libraries are applicable for explaining a wide range of graphical user interfaces but cannot handle specific visualization requirements. This paper describes a first step towards developing an onboarding library called VisAhoi, which is easy to integrate, extend, semi-automate, reuse, and customize. VisAhoi supports the creation of onboarding elements for different visualization types and datasets. We demonstrate how to extract and describe onboarding instructions using three well-known high-level descriptive visualization grammars - Vega-Lite, Plotly.js, and ECharts. We show the applicability of our library by performing two usage scenarios that describe the integration of VisAhoi into a VA tool for the analysis of high-throughput screening (HTS) data and, second, into a Flourish template to provide an authoring tool for data journalists for a treemap visualization. We provide a supplementary website that demonstrates the applicability of VisAhoi to various visualizations, including a bar chart, a horizon graph, a change matrix or heatmap, a scatterplot, and a treemap visualization

Peptide-equipped tobacco mosaic virus templates for selective and controllable biomineral deposition

The coating of regular-shaped, readily available nanorod biotemplates with inorganic compounds has attracted increasing interest during recent years. The goal is an effective, bioinspired fabrication of fiber-reinforced composites and robust, miniaturized technical devices. Major challenges in the synthesis of applicable mineralized nanorods lie in selectivity and adjustability of the inorganic material deposited on the biological, rod-shaped backbones, with respect to thickness and surface profile of the resulting coating, as well as the avoidance of aggregation into extended superstructures. Nanotubular tobacco mosaic virus (TMV) templates have proved particularly suitable towards this goal: Their multivalent protein coating can be modified by high-surface-density conjugation of peptides, inducing and governing silica deposition from precursor solutions in vitro. In this study, TMV has been equipped with mineralization-directing peptides designed to yield silica coatings in a reliable and predictable manner via precipitation from tetraethoxysilane (TEOS) precursors. Three peptide groups were compared regarding their influence on silica polymerization: (i) two peptide variants with alternating basic and acidic residues, i.e. lysine–aspartic acid (KD)$_{χ}$ motifs expected to act as charge-relay systems promoting TEOS hydrolysis and silica polymerization; (ii) a tetrahistidine-exposing polypeptide (CA$_{4}$H$_{4}$) known to induce silicification due to the positive charge of its clustered imidazole side chains; and (iii) two peptides with high ZnO binding affinity. Differential effects on the mineralization of the TMV surface were demonstrated, where a (KD)$_{χ}$ charge-relay peptide (designed in this study) led to the most reproducible and selective silica deposition. A homogenous coating of the biotemplate and tight control of shell thickness were achieved

Interactive Focus+Context Analysis of Time-Series and Provenance Data

Die effiziente Exploration großer und komplexer Datensätze ist eine fortwährende Herausforderung in Visual Analytics. Visualisiert man solche Datensätze als Ganzes führt das in der Regel zu einer chaotischen Darstellung, die es für den Anwender schwierig macht, potenziell interessante Teile aus den Daten zu identifizieren. Eine mögliche Lösung zur Reduzierung der Unordnung sind Fokus- und Kontexttechniken, welche ausgewählte Regionen detaillierter darstellen und gleichzeitig eine Übersicht mit reduzierten Details geben. Für große Datensätze bleibt die Auswahl der Fokusbereiche jedoch eine zeitaufwändige Aufgabe für den Anwender, sofern diese einzeln ausgewählt werden müssen. Des Weiteren ist das Interesse für eine bestimmte Teilmenge im Fall von zeitbasierten Daten möglicherweise nicht konstant, sondern verschiebt sich über die Zeit oder wechselt zu einer anderen Teilmenge. Folglich ist es notwendig, Fokus- und Kontexttechniken zu entwickeln, die speziell auf große, zeitbasierte Daten zugeschnitten sind. Diese Arbeit stellt vier interaktive Visualisierungsansätze zur Hervorhebung interessanter Teilmengen aus Zeitreihen und Provenienzdaten vor. Die Lösungen verwenden modulare Funktionen mit denen der Anwender das Interesse für die Teilmengen definieren kann. Die Funktionen können von einem oder mehreren Datenattributen, der Topologie des Graphen oder einer Kombination aus beidem gesteuert werden. Die praktische Anwendbarkeit wird anhand von verschiedenen Fallstudien aus den Bereichen Cloud Computing, Finanzwesen und biomedizinische Forschung gezeigt.Efficient exploration of large and complex datasets, as for instance, time-series and provenance data, is an ongoing research challenge in visual analytics. Visualizing such datasets in one go often leads to visual clutter, making it hard for users to identify potentially interesting data subsets. A possible solution to reduce the clutter is Focus+Context techniques, which visualize selected regions in greater detail while preserving an overview with reduced details. For large datasets, however, selecting focus regions can become a time-consuming task if each region must be selected individually. Furthermore, in the case of temporal data, the interest in a particular data subset might not remain constant but, rather, shift over time or switch to other data subsets. Consequently, it is necessary to develop Focus+Context solutions tailored to large temporal data. This thesis presents four interactive visualization approaches for highlighting potentially interesting subsets in time-series and provenance data. The solutions utilize modular degree of interest functions that are driven by one or multiple data attributes, the topology of the graph, or a combination of both. The practical applicability of these approaches is demonstrated by means of different case studies from cloud computing, Finance, and biomedical research.submitted by Holger Stitz, MSc.Universität Linz, Dissertation, 2019OeBB(VLID)366368

Provectories: Embedding-based Analysis of Interaction Provenance Data

Understanding user behavior patterns and visual analysis strategies is a long-standing challenge. Existing approaches rely largely on time-consuming manual processes such as interviews and the analysis of observational data. While it is technically possible to capture a history of user interactions and application states, it remains difficult to extract and describe analysis strategies based on interaction provenance. In this paper, we propose a novel visual approach to the meta-analysis of interaction provenance. We capture single and multiple user sessions as graphs of high-dimensional application states. Our meta-analysis is based on two different types of two-dimensional embeddings of these high-dimensional states: layouts based on (i) topology and (ii) attribute similarity. We applied these visualization approaches to synthetic and real user provenance data captured in two user studies. From our visualizations, we were able to extract patterns for data types and analytical reasoning strategies

A Process Model for Dashboard Onboarding

Dashboards are used ubiquitously to gain and present insights into data by means of interactive visualizations. To bridge the gap between non-expert dashboard users and potentially complex datasets and/or visualizations, a variety of onboarding strategies are employed, including videos, narration, and interactive tutorials. We propose a process model for dashboard onboarding which formalizes and unifies such diverse onboarding strategies. Our model introduces the onboarding loop alongside the dashboard usage loop. Unpacking the onboarding loop reveals how each onboarding strategy combines selected building blocks of the dashboard with an onboarding narrative. Specific means are applied to this narration sequence for onboarding, which results in onboarding artifacts that are presented to the user via an interface. We concretize these concepts by showing how our process model can be used to describe a selection of real-world onboarding examples. Finally, we discuss how our model can serve as an actionable blueprint for developing new onboarding systems

Design of Visualization Onboarding Concepts for a 2D Scatterplot in a Biomedical Visual Analytics Tool

Biomedical research is highly data-driven. Domain experts need to learn how to interpret complex data visualizations to gain insights. They often need help interpreting data visualizations as they are not part of their training. Integrating visualization onboarding concepts into visual analytics (VA) tools can support users in interpreting, reading, and extracting information from visual presentations. In this paper, we present the design of the onboarding concept for an interactive VA tool to analyze large-scale biological data, particularly high-throughput screening (HTS) data. We evaluated our onboarding design by conducting a cognitive walkthrough and interviews with thinking aloud. We also collected data on domain experts’ visualization literacy. The results of the cognitive walkthrough showed that domain experts positively commented on the onboarding design and proposed adjusting smaller aspects. The interviews showed that domain experts are well-trained in interpreting basic visualizations (e.g., scatterplot, bar chart, line chart). However, they need support correctly interpreting the data visualized in the scatterplot, as they are new to them. Another important insight was fitting the onboarding messages into the domain’s language

Design of Visualization Onboarding Concepts for a 2D Scatterplot in a Biomedical Visual Analytics Tool

Biomedical research is highly data-driven. Domain experts need to learn how to interpret complex data visualizations to gain insights. They often need help interpreting data visualizations as they are not part of their training. Integrating visualization onboarding concepts into visual analytics (VA) tools can support users in interpreting, reading, and extracting information from visual presentations. In this paper, we present the design of the onboarding concept for an in- teractive VA tool to analyze large scaled biological data, particularly high-throughput screening (HTS) data. We evaluated our onboard- ing design by conducting a cognitive walkthrough and interviews with thinking aloud. We also collected data on domain experts’ visu- alization literacy. The results of the cognitive walkthrough showed that domain experts positively commented on the onboarding design and proposed adjusting smaller aspects. The interviews showed that domain experts are well-trained in interpreting basic visualiza- tions (e.g., scatterplot, bar chart, line chart). However, they need support correctly interpreting the data visualized in the scatterplot, as they are new to them. Another important insight was fitting the onboarding messages into the domain’s language