Doctor of Philosophy

dissertationWith the ever-increasing amount of available computing resources and sensing devices, a wide variety of high-dimensional datasets are being produced in numerous fields. The complexity and increasing popularity of these data have led to new challenges and opportunities in visualization. Since most display devices are limited to communication through two-dimensional (2D) images, many visualization methods rely on 2D projections to express high-dimensional information. Such a reduction of dimension leads to an explosion in the number of 2D representations required to visualize high-dimensional spaces, each giving a glimpse of the high-dimensional information. As a result, one of the most important challenges in visualizing high-dimensional datasets is the automatic filtration and summarization of the large exploration space consisting of all 2D projections. In this dissertation, a new type of algorithm is introduced to reduce the exploration space that identifies a small set of projections that capture the intrinsic structure of high-dimensional data. In addition, a general framework for summarizing the structure of quality measures in the space of all linear 2D projections is presented. However, identifying the representative or informative projections is only part of the challenge. Due to the high-dimensional nature of these datasets, obtaining insights and arriving at conclusions based solely on 2D representations are limited and prone to error. How to interpret the inaccuracies and resolve the ambiguity in the 2D projections is the other half of the puzzle. This dissertation introduces projection distortion error measures and interactive manipulation schemes that allow the understanding of high-dimensional structures via data manipulation in 2D projections

Parallel Hierarchies: Interactive Visualization of Multidimensional Hierarchical Aggregates

Exploring multi-dimensional hierarchical data is a long-standing problem present in a wide range of fields such as bioinformatics, software systems, social sciences and business intelligence. While each hierarchical dimension within these data structures can be explored in isolation, critical information lies in the relationships between dimensions. Existing approaches can either simultaneously visualize multiple non-hierarchical dimensions, or only one or two hierarchical dimensions. Yet, the challenge of visualizing multi-dimensional hierarchical data remains open. To address this problem, we developed a novel data visualization approach -- Parallel Hierarchies -- that we demonstrate on a real-life SAP SE product called SAP Product Lifecycle Costing. The starting point of the research is a thorough customer-driven requirement engineering phase including an iterative design process. To avoid restricting ourselves to a domain-specific solution, we abstract the data and tasks gathered from users, and demonstrate the approach generality by applying Parallel Hierarchies to datasets from bioinformatics and social sciences. Moreover, we report on a qualitative user study conducted in an industrial scenario with 15 experts from 9 different companies. As a result of this co-innovation experience, several SAP customers requested a product feature out of our solution. Moreover, Parallel Hierarchies integration as a standard diagram type into SAP Analytics Cloud platform is in progress. This thesis further introduces different uncertainty representation methods applicable to Parallel Hierarchies and in general to flow diagrams. We also present a visual comparison taxonomy for time-series of hierarchically structured data with one or multiple dimensions. Moreover, we propose several visual solutions for comparing hierarchies employing flow diagrams. Finally, after presenting two application examples of Parallel Hierarchies on industrial datasets, we detail two validation methods to examine the effectiveness of the visualization solution. Particularly, we introduce a novel design validation table to assess the perceptual aspects of eight different visualization solutions including Parallel Hierarchies.:1 Introduction 1.1 Motivation and Problem Statement 1.2 Research Goals 1.3 Outline and Contributions 2 Foundations of Visualization 2.1 Information Visualization 2.1.1 Terms and Definition 2.1.2 What: Data Structures 2.1.3 Why: Visualization Tasks 2.1.4 How: Visualization Techniques 2.1.5 How: Interaction Techniques 2.2 Visual Perception 2.2.1 Visual Variables 2.2.2 Attributes of Preattentive and Attentive Processing 2.2.3 Gestalt Principles 2.3 Flow Diagrams 2.3.1 Classifications of Flow Diagrams 2.3.2 Main Visual Features 2.4 Summary 3 Related Work 3.1 Cross-tabulating Hierarchical Categories 3.1.1 Visualizing Categorical Aggregates of Item Sets 3.1.2 Hierarchical Visualization of Categorical Aggregates 3.1.3 Visualizing Item Sets and Their Hierarchical Properties 3.1.4 Hierarchical Visualization of Categorical Set Aggregates 3.2 Uncertainty Visualization 3.2.1 Uncertainty Taxonomies 3.2.2 Uncertainty in Flow Diagrams 3.3 Time-Series Data Visualization 3.3.1 Time & Data 3.3.2 User Tasks 3.3.3 Visual Representation 3.4 Summary ii Contents 4 Requirement Engineering Phase 4.1 Introduction 4.2 Environment 4.2.1 The Product 4.2.2 The Customers and Development Methodology 4.2.3 Lessons Learned 4.3 Visualization Requirements for Product Costing 4.3.1 Current Visualization Practice 4.3.2 Visualization Tasks 4.3.3 Data Structure and Size 4.3.4 Early Visualization Prototypes 4.3.5 Challenges and Lessons Learned 4.4 Data and Task Abstraction 4.4.1 Data Abstraction 4.4.2 Task Abstraction 4.5 Summary and Outlook 5 Parallel Hierarchies 5.1 Introduction 5.2 The Parallel Hierarchies Technique 5.2.1 The Individual Axis: Showing Hierarchical Categories 5.2.2 Two Interlinked Axes: Showing Pairwise Frequencies 5.2.3 Multiple Linked Axes: Propagating Frequencies 5.2.4 Fine-tuning Parallel Hierarchies through Reordering 5.3 Design Choices 5.4 Applying Parallel Hierarchies 5.4.1 US Census Data 5.4.2 Yeast Gene Ontology Annotations 5.5 Evaluation 5.5.1 Setup of the Evaluation 5.5.2 Procedure of the Evaluation 5.5.3 Results from the Evaluation 5.5.4 Validity of the Evaluation 5.6 Summary and Outlook 6 Visualizing Uncertainty in Flow Diagrams 6.1 Introduction 6.2 Uncertainty in Product Costing 6.2.1 Background 6.2.2 Main Causes of Bad Quality in Costing Data 6.3 Visualization Concepts 6.4 Uncertainty Visualization using Ribbons 6.4.1 Selected Visualization Techniques 6.4.2 Study Design and Procedure 6.4.3 Results 6.4.4 Discussion 6.5 Revised Visualization Approach using Ribbons 6.5.1 Application to Sankey Diagram 6.5.2 Application to Parallel Sets 6.5.3 Application to Parallel Hierarchies 6.6 Uncertainty Visualization using Nodes 6.6.1 Visual Design of Nodes 6.6.2 Expert Evaluation 6.7 Summary and Outlook 7 Visual Comparison Task 7.1 Introduction 7.2 Comparing Two One-dimensional Time Steps 7.2.1 Problem Statement 7.2.2 Visualization Design 7.3 Comparing Two N-dimensional Time Steps 7.4 Comparing Several One-dimensional Time Steps 7.5 Summary and Outlook 8 Parallel Hierarchies in Practice 8.1 Application to Plausibility Check Task 8.1.1 Plausibility Check Process 8.1.2 Visual Exploration of Machine Learning Results 8.2 Integration into SAP Analytics Cloud 8.2.1 SAP Analytics Cloud 8.2.2 Ocean to Table Project 8.3 Summary and Outlook 9 Validation 9.1 Introduction 9.2 Nested Model Validation Approach 9.3 Perceptual Validation of Visualization Techniques 9.3.1 Design Validation Table 9.3.2 Discussion 9.4 Summary and Outlook 10 Conclusion and Outlook 10.1 Summary of Findings 10.2 Discussion 10.3 Outlook A Questionnaires of the Evaluation B Survey of the Quality of Product Costing Data C Questionnaire of Current Practice Bibliograph

Casual Information Visualization on Exploring Spatiotemporal Data

The goal of this thesis is to study how the diverse data on the Web which are familiar to everyone can be visualized, and with a special consideration on their spatial and temporal information. We introduce novel approaches and visualization techniques dealing with different types of data contents: interactively browsing large amount of tags linking with geospace and time, navigating and locating spatiotemporal photos or videos in collections, and especially, providing visual supports for the exploration of diverse Web contents on arbitrary webpages in terms of augmented Web browsing

Advanced Visual Interfaces for Informed Decision-Making

This thesis presents new interactive visualization techniques and systems intended to support users with real-world decisions such as selecting a product from a large variety of similar offerings, finding appropriate wording as a non-native speaker, and assessing an alleged case of plagiarism. The Product Explorer is a significantly improved interactive Parallel Coordinates display for facilitating the product selection process in cases where many attributes and numerous alternatives have to be considered. A novel visual representation for categorical and ordered data with only few occurring values, the so-called extended areas, in combination with cubic curves for connecting the parallel axes, are crucial for providing an effective overview of the entire dataset and to facilitate the tracing of individual products. The visual query interface supports users in quickly narrowing down the product search to a small subset or even a single product. The scalability of the approach towards a large number of attributes and products is enhanced by the possibility of setting some constraints on final attributes and, therefore, reducing the number of considered attributes and data items. Furthermore, an attribute repository allows users to focus on the most important attributes at first and to bring in additional criteria for product selection later in the decision process. A user study confirmed that the Product Explorer is indeed an excellent tool for its intended purpose for casual users. The Wordgraph is a layered graph visualization for the interactive exploration of search results for complex keywords-in-context queries. The system relies on the Netspeak web service and is designed to support non-native speakers in finding customary phrases. Uncertainties about the commonness of phrases are expressed with the help of wildcard-based queries. The visualization presents the alternatives for the wildcards in a multi-column layout: one column per wildcard with the other query fragments in between. The Wordgraph visualization displays the sorted results for all wildcards at once by appropriately arranging the words of each column. A user study confirmed that this is a significant advantage over simple textual result lists. Furthermore, visual interfaces to filter, navigate, and expand the graph allow interactive refinement and expansion of wildcard-containing queries. Furthermore, this thesis presents an advanced visual analysis tool for assessing and presenting alleged cases of plagiarism and provides a three-level approach for exploring the so-called finding spots in their context. The overview shows the relationship of the entire suspicious document to the set of source documents. An intermediate glyph-based view reveals the structural and textual differences and similarities of a set of finding spots and their corresponding source text fragments. Eventually, the actual fragments of the finding spot can be shown in a side-by-side view with a novel structured wrapping of both the source, as well as the suspicious text. The three different levels of detail are tied together by versatile navigation and selection operations. Reviews with plagiarism experts confirm that this tool can effectively support their workflow and provides a significant improvement over existing static visualizations for assessing and presenting plagiarism cases. The three main contributions of this research have a lot in common aside from being carefully designed and scientifically grounded solutions to real-world decision problems. The first two visualizations facilitate the decision for a single possibility out of many alternatives, whereas the latter ones deal with text at varying levels of detail. All visual representations are clearly structured based on horizontal and vertical layers contained in a single view and they all employ edges for depicting the most important relationships between attributes, words, or different levels of detail. A detailed analysis considering the context of the established decision-making literature reveals that important steps of common decision models are well-supported by the three visualization systems presented in this thesis

A Systematic and Minimalist Approach to Lower Barriers in Visual Data Exploration

With the increasing availability and impact of data in our lives, we need to make quicker, more accurate, and intricate data-driven decisions. We can see and interact with data, and identify relevant features, trends, and outliers through visual data representations. In addition, the outcomes of data analysis reflect our cognitive processes, which are strongly influenced by the design of tools. To support visual and interactive data exploration, this thesis presents a systematic and minimalist approach. First, I present the Cognitive Exploration Framework, which identifies six distinct cognitive stages and provides a high-level structure to design guidelines, and evaluation of analysis tools. Next, in order to reduce decision-making complexities in creating effective interactive data visualizations, I present a minimal, yet expressive, model for tabular data using aggregated data summaries and linked selections. I demonstrate its application to common categorical, numerical, temporal, spatial, and set data types. Based on this model, I developed Keshif as an out-of-the-box, web-based tool to bootstrap the data exploration process. Then, I applied it to 160+ datasets across many domains, aiming to serve journalists, researchers, policy makers, businesses, and those tracking personal data. Using tools with novel designs and capabilities requires learning and help-seeking for both novices and experts. To provide self-service help for visual data interfaces, I present a data-driven contextual in-situ help system, HelpIn, which contrasts with separated and static videos and manuals. Lastly, I present an evaluation on design and graphical perception for dense visualization of sorted numeric data. I contrast the non-hierarchical treemaps against two multi-column chart designs, wrapped bars and piled bars. The results support that multi-column charts are perceptually more accurate than treemaps, and the unconventional piled bars may require more training to read effectively. This thesis contributes to our understanding on how to create effective data interfaces by systematically focusing on human-facing challenges through minimalist solutions. Future work to extend the power of data analysis to a broader public should continue to evaluate and improve design approaches to address many remaining cognitive, social, educational, and technical challenges

Visualization and Human-Machine Interaction

The digital age offers a lot of challenges in the eld of visualization. Visual imagery has been effectively used to communicate messages through the ages, to express both abstract and concrete ideas. Today, visualization has ever-expanding applications in science, engineering, education, medicine, entertainment and many other areas. Different areas of research contribute to the innovation in the eld of interactive visualization, such as data science, visual technology, Internet of things and many more. Among them, two areas of renowned importance are Augmented Reality and Visual Analytics. This thesis presents my research in the fields of visualization and human-machine interaction. The purpose of the proposed work is to investigate existing solutions in the area of Augmented Reality (AR) for maintenance. A smaller section of this thesis presents a minor research project on an equally important theme, Visual Analytics. Overall, the main goal is to identify the most important existing problems and then design and develop innovative solutions to address them. The maintenance application domain has been chosen since it is historically one of the first fields of application for Augmented Reality and it offers all the most common and important challenges that AR can arise, as described in chapter 2. Since one of the main problem in AR application deployment is reconfigurability of the application, a framework has been designed and developed that allows the user to create, deploy and update in real-time AR applications. Furthermore, the research focused on the problems related to hand-free interaction, thus investigating the area of speech-recognition interfaces and designing innovative solutions to address the problems of intuitiveness and robustness of the interface. On the other hand, the area of Visual Analytics has been investigated: among the different areas of research, multidimensional data visualization, similarly to AR, poses specific problems related to the interaction between the user and the machine. An analysis of the existing solutions has been carried out in order to identify their limitations and to point out possible improvements. Since this analysis delineates the scatterplot as a renowned visualization tool worthy of further research, different techniques for adapting its usage to multidimensional data are analyzed. A multidimensional scatterplot has been designed and developed in order to perform a comparison with another multidimensional visualization tool, the ScatterDice. The first chapters of my thesis describe my investigations in the area of Augmented Reality for maintenance. Chapter 1 provides definitions for the most important terms and an introduction to AR. The second chapter focuses on maintenance, depicting the motivations that led to choose this application domain. Moreover, the analysis concerning open problems and related works is described along with the methodology adopted to design and develop the proposed solutions. The third chapter illustrates how the adopted methodology has been applied in order to assess the problems described in the previous one. Chapter 4 describes the methodology adopted to carry out the tests and outlines the experimental results, whereas the fifth chapter illustrates the conclusions and points out possible future developments. Chapter 6 describes the analysis and research work performed in the eld of Visual Analytics, more specifically on multidimensional data visualizations. Overall, this thesis illustrates how the proposed solutions address common problems of visualization and human-machine interaction, such as interface de- sign, robustness of the interface and acceptance of new technology, whereas other problems are related to the specific research domain, such as pose tracking and reconfigurability of the procedure for the AR domain

Micro Visualizations: Design and Analysis of Visualizations for Small Display Spaces

The topic of this habilitation is the study of very small data visualizations, micro visualizations, in display contexts that can only dedicate minimal rendering space for data representations. For several years, together with my collaborators, I have been studying human perception, interaction, and analysis with micro visualizations in multiple contexts. In this document I bring together three of my research streams related to micro visualizations: data glyphs, where my joint research focused on studying the perception of small-multiple micro visualizations, word-scale visualizations, where my joint research focused on small visualizations embedded in text-documents, and small mobile data visualizations for smartwatches or fitness trackers. I consider these types of small visualizations together under the umbrella term ``micro visualizations.'' Micro visualizations are useful in multiple visualization contexts and I have been working towards a better understanding of the complexities involved in designing and using micro visualizations. Here, I define the term micro visualization, summarize my own and other past research and design guidelines and outline several design spaces for different types of micro visualizations based on some of the work I was involved in since my PhD.Le sujet de cette habilitation est l'étude de très petites visualisations de données, les micro visualisations, dans des contextes d'affichage qui ne peuvent consacrer qu'un espace de rendu minimal aux représentations de données. Depuis plusieurs années, avec mes collaborateurs, j'étudie la perception humaine, l'interaction et l'analyse conduite avec des micro visualisations dans de multiples contextes.Dans ce document, je rassemble trois de mes axes de recherche liés aux micro visualisations~: les glyphes de données, où ma recherche s'est concentrée sur l'étude de la perception de micro visualisations dans un context \textit{small-multiple}, les \textit{word-scale visualizations}, où ma recherche s'est concentrée sur les petites visualisations intégrées dans les documents textuels, et les petites visualisations de données mobiles pour les montres connectées. Je considère ces types de petites visualisations sous le terme générique de ``micro visualisations.'' Les micro visualisations sont utiles dans de multiples contextes de visualisation et j'ai travaillé à une meilleure compréhension de la complexité des conceptions et utilisations des micro visualisations. Je définirai ici le terme de micro visualisation, je résumerai mes propres recherches et celles d'autres chercheurs, ainsi que les directives de conception, et j'esquisserai plusieurs espaces de conception pour différents types de micro visualisations, sur la base de certains des travaux auxquels j'ai participé depuis mon doctorat

Taking shape: The data science of elastic shape analysis with practical applications

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London.A mathematical curve can represent many different objects, both physical and abstract, from the outline curve of an artefact in an image to the weight of growing animal to the set of frequencies used in a sound. Regardless of these variations, the curves can almost always vary non-linearly. One way to study shapes and their potential variations is elastic shape analysis, a rich theory of which has developed over the past twenty years. However, methods of elastic shape analysis are seldom utilized in practical applications on real-world data, especially outside of the mathematical shape analysis community. Our aim in this thesis is to explore some practical applications of elastic shape analysis. To do this, we work with various types of shape data, the majority of which are based on image datasets. As our focus is on two-dimensional curves, it is important to be able to robustly extract contours from images, before we can apply elastic shape analysis tools. In order to analyse the shapes in a dataset, we turn to methods of machine learning, to investigate the applications of elastic shape analysis in classification. In this thesis, we introduce an anthology of projects, in order to emphasise and under- stand the potential of elastic shape analysis in practical applications. There are four main projects in this thesis: (i) Classification of objects using outlines and the comparisons between methods of elastic shape analysis, geometric morphometrics, and human experts, with a focus on ancient Greek vases, (ii) Mussel species identification and a demonstra- tion that shape may not be enough in some applications, (iii) A novel tool to monitor the development of k ̄ak ̄ap ̄o chicks, and (iv) Classifying individual kiwi based on acoustic data from their calls. By combining tools from computer vision and machine learning with methods of elastic shape analysis, we introduce a practical framework for the application of elastic shape analysis, through a data science lens