Visualisation of Large-Scale Call-Centre Data

The contact centre industry employs 4% of the entire United King-dom and United States’ working population and generates gigabytes of operational data that require analysis, to provide insight and to improve efficiency. This thesis is the result of a collaboration with QPC Limited who provide data collection and analysis products for call centres. They provided a large data-set featuring almost 5 million calls to be analysed. This thesis utilises novel visualisation techniques to create tools for the exploration of the large, complex call centre data-set and to facilitate unique observations into the data.A survey of information visualisation books is presented, provid-ing a thorough background of the field. Following this, a feature-rich application that visualises large call centre data sets using scatterplots that support millions of points is presented. The application utilises both the CPU and GPU acceleration for processing and filtering and is exhibited with millions of call events.This is expanded upon with the use of glyphs to depict agent behaviour in a call centre. A technique is developed to cluster over-lapping glyphs into a single parent glyph dependant on zoom level and a customizable distance metric. This hierarchical glyph repre-sents the mean value of all child agent glyphs, removing overlap and reducing visual clutter. A novel technique for visualising individually tailored glyphs using a Graphics Processing Unit is also presented, and demonstrated rendering over 100,000 glyphs at interactive frame rates. An open-source code example is provided for reproducibility.Finally, a novel interaction and layout method is introduced for improving the scalability of chord diagrams to visualise call transfers. An exploration of sketch-based methods for showing multiple links and direction is made, and a sketch-based brushing technique for filtering is proposed. Feedback from domain experts in the call centre industry is reported for all applications developed

Doctor of Philosophy

dissertationCorrelation is a powerful relationship measure used in many fields to estimate trends and make forecasts. When the data are complex, large, and high dimensional, correlation identification is challenging. Several visualization methods have been proposed to solve these problems, but they all have limitations in accuracy, speed, or scalability. In this dissertation, we propose a methodology that provides new visual designs that show details when possible and aggregates when necessary, along with robust interactive mechanisms that together enable quick identification and investigation of meaningful relationships in large and high-dimensional data. We propose four techniques using this methodology. Depending on data size and dimensionality, the most appropriate visualization technique can be provided to optimize the analysis performance. First, to improve correlation identification tasks between two dimensions, we propose a new correlation task-specific visualization method called correlation coordinate plot (CCP). CCP transforms data into a powerful coordinate system for estimating the direction and strength of correlations among dimensions. Next, we propose three visualization designs to optimize correlation identification tasks in large and multidimensional data. The first is snowflake visualization (Snowflake), a focus+context layout for exploring all pairwise correlations. The next proposed design is a new interactive design for representing and exploring data relationships in parallel coordinate plots (PCPs) for large data, called data scalable parallel coordinate plots (DSPCP). Finally, we propose a novel technique for storing and accessing the multiway dependencies through visualization (MultiDepViz). We evaluate these approaches by using various use cases, compare them to prior work, and generate user studies to demonstrate how our proposed approaches help users explore correlation in large data efficiently. Our results confirmed that CCP/Snowflake, DSPCP, and MultiDepViz methods outperform some current visualization techniques such as scatterplots (SCPs), PCPs, SCP matrix, Corrgram, Angular Histogram, and UntangleMap in both accuracy and timing. Finally, these approaches are applied in real-world applications such as a debugging tool, large-scale code performance data, and large-scale climate data

Interactive Visualization and Exploration of High-Dimensional Data

Visualizing data is an essential part of good statistical practice. Plots are useful for revealing structure in the data, checking model assumptions, detecting outliers and finding unanticipated patterns. Post-analysis visualization is commonly used to communicate the results of statistical analyses. The availability of good statistical visualization software is key in effectively performing data analysis and in exploring and developing new methods for data visualization. Compared to static visualization, interactive visualization adds natural and powerful ways to explore the data. With interactive visualization an analyst can dive into the data and quickly react to visual clues by, for example, re-focusing and creating interactive queries of the data. Further, linking visual attributes of the data points such as color and size allows the analyst to compare different visual representations of the data such as histograms and scatterplots. In this thesis, we explore and develop new interactive data visualization and exploration tools for high-dimensional data. The original focus of our research was a software implementation of navigation graphs. Navigation graphs are navigational infrastructures for controlled exploration of high-dimensional data. As part of this thesis, we developed the first interactive implementation of these navigation graphs called RnavGraph. With RnavGraph we explored various features for enhancing the usability of navigation graphs. We concluded that a powerful interactive scatterplot display and methods to deal with large graphs were two areas that would add great value to the navigation graph framework. RnavGraph's scatterplot display proved to be particularly useful for data analysis and we continued our research with the design and implementation of a general-purpose interactive visualization toolkit called loon. The core contributions of loon are as follows. loon implements a general design for interactive statistical graphic displays that supports layering of visual information such as point objects, lines and polygons. These displays further support zooming, panning and selection, and modification and deactivation of plot elements and layers. Interactions with plots are provided with mouse and keyboard gestures as well as via command line control and with inspectors. These inspectors provide graphical user interfaces for modifying and overseeing the plots. loon also implements a novel dynamic linking mechanism that can be used to assign the plots that are to be linked and the linking rules at run time. Additionally, loon's design provides several different types of event bindings to add and customize functionality of loon's displays. In this thesis, we discuss loon's design and framework by giving concrete examples that show how these design choices can be used to efficiently explore and visualize data interactively. These examples revolve around loon's statistical interactive displays such as histograms, scatterplots and graph displays. We also illustrate how loon's design can be used to layer on plots relevant statistical information and model fits such as density estimates, contours, regression lines and geographical maps for spatial data analysis. loon is implemented in Tcl and Tk and we explore the integration of loon's framework into a complete statistical computing environment such as R. We show examples of statistical analyses performed in R that are enhanced with interactivity using loon. loon also implements a number of new tools for high-dimensional data exploration. The serialaxes display represents the data using parallel or radial coordinates. The scatterplot display supports high-dimensional point glyphs such as serialaxes glyphs, polygons and images. loon's navigation graphs allow for multiple navigators and for direct manipulation of a graph which includes deactivating nodes and their adjoining edges. To deal with large graphs, we propose and implement environments for creating navigation graphs interactively by filtering the nodes with respect to some node-associated relevant measures. Such measures include the correlation of variable pairs and the graph-based scagnostics measures. We use sliders, histograms and scatterplot matrices to interactively filter the nodes based on the value of their associated measure. Measures are kept generic and can be recalculated for the subset of selected data points. As another tool for exploring high-dimensional data, we introduce a setup that allows the analyst to select points and have their k-nearest neighboring points highlighted automatically. The space to calculate the inter-point distances that determine the k-nearest neighbors can be chosen dynamically. Finally, we propose a new high-dimensional point glyph called the spiro glyph. While some of loon's interaction features have appeared in part or in whole in statistical systems in the past 40 years (e.g. brushing, panning, zooming, linking plots, etc.), no other equally rich system has provided (or continues to provide) an interactive data visualization system integrated with a widely available and stable statistical system like R. Both Tcl and R are well suited for rapid prototyping of software and statistical methods; with loon rapid prototyping of interactive data visualization tools and methods become possible as well

Text in Visualization: Extending the Visualization Design Space

This thesis is a systematic exploration and expansion of the design space of data visualization specifically with regards to text. A critical analysis of text in data visualizations reveals gaps in existing frameworks and the use of text in practice. A cross-disciplinary review across fields such as typography, cartography and technical applications yields typographic techniques to encode data into text and provides the scope for the expanded design space. Mapping new attributes, techniques and considerations back to well understood visualization principles organizes the design space of text in visualization. This design space includes: 1) text as a primary data type literally encoded into alphanumeric glyphs, 2) typographic attributes, such as bold and italic, capable of encoding additional data onto literal text, 3) scope of mark, ranging from individual glyphs, syllables and words; to sentences, paragraphs and documents, and 4) layout of these text elements applicable most known visualization techniques and text specific techniques such as tables. This is the primary contribution of this thesis (Part A and B). Then, this design space is used to facilitate the design, implementation and evaluation of new types of visualization techniques, ranging from enhancements of existing techniques, such as, extending scatterplots and graphs with literal marks, stem & leaf plots with multivariate glyphs and broader scope, and microtext line charts; to new visualization techniques, such as, multivariate typographic thematic maps; text formatted to facilitate skimming; and proportionally encoding quantitative values in running text – all of which are new contributions to the field (Part C). Finally, a broad evaluation across the framework and the sample visualizations with cross-discipline expert critiques and a metrics based approach reveals some concerns and many opportunities pointing towards a breadth of future research work now possible with this new framework. (Part D and E)

A Survey of Information Visualization Books

Information visualization is a rapidly evolving field with a growing volume of scientific literature and texts continually published.To keep abreast of the latest developments in the domain, survey papers and state-of-the-art reviews provide valuable tools formanaging the large quantity of scientific literature. Recently a survey of survey papers (SoS) was published to keep track ofthe quantity of refereed survey papers in information visualization conferences and journals. However no such resources existto inform readers of the large volume of books being published on the subject, leaving the possibility of valuable knowledgebeing overlooked. We present the first literature survey of information visualization books that addresses this challenge bysurveying the large volume of books on the topic of information visualization and visual analytics. This unique survey addressessome special challenges associated with collections of books (as opposed to research papers) including searching, browsingand cost. This paper features a novel two-level classification based on both books and chapter topics examined in each book,enabling the reader to quickly identify to what depth a topic of interest is covered within a particular book. Readers can usethis survey to identify the most relevant book for their needs amongst a quickly expanding collection. In indexing the landscapeof information visualization books, this survey provides a valuable resource to both experienced researchers and newcomers inthe data visualization discipline

Interactive and Static Statistical Graphics: Bridge to Integration

There are plenty of graphical packages in R which play an important role in building graphics for data analysis, either static graphics (e.g., `graphics`, `grid`, `ggplot2`) or interactive graphics (e.g., `loon`, `shiny`). Each of them has certain strengths and weaknesses. Typically, analysts only use one graphical system at a time during data analysis. However, it may not be sufficient in some circumstances. To better achieve goals, analysts sometimes need more than one graphical packages. For example, an analyst aims to use interactive plots to uncover patterns of interest in data exploration, in which case, a web-based app or an animation could better deliver the analysis dynamically in the presentation. Unfortunately, due to the dissimilarity of the design, data analysis using multiple graphical systems could be too complicated to accomplish. To simplify the process, the idea of ``bridge'' is introduced. A bridge is a peer to peer transformation and works as a connection to map elements (i.e., visual display or visual structure) from one graphical system to another. Usually, the difficulty level of building a bridge mainly depends on how well the abstraction level can be matched. In this thesis, we mainly focus on four packages. The graphical system `loon` provides interactive visualization toolkit for data exploration. The package `ggplot2` offers tools to extend the flexibility of drawing static plots in data analysis based upon a grammar of graphics. The package `grid` is a core graphical system in R, providing low-level, general purpose graphics functions. The package `shiny` provides interactive web applications in R. To integrate the strengths of each, three bridges are introduced: bridge `loon.ggplot` is to transform a `loon` widget to a `ggplot` object, or backwards; bridge `loonGrob` is to turn a `loon` widget to a static `grid` graphic; bridge `loon.shiny` is to render a `loon` widget into a `shiny` web app. In addition, a new package `loon.tourr` is also discussed. Even though it is not a bridge, it could be useful to help find interesting lower projections from a high dimensional subspace in an interactive way

Visually Mining Interesting Patterns in Multivariate Datasets

Data mining for patterns and knowledge discovery in multivariate datasets are very important processes and tasks to help analysts understand the dataset, describe the dataset, and predict unknown data values. However, conventional computer-supported data mining approaches often limit the user from getting involved in the mining process and performing interactions during the pattern discovery. Besides, without the visual representation of the extracted knowledge, the analysts can have difficulty explaining and understanding the patterns. Therefore, instead of directly applying automatic data mining techniques, it is necessary to develop appropriate techniques and visualization systems that allow users to interactively perform knowledge discovery, visually examine the patterns, adjust the parameters, and discover more interesting patterns based on their requirements. In the dissertation, I will discuss different proposed visualization systems to assist analysts in mining patterns and discovering knowledge in multivariate datasets, including the design, implementation, and the evaluation. Three types of different patterns are proposed and discussed, including trends, clusters of subgroups, and local patterns. For trend discovery, the parameter space is visualized to allow the user to visually examine the space and find where good linear patterns exist. For cluster discovery, the user is able to interactively set the query range on a target attribute, and retrieve all the sub-regions that satisfy the user\u27s requirements. The sub-regions that satisfy the same query and are neareach other are grouped and aggregated to form clusters. For local pattern discovery, the patterns for the local sub-region with a focal point and its neighbors are computationally extracted and visually represented. To discover interesting local neighbors, the extracted local patterns are integrated and visually shown to the analysts. Evaluations of the three visualization systems using formal user studies are also performed and discussed

Visualizing Spatio-Temporal data

The amount of spatio-temporal data produced everyday has sky rocketed in the recent years due to the commercial GPS systems and smart devices. Together with this, the need for tools and techniques to analyze this kind of data have also increased. A major task of spatio-temporal data analysis is to discover relationships and patterns among spatially and temporally scattered events. However, most of the existing visualization techniques implement a top-down approach i.e, they require prior knowledge of existing patterns. In this dissertation, I present my novel visualization technique called Storygraph which supports bottom-up discovery of patterns. Since Storygraph presents and integrated view, analysis of events can be done with losing either of time or spatial contexts. In addition, Storygraph can handle spatio-temporal uncertainty making it ideal for data being extracted from text. In the subsequent chapters, I demonstrate the versatility and the effectiveness of the Storygraph along with case studies from my published works. Finally, I also talk about edge bundling in Storygraph to enhance the aesthetics and improve the readability of Storygraph