EEVi – framework for evaluating the effectiveness of visualization in cyber-security

Cyber-security visualization is an up-and-coming area which aims to reduce security analysts’ workload by presenting information as visual analytics rather than a string of text and characters. But the adoption of the resultant visualizations has not increased. The literature indicates a research gap of a lack of guidelines and standardized evaluation techniques for effective visualization in cyber-security, as a reason for it. Therefore, this research addresses the research gap by developing a framework called EEVi for effective cyber-security visualizations for the performed task. The term ‘effective visualization’ can be defined as the features of visualization that are crucial to perform a certain task successfully. EEVi has been developed by analyzing qualitative data that leads to the formation of cognitive relationships (called links) between data that act as guidelines for effective cyber-security visualization in terms of the performed task. The methodology to develop this framework can be applied to other fields to understand cognitive relationships between data. Additionally, the analysis presents a glimpse into the usage of EEVi in cyber-security visualization

Conceptual Model of Visual Analytics for Hands-on Cybersecurity Training

Hands-on training is an effective way to practice theoretical cybersecurity concepts and increase participants’ skills. In this paper, we discuss the application of visual analytics principles to the design, execution, and evaluation of training sessions. We propose a conceptual model employing visual analytics that supports the sensemaking activities of users involved in various phases of the training life cycle. The model emerged from our long-term experience in designing and organizing diverse hands-on cybersecurity training sessions. It provides a classification of visualizations and can be used as a framework for developing novel visualization tools supporting phases of the training life-cycle. We demonstrate the model application on examples covering two types of cybersecurity training programs

Retrospective on a Decade of Research in Visualization for Cybersecurity

Over the past decade, the visualization for cybersecurity (VizSec) research community has adapted many information visualization techniques to support the critical work of cyber analysts. While these efforts have yielded many specialized tools and platforms, the community lacks a unified approach to the design and implementation of these systems. In this work, we provide a retrospective analysis of the past decade of VizSec publications, with an eye toward developing a more cohesive understanding of the emerging patterns of design at work in our community. We identify common thematic groupings among existing work, as well as several interesting pat- terns of design around the utilization of various visual encodings. We also discuss existing gaps in the adaptation of information visualization techniques to cybersecurity applications, and recommend avenues for future exploration

User-Centered Design to Enhance IoT Cybersecurity Awareness of Non-Experts in Smart Buildings

Smart buildings, building automation and operational management have increasingly begun to incorporate Internet of Things (IoT) technology. Therefore, they have become susceptible to common cyber attacks targetting IoT devices. However, there is still a lack of an effective way of monitoring the cybersecurity situation of smart devices, IoT sensors and networks. During the operational lifecycle it may also not be easy for non-experts to discern cybersecurity issues from malfunctioning or physical safety. Therefore, we propose visualization prototypes that provide both safety and cybersecurity status of IoT devices for non-expert users in smart buildings. By utilising a user-centered design method, the visualization dashboards are developed based on requirements of two user roles - House managers and Residents. The user test results have shown the capabilities and effectiveness of leveraging dashboards to increase cybersecurity awareness in smart buildings.Peer reviewe

Human–Computer Interaction considerations when developing cyber ranges

The number of cyber-attacks are continuing to rise globally. It is therefore vital for organisations to develop the necessary skills to secure their assets and to protect critical national infrastructure. In this short paper, we outline human-computer interaction elements which should be considered when developing a cybersecurity training platform, in an effort to maintain levels of user engagement. We provide an overview of existing training platforms before covering specialist cyber ranges. Aspects of human-computer interaction are covered with regards to their relevance in the context of cyber ranges. We conclude with design suggestions when developing a cyber range platform

Doctor of Philosophy

dissertationThis dissertation establishes a new visualization design process model devised to guide visualization designers in building more effective and useful visualization systems and tools. The novelty of this framework includes its flexibility for iteration, actionability for guiding visualization designers with concrete steps, concise yet methodical definitions, and connections to other visualization design models commonly used in the field of data visualization. In summary, the design activity framework breaks down the visualization design process into a series of four design activities: understand, ideate, make, and deploy. For each activity, the framework prescribes a descriptive motivation, list of design methods, and expected visualization artifacts. To elucidate the framework, two case studies for visualization design illustrate these concepts, methods, and artifacts in real-world projects in the field of cybersecurity. For example, these projects employ user-centered design methods, such as personas and data sketches, which emphasize our teams' motivations and visualization artifacts with respect to the design activity framework. These case studies also serve as examples for novice visualization designers, and we hypothesized that the framework could serve as a pedagogical tool for teaching and guiding novices through their own design process to create a visualization tool. To externally evaluate the efficacy of this framework, we created worksheets for each design activity, outlining a series of concrete, tangible steps for novices. In order to validate the design worksheets, we conducted 13 student observations over the course of two months, received 32 online survey responses, and performed a qualitative analysis of 11 in-depth interviews. Students found the worksheets both useful and effective for framing the visualization design process. Next, by applying the design activity framework to technique-driven and evaluation-based research projects, we brainstormed possible extensions to the design model. Lastly, we examined implications of the design activity framework and present future work in this space. The visualization community is challenged to consider how to more effectively describe, capture, and communicate the complex, iterative nature of data visualization design throughout research, design, development, and deployment of visualization systems and tools

Novel Alert Visualization: The Development of a Visual Analytics Prototype for Mitigation of Malicious Insider Cyber Threats

Cyber insider threat is one of the most difficult risks to mitigate in organizations. However, innovative validated visualizations for cyber analysts to better decipher and react to detected anomalies has not been reported in literature or in industry. Attacks caused by malicious insiders can cause millions of dollars in losses to an organization. Though there have been advances in Intrusion Detection Systems (IDSs) over the last three decades, traditional IDSs do not specialize in anomaly identification caused by insiders. There is also a profuse amount of data being presented to cyber analysts when deciphering big data and reacting to data breach incidents using complex information systems. Information visualization is pertinent to the identification and mitigation of malicious cyber insider threats. The main goal of this study was to develop and validate, using Subject Matter Experts (SME), an executive insider threat dashboard visualization prototype. Using the developed prototype, an experimental study was conducted, which aimed to assess the perceived effectiveness in enhancing the analysts’ interface when complex data correlations are presented to mitigate malicious insiders cyber threats. Dashboard-based visualization techniques could be used to give full visibility of network progress and problems in real-time, especially within complex and stressful environments. For instance, in an Emergency Room (ER), there are four main vital signs used for urgent patient triage. Cybersecurity vital signs can give cyber analysts clear focal points during high severity issues. Pilots must expeditiously reference the Heads Up Display (HUD), which presents only key indicators to make critical decisions during unwarranted deviations or an immediate threat. Current dashboard-based visualization techniques have yet to be fully validated within the field of cybersecurity. This study developed a visualization prototype based on SME input utilizing the Delphi method. SMEs validated the perceived effectiveness of several different types of the developed visualization dashboard. Quantitative analysis of SME’s perceived effectiveness via self-reported value and satisfaction data as well as qualitative analysis of feedback provided during the experiments using the prototype developed were performed. This study identified critical cyber visualization variables and identified visualization techniques. The identifications were then used to develop QUICK.v™ a prototype to be used when mitigating potentially malicious cyber insider threats. The perceived effectiveness of QUICK.v™ was then validated. Insights from this study can aid organizations in enhancing cybersecurity dashboard visualizations by depicting only critical cybersecurity vital signs

Supporting social innovation through visualisations of community interactions

Online communities that form through the introduction of sociotechnical platforms require significant effort to cultivate and sustain. Providing open, transparent information on community behaviour can motivate participation from community members themselves, while also providing platform administrators with detailed interaction dynamics. However, challenges arise in both understanding what information is conducive to engagement and sustainability, and then how best to represent this information to platform stakeholders. Towards a better understanding of these challenges, we present the design, implementation, and evaluation of a set of simple visualisations integrated into a Collective Awareness Platform for Social Innovation platform titled commonfare.net. We discuss the promise and challenge of bringing social innovation into the digital age, in terms of supporting sustained platform use and collective action, and how the introduction of community visualisations has been directed towards achieving this goal

A Framework for Creative Visualization-Opportunities Workshops

Applied visualization researchers often work closely with domain collaborators to explore new and useful applications of visualization. The early stages of collaborations are typically time consuming for all stakeholders as researchers piece together an understanding of domain challenges from disparate discussions and meetings. A number of recent projects, however, report on the use of creative visualization-opportunities (CVO) workshops to accelerate the early stages of applied work, eliciting a wealth of requirements in a few days of focused work. Yet, there is no established guidance for how to use such workshops effectively. In this paper, we present the results of two-year collaboration in which we analyzed the use of 17 workshops in 10 visualization contexts. Its primary contribution is a framework for CVO workshops that 1) identifies a process model for using workshops; 2) describes a structure of what happens within effective workshops; 3) recommends 25 actionable guidelines for future workshops; and 4) presents an example workshop and workshop methods. The creation of this framework exemplifies the use of critical reflection to learn about visualization in practice from diverse studies and experience