A Task Analysis of Static Binary Reverse Engineering for Security

Software is ubiquitous in society, but understanding it, especially without access to source code, is both non-trivial and critical to security. A specialized group of cyber defenders conducts reverse engineering (RE) to analyze software. The expertise-driven process of software RE is not well understood, especially from the perspective of workflows and automated tools. We conducted a task analysis to explore the cognitive processes that analysts follow when using static techniques on binary code. Experienced analysts were asked to statically find a vulnerability in a small binary that could allow for unverified access to root privileges. Results show a highly iterative process with commonly used cognitive states across participants of varying expertise, but little standardization in process order and structure. A goal-centered analysis offers a different perspective about dominant RE states. We discuss implications about the nature of RE expertise and opportunities for new automation to assist analysts using static techniques

Supporting Advances in Human-Systems Coordination through Simulation of Diverse, Distributed Expertise

Distributed expertise task environments represent a critical, but challenging, area of team performance. As teams work together to perform complex tasks, they share much information and expertise to efficiently and effectively coordinate activities. Information coordination and alignment is affected by many factors, including communication styles and distributions of domain and interaction expertise. This study was part of a series of work performed in the authors’ lab to explore feasibility of using software simulation methods as a complement to other human factors methods to explore information alignment in teams. More specifically, this study aimed to operationalize specific parameters identified in group dynamics, management, and cognitive psychology literatures. Such research can provide an operationalized model that incorporates some of these key factors in information alignment and how these factors impact overall task performance of teams in complex environments. Simulation methods were applied to explore time-based performance outcomes. Model convergence and functionality were established through a series of model-based statistical analyses, which can be later validated with supplementary field studies. Results indicate that this style of simulation modeling is feasible, and provides directions for additional examination of factors affecting team configuration, process, and performance in complex systems

Identifying Expertise Gaps in Cyber Incident Response: Cyber Defender Needs vs. Technological Development

Incident response is an area within cyber defense that is responsible for detecting, mitigating, and preventing threats within a given network. Like other areas of cyber security, incident response is experiencing a shortage of qualified workers which has led to technological development aimed at alleviating labor-related pressures on organizations. A cognitive task analysis was conducted with incident response experts to capture expertise requirements and used an existing construct to help prioritize development of new technology. Findings indicated that current software development incorporates factors such as analyst efficiency and consistency. Gaps were identified regarding communication and team navigation that are inherent to dynamic team environments. This research identified which expertise areas are needed at lower-tier levels of incident response and which of those areas current automation platforms are addressing. These gaps help focus future studies by bridging expertise research to development efforts

Determining System Requirements for Human-Machine Integration in Cyber Security Incident Response

In 2019, cyber security is considered one of the most significant threats to the global economy and national security. Top U.S. agencies have acknowledged this fact, and provided direction regarding strategic priorities and future initiatives within the domain. However, there is still a lack of basic understanding of factors that impact complexity, scope, and effectiveness of cyber defense efforts. Computer security incident response is the short-term process of detecting, identifying, mitigating, and resolving a potential security threat to a network. These activities are typically conducted in computer security incident response teams (CSIRTs) comprised of human analysts that are organized into hierarchical tiers and work closely with many different computational tools and programs. Despite the fact that CSIRTs often provide the first line of defense to a network, there is currently a substantial global skills shortage of analysts to fill open positions. Research and development efforts from educational and technological perspectives have been independently ineffective at addressing this shortage due to time lags in meeting demand and associated costs. This dissertation explored how to combine the two approaches by considering how human-centered research can inform development of computational solutions toward augmenting human analyst capabilities. The larger goal of combining these approaches is to effectively complement human expertise with technological capability to alleviate pressures from the skills shortage. Insights and design recommendations for hybrid systems to advance the current state of security automation were developed through three studies. The first study was an ethnographic field study which focused on collecting and analyzing contextual data from three diverse CSIRTs from different sectors; the scope extended beyond individual incident response tasks to include aspects of organization and information sharing within teams. Analysis revealed larger design implications regarding collaboration and coordination in different team environments, as well as considerations about usefulness and adoption of automation. The second study was a cognitive task analysis with CSIR experts with diverse backgrounds; the interviews focused on expertise requirements for information sharing tasks in CSIRTs. Outputs utilized a dimensional expertise construct to identify and prioritize potential expertise areas for augmentation with automated tools and features. Study 3 included a market analysis of current automation platforms based on the expertise areas identified in Study 2, and used Systems Engineering methodologies to develop concepts and functional architectures for future system (and feature) development. Findings of all three studies support future directions for hybrid automation development in CSIR by identifying social and organizational factors beyond traditional tool design in security that supports human-systems integration. Additionally, this dissertation delivered functional considerations for automated technology that can augment human capabilities in incident response; these functions support better information sharing between humans and between humans and technological systems. By pursuing human-systems integration in CSIR, research can help alleviate the skills shortage by identifying where automation can dynamically assist with information sharing and expertise development. Future research can expand upon the expertise framework developed for CSIR and extend the application of proposed augmenting functions in other domains

Supporting Advances in Human-Systems Coordination through Simulation of Diverse, Distributed Expertise

Distributed expertise task environments represent a critical, but challenging, area of team performance. As teams work together to perform complex tasks, they share much information and expertise to efficiently and effectively coordinate activities. Information coordination and alignment is affected by many factors, including communication styles and distributions of domain and interaction expertise. This study was part of a series of work performed in the authors’ lab to explore feasibility of using software simulation methods as a complement to other human factors methods to explore information alignment in teams. More specifically, this study aimed to operationalize specific parameters identified in group dynamics, management, and cognitive psychology literatures. Such research can provide an operationalized model that incorporates some of these key factors in information alignment and how these factors impact overall task performance of teams in complex environments. Simulation methods were applied to explore time-based performance outcomes. Model convergence and functionality were established through a series of model-based statistical analyses, which can be later validated with supplementary field studies. Results indicate that this style of simulation modeling is feasible, and provides directions for additional examination of factors affecting team configuration, process, and performance in complex systems