Information Pooling Bias in Collaborative Cyber Forensics

abstract: Cyber threats are growing in number and sophistication making it important to continually study and improve all dimensions of cyber defense. Human teamwork in cyber defense analysis has been overlooked even though it has been identified as an important predictor of cyber defense performance. Also, to detect advanced forms of threats effective information sharing and collaboration between the cyber defense analysts becomes imperative. Therefore, through this dissertation work, I took a cognitive engineering approach to investigate and improve cyber defense teamwork. The approach involved investigating a plausible team-level bias called the information pooling bias in cyber defense analyst teams conducting the detection task that is part of forensics analysis through human-in-the-loop experimentation. The approach also involved developing agent-based models based on the experimental results to explore the cognitive underpinnings of this bias in human analysts. A prototype collaborative visualization tool was developed by considering the plausible cognitive limitations contributing to the bias to investigate whether a cognitive engineering-driven visualization tool can help mitigate the bias in comparison to off-the-shelf tools. It was found that participant teams conducting the collaborative detection tasks as part of forensics analysis, experience the information pooling bias affecting their performance. Results indicate that cognitive friendly visualizations can help mitigate the effect of this bias in cyber defense analysts. Agent-based modeling produced insights on internal cognitive processes that might be contributing to this bias which could be leveraged in building future visualizations. This work has multiple implications including the development of new knowledge about the science of cyber defense teamwork, a demonstration of the advantage of developing tools using a cognitive engineering approach, a demonstration of the advantage of using a hybrid cognitive engineering methodology to study teams in general and finally, a demonstration of the effect of effective teamwork on cyber defense performance.Dissertation/ThesisDoctoral Dissertation Applied Psychology 201

Improving Information Alignment and Distributed Coordination for Secure Information Supply Chains

Industries are constantly striving to incorporate the latest technology systems into their operations so that they can maintain a competitive edge in their respective markets. However, even when they are able to stay up to speed with technological advancement, there continues to be a gap between the workforce skill set and available technologies. Organizations may acquire advanced systems, yet end up spending extended periods of time in the implementation and deployment phases, resulting in lost resources and productivity. The primary focus of this research is on streamlining the implementation and integration of new information technology systems to avoid the dire consequences of the process being prolonged or inefficient. Specifically, the goal of this research is to mitigate business challenges in information sharing and availability for employees and managers interacting with business tools and each other. This was accomplished by first interviewing work professionals in order to identify gap parameters. Based on the interview findings, recommendations were made in order to enhance the usability of existing tools. At this point, the research setting was shifted from network operations to supply chain operations due to the restrictive nature of network operations. The research team succeeded in developing a user-centered methodology to implement and deploy new business systems to mitigate risk during integration of new systems as the transition is made from the classic way of performing tasks. While this methodology was studied in supply chain operations, it enabled the identification of a common trend of challenges in operations work settings, regardless of the business application. Hence the findings of this research can be extrapolated to any business setting, besides the ones actually studied by the team. In addition, this research ensures that operational teams are able to maximize their benefit out of the technology available, thus enabling them to keep up with the rapidly evolving world of technology while minimizing sacrifices in resources or productivity in the process

Decision Support Elements and Enabling Techniques to Achieve a Cyber Defence Situational Awareness Capability

[ES] La presente tesis doctoral realiza un análisis en detalle de los elementos de decisión necesarios para mejorar la comprensión de la situación en ciberdefensa con especial énfasis en la percepción y comprensión del analista de un centro de operaciones de ciberseguridad (SOC). Se proponen dos arquitecturas diferentes basadas en el análisis forense de flujos de datos (NF3). La primera arquitectura emplea técnicas de Ensemble Machine Learning mientras que la segunda es una variante de Machine Learning de mayor complejidad algorítmica (lambda-NF3) que ofrece un marco de defensa de mayor robustez frente a ataques adversarios. Ambas propuestas buscan automatizar de forma efectiva la detección de malware y su posterior gestión de incidentes mostrando unos resultados satisfactorios en aproximar lo que se ha denominado un SOC de próxima generación y de computación cognitiva (NGC2SOC). La supervisión y monitorización de eventos para la protección de las redes informáticas de una organización debe ir acompañada de técnicas de visualización. En este caso, la tesis aborda la generación de representaciones tridimensionales basadas en métricas orientadas a la misión y procedimientos que usan un sistema experto basado en lógica difusa. Precisamente, el estado del arte muestra serias deficiencias a la hora de implementar soluciones de ciberdefensa que reflejen la relevancia de la misión, los recursos y cometidos de una organización para una decisión mejor informada. El trabajo de investigación proporciona finalmente dos áreas claves para mejorar la toma de decisiones en ciberdefensa: un marco sólido y completo de verificación y validación para evaluar parámetros de soluciones y la elaboración de un conjunto de datos sintéticos que referencian unívocamente las fases de un ciberataque con los estándares Cyber Kill Chain y MITRE ATT & CK.[CA] La present tesi doctoral realitza una anàlisi detalladament dels elements de decisió necessaris per a millorar la comprensió de la situació en ciberdefensa amb especial èmfasi en la percepció i comprensió de l'analista d'un centre d'operacions de ciberseguretat (SOC). Es proposen dues arquitectures diferents basades en l'anàlisi forense de fluxos de dades (NF3). La primera arquitectura empra tècniques de Ensemble Machine Learning mentre que la segona és una variant de Machine Learning de major complexitat algorítmica (lambda-NF3) que ofereix un marc de defensa de major robustesa enfront d'atacs adversaris. Totes dues propostes busquen automatitzar de manera efectiva la detecció de malware i la seua posterior gestió d'incidents mostrant uns resultats satisfactoris a aproximar el que s'ha denominat un SOC de pròxima generació i de computació cognitiva (NGC2SOC). La supervisió i monitoratge d'esdeveniments per a la protecció de les xarxes informàtiques d'una organització ha d'anar acompanyada de tècniques de visualització. En aquest cas, la tesi aborda la generació de representacions tridimensionals basades en mètriques orientades a la missió i procediments que usen un sistema expert basat en lògica difusa. Precisament, l'estat de l'art mostra serioses deficiències a l'hora d'implementar solucions de ciberdefensa que reflectisquen la rellevància de la missió, els recursos i comeses d'una organització per a una decisió més ben informada. El treball de recerca proporciona finalment dues àrees claus per a millorar la presa de decisions en ciberdefensa: un marc sòlid i complet de verificació i validació per a avaluar paràmetres de solucions i l'elaboració d'un conjunt de dades sintètiques que referencien unívocament les fases d'un ciberatac amb els estàndards Cyber Kill Chain i MITRE ATT & CK.[EN] This doctoral thesis performs a detailed analysis of the decision elements necessary to improve the cyber defence situation awareness with a special emphasis on the perception and understanding of the analyst of a cybersecurity operations center (SOC). Two different architectures based on the network flow forensics of data streams (NF3) are proposed. The first architecture uses Ensemble Machine Learning techniques while the second is a variant of Machine Learning with greater algorithmic complexity (lambda-NF3) that offers a more robust defense framework against adversarial attacks. Both proposals seek to effectively automate the detection of malware and its subsequent incident management, showing satisfactory results in approximating what has been called a next generation cognitive computing SOC (NGC2SOC). The supervision and monitoring of events for the protection of an organisation's computer networks must be accompanied by visualisation techniques. In this case, the thesis addresses the representation of three-dimensional pictures based on mission oriented metrics and procedures that use an expert system based on fuzzy logic. Precisely, the state-of-the-art evidences serious deficiencies when it comes to implementing cyber defence solutions that consider the relevance of the mission, resources and tasks of an organisation for a better-informed decision. The research work finally provides two key areas to improve decision-making in cyber defence: a solid and complete verification and validation framework to evaluate solution parameters and the development of a synthetic dataset that univocally references the phases of a cyber-attack with the Cyber Kill Chain and MITRE ATT & CK standards.Llopis Sánchez, S. (2023). Decision Support Elements and Enabling Techniques to Achieve a Cyber Defence Situational Awareness Capability [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/19424

Designing a framework for data populating alarms based on MITRE techniques

In this paper we aim to develop a proof of concept framework as a step-by-step process for identifying what type of information and log types a SOC analyst needs to analyze and handle an alarm based on the alarms MITRE technique. To solve this, it was decided that using both theoretical and experimental research methodologies could be advantageous. Hence we first used a Systematic Literature Review to search, screen, and select relevant literature. Followed by the usage of Design Science Research method for conducting the research based upon a theoretical basis, and an experimental process. To develop a framework consisting of an easy to understand and independent step-by-step process. The proof of concept framework introduced in this paper, is an eight step process describing how one may proceed when gathering data needed for automating information gathering based on alarms MITRE techniques. In these eight steps it revolves around three main concepts, which are gathering a theoretical foundation by research and discussion, improving the theoretical foundation by testing and adjusting, and ends with a continuous process of maintaining the constructed automations when used in a production setting. This framework produced accurate results when tested during research, and we believe it should be further explored and tested in a larger scale. Also it should be considered a stepping stone into further automating the whole alarm handling process, from gathering data to response

Socialbots and the Challenges of Cyberspace Awareness

As security communities brace for the emerging social automation based threats, we examine the mechanisms of developing situation awareness in cyberspace and the governance issues that socialbots bring into this existing paradigm of cyber situation awareness. We point out that an organisation's situation awareness in cyberspace is a phenomena fundamentally distinct from the original conception of situation awareness, requiring continuous data exchange and knowledge management where the standard implementation mechanisms require significant policy attention in light of threats like malicious social automation. We conceptualise Cyberspace Awareness as a socio-technical phenomena with Syntactic, Semantic, and Operatic dimensions - each subject to a number of stressors which are exacerbated under social automation based threats. The paper contributes to the ideas of situational awareness in cyberspace, and characterises the challenges therein around tackling the increasingly social and often pervasive, automation in cyber threat environments

Designing a framework for data populating alarms based on mitre techniques

In this paper we aim to develop a proof of concept framework as a step-by-step process for identifying what type of information and log types a SOC analyst needs to analyze and handle an alarm based on the alarms MITRE technique. To solve this, it was decided that using both theoretical and experimental research methodologies could be advantageous. Hence we first used a Systematic Literature Review to search, screen, and select relevant literature. Followed by the usage of Design Science Research method for conducting the research based upon a theoretical basis, and an experimental process. To develop a framework consisting of an easy to understand and independent step-by-step process. The proof of concept framework introduced in this paper, is an eight step process describing how one may proceed when gathering data needed for automating information gathering based on alarms MITRE techniques. In these eight steps it revolves around three main concepts, which are gathering a theoretical foundation by research and discussion, improving the theoretical foundation by testing and adjusting, and ends with a continuous process of maintaining the constructed automations when used in a production setting. This framework produced accurate results when tested during research, and we believe it should be further explored and tested in a larger scale. Also it should be considered a stepping stone into further automating the whole alarm handling process, from gathering data to response

Artificial intelligence and UK national security: Policy considerations

RUSI was commissioned by GCHQ to conduct an independent research study into the use of artificial intelligence (AI) for national security purposes. The aim of this project is to establish an independent evidence base to inform future policy development regarding national security uses of AI. The findings are based on in-depth consultation with stakeholders from across the UK national security community, law enforcement agencies, private sector companies, academic and legal experts, and civil society representatives. This was complemented by a targeted review of existing literature on the topic of AI and national security. The research has found that AI offers numerous opportunities for the UK national security community to improve efficiency and effectiveness of existing processes. AI methods can rapidly derive insights from large, disparate datasets and identify connections that would otherwise go unnoticed by human operators. However, in the context of national security and the powers given to UK intelligence agencies, use of AI could give rise to additional privacy and human rights considerations which would need to be assessed within the existing legal and regulatory framework. For this reason, enhanced policy and guidance is needed to ensure the privacy and human rights implications of national security uses of AI are reviewed on an ongoing basis as new analysis methods are applied to data

Cyber-risks in the Industrial Internet of Things (IIoT): towards a method for continuous assessment.

Continuous risk monitoring is considered in the context of cybersecurity management for the Industrial Internet-of-Thing. Cyber risk management best practice is for security controls to be deployed and configured in order to bring down risk exposure to an acceptable level. However, threats and known vulnerabilities are subject to change, and estimates of risk are subject to many uncertainties, so it is important to review risk assessments and update controls when required. Risks are typically reviewed periodically (e.g. once per month), but the accelerating pace of change means that this approach is not sustainable, and there is a requirement for continuous monitoring of cybersecurity risks. The method described in this paper aims to alert security staff of significant changes or trends in estimated risk exposure to facilitate rational and timely decisions. Additionally, it helps predict the success and impact of a nascent security breach allowing better prioritisation of threats and selection of appropriate responses. The method is illustrated using a scenario based on environmental control in a data centre

Conceptual Model of Visual Analytics for Hands-on Cybersecurity Training

Hands-on training is an effective way to practice theoretical cybersecurity concepts and increase participants’ skills. In this paper, we discuss the application of visual analytics principles to the design, execution, and evaluation of training sessions. We propose a conceptual model employing visual analytics that supports the sensemaking activities of users involved in various phases of the training life cycle. The model emerged from our long-term experience in designing and organizing diverse hands-on cybersecurity training sessions. It provides a classification of visualizations and can be used as a framework for developing novel visualization tools supporting phases of the training life-cycle. We demonstrate the model application on examples covering two types of cybersecurity training programs