Improving Cyber Situational Awareness via Data mining and Predictive Analytic Techniques

As cyber-attacks have become more common in everyday life, there is a need for maintaining and improving cyber security standards in any business or industry. Cyber Situational Awareness (CSA) is a broad strategy which can be adopted by any business or government to tackle cyber-attacks and incidents. CSA is based on current and past incidents, elements and actors in any system. Managers and decision makers need to monitor their systems constantly to understand ongoing events and changes which it can lead to predict future incidents. Prediction of future cyber incidents then can guide cyber managers to be prepared against future cyber threats and breaches. This research aims to improve cyber situational awareness by developing a framework based on data mining techniques specifically classification methods known as predictive approaches and Open Source Intelligence (OSINT). OSINT is another important element in this research because not only it is accessible publicly but also it is cost effective and research friendly. This research highlights the importance of understanding past and current CSA, which it can lead to more preparation against future cyber threats, and cyber security experts can use the developed framework with other different methods and provide a comprehensive strategy to improve cyber security and safety

Adaptive Traffic Fingerprinting for Darknet Threat Intelligence

Darknet technology such as Tor has been used by various threat actors for organising illegal activities and data exfiltration. As such, there is a case for organisations to block such traffic, or to try and identify when it is used and for what purposes. However, anonymity in cyberspace has always been a domain of conflicting interests. While it gives enough power to nefarious actors to masquerade their illegal activities, it is also the cornerstone to facilitate freedom of speech and privacy. We present a proof of concept for a novel algorithm that could form the fundamental pillar of a darknet-capable Cyber Threat Intelligence platform. The solution can reduce anonymity of users of Tor, and considers the existing visibility of network traffic before optionally initiating targeted or widespread BGP interception. In combination with server HTTP response manipulation, the algorithm attempts to reduce the candidate data set to eliminate client-side traffic that is most unlikely to be responsible for server-side connections of interest. Our test results show that MITM manipulated server responses lead to expected changes received by the Tor client. Using simulation data generated by shadow, we show that the detection scheme is effective with false positive rate of 0.001, while sensitivity detecting non-targets was 0.016+-0.127. Our algorithm could assist collaborating organisations willing to share their threat intelligence or cooperate during investigations.Comment: 26 page

Probabilistic Modeling and Inference for Obfuscated Network Attack Sequences

Prevalent computing devices with networking capabilities have become critical network infrastructure for government, industry, academia and every-day life. As their value rises, the motivation driving network attacks on this infrastructure has shifted from the pursuit of notoriety to the pursuit of profit or political gains, leading to network attack on various scales. Facing diverse network attack strategies and overwhelming alters, much work has been devoted to correlate observed malicious events to pre-defined scenarios, attempting to deduce the attack plans based on expert models of how network attacks may transpire. We started the exploration of characterizing network attacks by investigating how temporal and spatial features of attack sequence can be used to describe different types of attack sources in real data set. Attack sequence models were built from real data set to describe different attack strategies. Based on the probabilistic attack sequence model, attack predictions were made to actively predict next possible actions. Experiments through attack predictions have revealed that sophisticated attackers can employ a number of obfuscation techniques to confuse the alert correlation engine or classifier. Unfortunately, most exiting work treats attack obfuscations by developing ad-hoc fixes to specific obfuscation technique. To this end, we developed an attack modeling framework that enables a systematical analysis of obfuscations. The proposed framework represents network attack strategies as general finite order Markov models and integrates it with different attack obfuscation models to form probabilistic graphical model models. A set of algorithms is developed to inference the network attack strategies given the models and the observed sequences, which are likely to be obfuscated. The algorithms enable an efficient analysis of the impact of different obfuscation techniques and attack strategies, by determining the expected classification accuracy of the obfuscated sequences. The algorithms are developed by integrating the recursion concept in dynamic programming and the Monte-Carlo method. The primary contributions of this work include the development of the formal framework and the algorithms to evaluate the impact of attack obfuscations. Several knowledge-driven attack obfuscation models are developed and analyzed to demonstrate the impact of different types of commonly used obfuscation techniques. The framework and algorithms developed in this work can also be applied to other contexts beyond network security. Any behavior sequences that might suffer from noise and require matching to pre-defined models can use this work to recover the most likely original sequence or evaluate quantitatively the expected classification accuracy one can achieve to separate the sequences

R-CAD: Rare Cyber Alert Signature Relationship Extraction Through Temporal Based Learning

The large number of streaming intrusion alerts make it challenging for security analysts to quickly identify attack patterns. This is especially difficult since critical alerts often occur too rarely for traditional pattern mining algorithms to be effective. Recognizing the attack speed as an inherent indicator of differing cyber attacks, this work aggregates alerts into attack episodes that have distinct attack speeds, and finds attack actions regularly co-occurring within the same episode. This enables a novel use of the constrained SPADE temporal pattern mining algorithm to extract consistent co-occurrences of alert signatures that are indicative of attack actions that follow each other. The proposed Rare yet Co-occurring Attack action Discovery (R-CAD) system extracts not only the co-occurring patterns but also the temporal characteristics of the co-occurrences, giving the `strong rules\u27 indicative of critical and repeated attack behaviors. Through the use of a real-world dataset, we demonstrate that R-CAD helps reduce the overwhelming volume and variety of intrusion alerts to a manageable set of co-occurring strong rules. We show specific rules that reveal how critical attack actions follow one another and in what attack speed

Artificial intelligence and UK national security: Policy considerations

RUSI was commissioned by GCHQ to conduct an independent research study into the use of artificial intelligence (AI) for national security purposes. The aim of this project is to establish an independent evidence base to inform future policy development regarding national security uses of AI. The findings are based on in-depth consultation with stakeholders from across the UK national security community, law enforcement agencies, private sector companies, academic and legal experts, and civil society representatives. This was complemented by a targeted review of existing literature on the topic of AI and national security. The research has found that AI offers numerous opportunities for the UK national security community to improve efficiency and effectiveness of existing processes. AI methods can rapidly derive insights from large, disparate datasets and identify connections that would otherwise go unnoticed by human operators. However, in the context of national security and the powers given to UK intelligence agencies, use of AI could give rise to additional privacy and human rights considerations which would need to be assessed within the existing legal and regulatory framework. For this reason, enhanced policy and guidance is needed to ensure the privacy and human rights implications of national security uses of AI are reviewed on an ongoing basis as new analysis methods are applied to data

Advanced persistent threats detection based on deep learning approach.

Advanced Persistent Threats (APTs) have been a major challenge in securing both Information Technology (IT) and Operational Technology (OT) systems. APT is a sophisticated attack that masquerade their actions to navigates around defenses, breach networks, often, over multiple network hosts and evades detection. It also uses "low-and-slow" approach over a long period of time. Resource availability, integrity, and confidentiality of the operational cyber-physical systems (CPS) state and control is highly impacted by the safety and security measures in place. A framework multi-stage detection approach termed "APTDASAC" to detect different tactics, techniques, and procedures (TTPs) used during various APT steps is proposed. Implementation was carried out in three stages: (i) Data input and probing layer - this involves data gathering and preprocessing, (ii) Data analysis layer; applies the core process of "APTDASAC" to learn the behaviour of attack steps from the sequence data, correlate and link the related output and, (iii) Decision layer; the ensemble probability approach is utilized to integrate the output and make attack prediction. The framework was validated with three different datasets and three case studies. The proposed approach achieved a significant attacks detection capability of 86.36% with loss as 0.32%, demonstrating that attack detection techniques applied that performed well in one domain may not yield the same good result in another domain. This suggests that robustness and resilience of operational systems state to withstand attack and maintain system performance are regulated by the safety and security measures in place, which is specific to the system in question