ATLANTIDES: Automatic Configuration for Alert Verification in Network Intrusion Detection Systems

We present an architecture designed for alert verification (i.e., to reduce false positives) in network intrusion-detection systems. Our technique is based on a systematic (and automatic) anomaly-based analysis of the system output, which provides useful context information regarding the network services. The false positives raised by the NIDS analyzing the incoming traffic (which can be either signature- or anomaly-based) are reduced by correlating them with the output anomalies. We designed our architecture for TCP-based network services which have a client/server architecture (such as HTTP). Benchmarks show a substantial reduction of false positives between 50% and 100%

Intrusion Alert Correlation Technique Analysis for Heterogeneous Log

Intrusion alert correlation is multi-step processes that receives alerts from heterogeneous log resources as input and produce a high-level description of the malicious activity on the network. The objective of this study is to analyse the current alert correlation technique and identify the significant criteria in each technique that can improve the Intrusion Detection System(IDS) problem such as prone to alert flooding, contextual problem, false alert and scalability. The existing alert correlation techniques had been reviewed and analysed. From the analysis, six capability criteria have been identified to improve the current alert correlation technique. They are capability to do alert reduction, alert clustering,identify multistep attack, reduce false alert, detect known attack and detect unknown attack

APHRODITE: an Anomaly-based Architecture for False Positive Reduction

We present APHRODITE, an architecture designed to reduce false positives in network intrusion detection systems. APHRODITE works by detecting anomalies in the output traffic, and by correlating them with the alerts raised by the NIDS working on the input traffic. Benchmarks show a substantial reduction of false positives and that APHRODITE is effective also after a "quick setup", i.e. in the realistic case in which it has not been "trained" and set up optimall

Komponenten für kooperative Intrusion-Detection in dynamischen Koalitionsumgebungen

Koalitionsumgebungen sollen für alle miteinander kooperierenden Mitglieder einen Vorteil bei der Verfolgung eines gemeinsamen Ziels erbringen. Dies gilt für die verschiedensten Anwendungsbereiche, etwa bei kooperierenden Strafverfolgungsbehörden, Wirtschaftsunternehmen oder Streitkräfte. Auch bei der Erkennung von sicherheitsrelevanten Vorgängen in vernetzten Computersystemen erhofft man sich von der Zusammenarbeit eine verbesserte Erkennungsfähigkeit sowie eine schnelle und koordinierte Reaktion auf Einbruchsversuche. Dieser Beitrag stellt verschiedene praxisorientierte Werkzeuge für die koalitionsweite Vernetzung von Ereignismeldungs-produzierenden Sicherheitswerkzeugen vor, die wesentliche Probleme des Anwendungsszenarios lösen helfen: Frühzeitige Anomaliewarnung – ein graphbasierter Anomaliedetektor wird als adaptives Frühwarnmodul für großflächige und koordinierte Angriffe, z.B. Internet-Würmer, eingesetzt. Informationsfilterung – Meldungen werden beim Verlassen der lokalen Domäne entsprechend der domänenspezifischen Richtlinien zur Informationsweitergabe modifiziert (d.h. insbesondere anonymisiert bzw. pseudonymisiert). Datenreduktion – zusätzliche Filter zur Datenreduzierung auf der Basis von vordefinierten Abhängigkeitsregeln steigern die Handhabbarkeit des Datenflusses. Die Funktionsfähigkeit der genannten Komponenten wird derzeit in Form einer prototypischen Implementierung eines Meta-IDS für dynamische Koalitionsumgebungen nachgewiesen

A lightweight intrusion alert fusion system

In this paper, we present some practical experience on implementing an alert fusion mechanism from our project. After investigation on most of the existing alert fusion systems, we found the current body of work alternatively weighed down in the mire of insecure design or rarely deployed because of their complexity. As confirmed by our experimental analysis, unsuitable mechanisms could easily be submerged by an abundance of useless alerts. Even with the use of methods that achieve a high fusion rate and low false positives, attack is also possible. To find the solution, we carried out analysis on a series of alerts generated by well-known datasets as well as realistic alerts from the Australian Honey-Pot. One important finding is that one alert has more than an 85% chance of being fused in the following 5 alerts. Of particular importance is our design of a novel lightweight Cache-based Alert Fusion Scheme, called CAFS. CAFS has the capacity to not only reduce the quantity of useless alerts generated by IDS (Intrusion Detection System), but also enhance the accuracy of alerts, therefore greatly reducing the cost of fusion processing. We also present reasonable and practical specifications for the target-oriented fusion policy that provides a quality guarantee on alert fusion, and as a result seamlessly satisfies the process of successive correlation. Our experimental results showed that the CAFS easily attained the desired level of survivable, inescapable alert fusion design. Furthermore, as a lightweight scheme, CAFS can easily be deployed and excel in a large amount of alert fusions, which go towards improving the usability of system resources. To the best of our knowledge, our work is a novel exploration in addressing these problems from a survivable, inescapable and deployable point of view

HeAT PATRL: Network-Agnostic Cyber Attack Campaign Triage With Pseudo-Active Transfer Learning

SOC (Security Operation Center) analysts historically struggled to keep up with the growing sophistication and daily prevalence of cyber attackers. To aid in the detection of cyber threats, many tools like IDS’s (Intrusion Detection Systems) are utilized to monitor cyber threats on a network. However, a common problem with these tools is the volume of the logs generated is extreme and does not stop, further increasing the chance for an adversary to go unnoticed until it’s too late. Typically, the initial evidence of an attack is not an isolated event but a part of a larger attack campaign describing prior events that the attacker took to reach their final goal. If an analyst can quickly identify each step of an attack campaign, a timely response can be made to limit the impact of the attack or future attacks. In this work, we ask the question “Given IDS alerts, can we extract out the cyber-attack kill chain for an observed threat that is meaningful to the analyst?” We present HeAT-PATRL, an IDS attack campaign extractor that leverages multiple deep machine learning techniques, network-agnostic feature engineering, and the analyst’s knowledge of potential threats to extract out cyber-attack campaigns from IDS alert logs. HeAT-PATRL is the culmination of two works. Our first work “PATRL” (Pseudo-Active Transfer Learning), translates the complex alert signature description to the Action-Intent Framework (AIF), a customized set of attack stages. PATRL employs a deep language model with cyber security texts (CVE’s, C-Sec Blogs, etc.) and then uses transfer learning to classify alert descriptions. To further leverage the cyber-context learned in the language model, we develop Pseudo-Active learning to self-label unknown unlabeled alerts to use as additional training data. We show PATRL classifying the entire Suricata database (~70k signatures) with a top-1 of 87\% and top-3 of 99\% with less than 1,200 manually labeled signatures. The final work, HeAT (Heated Alert Triage), captures the analyst’s domain knowledge and opinion of the contribution of IDS events to an attack campaign given a critical IoC (indicator of compromise). We developed network-agnostic features to characterize and generalize attack campaign contributions so that prior triages can aid in identifying attack campaigns for other attack types, new attackers, or network infrastructures. With the use of cyber-attack competition data (CPTC) and data from a real SOC operation, we demonstrate that the HeAT process can identify campaigns reflective of the analysts thinking while greatly reducing the number of actions to be assessed by the analyst. HeAT has the unique ability to uncover attack campaigns meaningful to the analyst across drastically different network structures while maintaining the important attack campaign relationships defined by the analyst

A graph oriented approach for network forensic analysis

Network forensic analysis is a process that analyzes intrusion evidence captured from networked environment to identify suspicious entities and stepwise actions in an attack scenario. Unfortunately, the overwhelming amount and low quality of output from security sensors make it difficult for analysts to obtain a succinct high-level view of complex multi-stage intrusions. This dissertation presents a novel graph based network forensic analysis system. The evidence graph model provides an intuitive representation of collected evidence as well as the foundation for forensic analysis. Based on the evidence graph, we develop a set of analysis components in a hierarchical reasoning framework. Local reasoning utilizes fuzzy inference to infer the functional states of an host level entity from its local observations. Global reasoning performs graph structure analysis to identify the set of highly correlated hosts that belong to the coordinated attack scenario. In global reasoning, we apply spectral clustering and Pagerank methods for generic and targeted investigation respectively. An interactive hypothesis testing procedure is developed to identify hidden attackers from non-explicit-malicious evidence. Finally, we introduce the notion of target-oriented effective event sequence (TOEES) to semantically reconstruct stealthy attack scenarios with less dependency on ad-hoc expert knowledge. Well established computation methods used in our approach provide the scalability needed to perform post-incident analysis in large networks. We evaluate the techniques with a number of intrusion detection datasets and the experiment results show that our approach is effective in identifying complex multi-stage attacks

Mining Alarm Clusters to Improve Alarm Handling Efficiency

It is a well-known problem that intrusion detection systems overload their human operators by triggering thousands of alarms per day. As a matter of fact, we have been asked by one of our service divisions to help them deal with this problem. This paper presents the results of our research, validated thanks to a large set of operational data. We show that alarms should be managed by identifying and resolving their root causes. Alarm clustering is introduced as a method that supports the discovery of root causes. The general alarm clustering problem is proved to be NP-complete, an approximation algorithm is proposed, and experiments are presented