Alert Correlation through a Multi Components Architecture

Alert correlation is a process that analyzes the raw alerts produced by one or more intrusion detection systems, reduces nonrelevant ones, groups together alerts based on similarity and causality relationships between them and finally makes aconcise and meaningful view of occurring or attempted intrusions. Unfortunately, most correlation approaches use just a few components that aim only specific correlation issues and so cause reduction in correlation rate. This paper uses a general correlation model that has already been presented in [9] and is consisted of a comprehensive set of components. Then some changes are applied in the component that is related to multi-step attack scenario to detect them better and so to improve semantic level of alerts. The results of experiments with DARPA 2000 data set obviously show the effectiveness of the proposed approach.DOI:http://dx.doi.org/10.11591/ijece.v3i4.277

Statistical anomaly denial of service and reconnaissance intrusion detection

This dissertation presents the architecture, methods and results of the Hierarchical Intrusion Detection Engine (HIDE) and the Reconnaissance Intrusion Detection System (RIDS); the former is denial-of-service (DoS) attack detector while the latter is a scan and probe (P&S) reconnaissance detector; both are statistical anomaly systems. The HIDE is a packet-oriented, observation-window using, hierarchical, multi-tier, anomaly based network intrusion detection system, which monitors several network traffic parameters simultaneously, constructs a 64-bin probability density function (PDF) for each, statistically compares it to a reference PDF of normal behavior using a similarity metric, then combines the results into an anomaly status vector that is classified by a neural network classifier. Three different data sets have been utilized to test the performance of HIDE; they are OPNET simulation data, DARPA\u2798 intrusion detection evaluation data and the CONEX TESTBED attack data. The results showed that HIDE can reliably detect DoS attacks with high accuracy and very low false alarm rates on all data sets. In particular, the investigation using the DARPA\u2798 data set yielded an overall total misclassification rate of 0.13%, false negative rate of 1.42%, and false positive rate of 0.090%; the latter implies a rate of only about 2.6 false alarms per day. The RIDS is a session oriented, statistical tool, that relies on training to model the parameters of its algorithms, capable of detecting even distributed stealthy reconnaissance attacks. It consists of two main functional modules or stages: the Reconnaissance Activity Profiler (RAP) and the Reconnaissance Alert Correlater (RAC). The RAP is a session-oriented module capable of detecting stealthy scanning and probing attacks, while the RAG is an alert-correlation module that fuses the RAP alerts into attack scenarios and discovers the distributed stealthy attack scenarios. RIDS has been evaluated against two data sets: (a) the DARPA\u2798 data, and (b) 3 weeks of experimental data generated using the CONEX TESTBED network. The RIDS has demonstrably achieved remarkable success; the false positive, false negative and misclassification rates found are low, less than 0.1%, for most reconnaissance attacks; they rise to about 6% for distributed highly stealthy attacks; the latter is a most challenging type of attack, which has been difficult to detect effectively until now

Анализ методов корреляции событий безопасности в SIEM-системах. Часть 1.

The paper is devoted to the analysis of security event correlation methods in Security Information and Event Management (SIEM) systems. The correlation process is considered to be a multilevel hierarchy of stages. The goal of each stage consists in executing appropriate operations on security data being processed. Based on this analysis we outline each correlation stage and their interaction scheme.Статья посвящена анализу методов корреляции событий безопасности в системах управления информацией и событиями безопасности (SIEM-системах). Процесс корреляции событий безопасности рассматривается в виде многоуровневой иерархии этапов, цель каждого из которых заключается в выполнении определенных операций над обрабатываемыми данными безопасности. На основе результатов проведенного анализа в работе приводится описание каждого этапа процесса корреляции и схемы их взаимодействия

Enhancing Computer Network Security through Improved Outlier Detection for Data Streams

V několika posledních letech se metody strojového učení (zvláště ty zabývající se detekcí odlehlých hodnot - OD) v oblasti kyberbezpečnosti opíraly o zjišťování anomálií síťového provozu spočívajících v nových schématech útoků. Detekce anomálií v počítačových sítích reálného světa se ale stala stále obtížnější kvůli trvalému nárůstu vysoce objemných, rychlých a dimenzionálních průběžně přicházejících dat (SD), pro která nejsou k dispozici obecně uznané a pravdivé informace o anomalitě. Účinná detekční schémata pro vestavěná síťová zařízení musejí být rychlá a paměťově nenáročná a musejí být schopna se potýkat se změnami konceptu, když se vyskytnou. Cílem této disertace je zlepšit bezpečnost počítačových sítí zesílenou detekcí odlehlých hodnot v datových proudech, obzvláště SD, a dosáhnout kyberodolnosti, která zahrnuje jak detekci a analýzu, tak reakci na bezpečnostní incidenty jako jsou např. nové zlovolné aktivity. Za tímto účelem jsou v práci navrženy čtyři hlavní příspěvky, jež byly publikovány nebo se nacházejí v recenzním řízení časopisů. Zaprvé, mezera ve volbě vlastností (FS) bez učitele pro zlepšování již hotových metod OD v datových tocích byla zaplněna navržením volby vlastností bez učitele pro detekci odlehlých průběžně přicházejících dat označované jako UFSSOD. Následně odvozujeme generický koncept, který ukazuje dva aplikační scénáře UFSSOD ve spojení s online algoritmy OD. Rozsáhlé experimenty ukázaly, že UFSSOD coby algoritmus schopný online zpracování vykazuje srovnatelné výsledky jako konkurenční metoda upravená pro OD. Zadruhé představujeme nový aplikační rámec nazvaný izolovaný les založený na počítání výkonu (PCB-iForest), jenž je obecně schopen využít jakoukoliv online OD metodu založenou na množinách dat tak, aby fungovala na SD. Do tohoto algoritmu integrujeme dvě varianty založené na klasickém izolovaném lese. Rozsáhlé experimenty provedené na 23 multidisciplinárních datových sadách týkajících se bezpečnostní problematiky reálného světa ukázaly, že PCB-iForest jasně překonává už zavedené konkurenční metody v 61 % případů a dokonce dosahuje ještě slibnějších výsledků co do vyváženosti mezi výpočetními náklady na klasifikaci a její úspěšností. Zatřetí zavádíme nový pracovní rámec nazvaný detekce odlehlých hodnot a rozpoznávání schémat útoku proudovým způsobem (SOAAPR), jenž je na rozdíl od současných metod schopen zpracovat výstup z různých online OD metod bez učitele proudovým způsobem, aby získal informace o nových schématech útoku. Ze seshlukované množiny korelovaných poplachů jsou metodou SOAAPR vypočítány tři různé soukromí zachovávající podpisy podobné otiskům prstů, které charakterizují a reprezentují potenciální scénáře útoku s ohledem na jejich komunikační vztahy, projevy ve vlastnostech dat a chování v čase. Evaluace na dvou oblíbených datových sadách odhalila, že SOAAPR může soupeřit s konkurenční offline metodou ve schopnosti korelace poplachů a významně ji překonává z hlediska výpočetního času . Navíc se všechny tři typy podpisů ve většině případů zdají spolehlivě charakterizovat scénáře útoků tím, že podobné seskupují k sobě. Začtvrté představujeme algoritmus nepárového kódu autentizace zpráv (Uncoupled MAC), který propojuje oblasti kryptografického zabezpečení a detekce vniknutí (IDS) pro síťovou bezpečnost. Zabezpečuje síťovou komunikaci (autenticitu a integritu) kryptografickým schématem s podporou druhé vrstvy kódy autentizace zpráv, ale také jako vedlejší efekt poskytuje funkcionalitu IDS tak, že vyvolává poplach na základě porušení hodnot nepárového MACu. Díky novému samoregulačnímu rozšíření algoritmus adaptuje svoje vzorkovací parametry na základě zjištění škodlivých aktivit. Evaluace ve virtuálním prostředí jasně ukazuje, že schopnost detekce se za běhu zvyšuje pro různé scénáře útoku. Ty zahrnují dokonce i situace, kdy se inteligentní útočníci snaží využít slabá místa vzorkování.ObhájenoOver the past couple of years, machine learning methods - especially the Outlier Detection (OD) ones - have become anchored to the cyber security field to detect network-based anomalies rooted in novel attack patterns. Due to the steady increase of high-volume, high-speed and high-dimensional Streaming Data (SD), for which ground truth information is not available, detecting anomalies in real-world computer networks has become a more and more challenging task. Efficient detection schemes applied to networked, embedded devices need to be fast and memory-constrained, and must be capable of dealing with concept drifts when they occur. The aim of this thesis is to enhance computer network security through improved OD for data streams, in particular SD, to achieve cyber resilience, which ranges from the detection, over the analysis of security-relevant incidents, e.g., novel malicious activity, to the reaction to them. Therefore, four major contributions are proposed, which have been published or are submitted journal articles. First, a research gap in unsupervised Feature Selection (FS) for the improvement of off-the-shell OD methods in data streams is filled by proposing Unsupervised Feature Selection for Streaming Outlier Detection, denoted as UFSSOD. A generic concept is retrieved that shows two application scenarios of UFSSOD in conjunction with online OD algorithms. Extensive experiments have shown that UFSSOD, as an online-capable algorithm, achieves comparable results with a competitor trimmed for OD. Second, a novel unsupervised online OD framework called Performance Counter-Based iForest (PCB-iForest) is being introduced, which generalized, is able to incorporate any ensemble-based online OD method to function on SD. Two variants based on classic iForest are integrated. Extensive experiments, performed on 23 different multi-disciplinary and security-related real-world data sets, revealed that PCB-iForest clearly outperformed state-of-the-art competitors in 61 % of cases and even achieved more promising results in terms of the tradeoff between classification and computational costs. Third, a framework called Streaming Outlier Analysis and Attack Pattern Recognition, denoted as SOAAPR is being introduced that, in contrast to the state-of-the-art, is able to process the output of various online unsupervised OD methods in a streaming fashion to extract information about novel attack patterns. Three different privacy-preserving, fingerprint-like signatures are computed from the clustered set of correlated alerts by SOAAPR, which characterize and represent the potential attack scenarios with respect to their communication relations, their manifestation in the data's features and their temporal behavior. The evaluation on two popular data sets shows that SOAAPR can compete with an offline competitor in terms of alert correlation and outperforms it significantly in terms of processing time. Moreover, in most cases all three types of signatures seem to reliably characterize attack scenarios to the effect that similar ones are grouped together. Fourth, an Uncoupled Message Authentication Code algorithm - Uncoupled MAC - is presented which builds a bridge between cryptographic protection and Intrusion Detection Systems (IDSs) for network security. It secures network communication (authenticity and integrity) through a cryptographic scheme with layer-2 support via uncoupled message authentication codes but, as a side effect, also provides IDS-functionality producing alarms based on the violation of Uncoupled MAC values. Through a novel self-regulation extension, the algorithm adapts its sampling parameters based on the detection of malicious actions on SD. The evaluation in a virtualized environment clearly shows that the detection rate increases over runtime for different attack scenarios. Those even cover scenarios in which intelligent attackers try to exploit the downsides of sampling

A Simulation Based SIEM Framework to Attribute and Predict Attacks

We present a Security Information and Event Management (SIEM) framework to correlate, attribute and predict attacks against an ICT system. The output of the assessment of ICT risk, that exploits multiple simulations of attacks against the system, drives the building of a SIEM database. This database enables the SIEM to correlate sequences of detected attacks, to probabilistically attribute and predict attacks, and to discover 0-day vulnerability. After describing the framework and its prototype implementation, we discuss the experimental results on the main SIEM capabilities

Realtime Intrusion Risk Assessment Model based on Attack and Service Dependency Graphs

Network services are becoming larger and increasingly complex to manage. It is extremely critical to maintain the users QoS, the response time of applications, and critical services in high demand. On the other hand, we see impressive changes in the ways in which attackers gain access to systems and infect services. When an attack is detected, an Intrusion Response System (IRS) is responsible to accurately assess the value of the loss incurred by a compromised resource and apply the proper responses to mitigate attack. Without having a proper risk assessment, our automated IRS will reduce network performance, wrongly disconnect users from the network, or result in high costs for administrators reestablishing services, and become a DoS attack for our network, which will eventually have to be disabled. In this paper, we address these challenges and we propose a new model to combine the Attack Graph and Service Dependency Graph approaches to calculate the impact of an attack more accurately compared to other existing solutions. To show the effectiveness of our model, a sophisticated multi-step attack was designed to compromise a web server, as well as to acquire root privilege. Our results illustrate the efficiency of the proposed model and confirm the feasibility of the approach in real-time

Advanced attack tree based intrusion detection

Computer network systems are constantly under attack or have to deal with attack attempts. The first step in any network’s ability to fight against intrusive attacks is to be able to detect intrusions when they are occurring. Intrusion Detection Systems (IDS) are therefore vital in any kind of network, just as antivirus is a vital part of a computer system. With the increasing computer network intrusion sophistication and complexity, most of the victim systems are compromised by sophisticated multi-step attacks. In order to provide advanced intrusion detection capability against the multi-step attacks, it makes sense to adopt a rigorous and generalising view to tackling intrusion attacks. One direction towards achieving this goal is via modelling and consequently, modelling based detection. An IDS is required that has good quality of detection capability, not only to be able to detect higher-level attacks and describe the state of ongoing multi-step attacks, but also to be able to determine the achievement of high-level attack detection even if any of the modelled low-level attacks are missed by the detector, because no alert being generated may represent that the corresponding low-level attack is either not being conducted by the adversary or being conducted by the adversary but evades the detection. This thesis presents an attack tree based intrusion detection to detect multistep attacks. An advanced attack tree modelling technique, Attack Detection Tree, is proposed to model the multi-step attacks and facilitate intrusion detection. In addition, the notion of Quality of Detectability is proposed to describe the ongoing states of both intrusion and intrusion detection. Moreover, a detection uncertainty assessment mechanism is proposed to apply the measured evidence to deal with the uncertainty issues during the assessment process to determine the achievement of high-level attacks even if any modelled low-level incidents may be missing