7 research outputs found
A lightweight intrusion alert fusion system
In this paper, we present some practical experience on implementing an alert fusion mechanism from our project. After investigation on most of the existing alert fusion systems, we found the current body of work alternatively weighed down in the mire of insecure design or rarely deployed because of their complexity. As confirmed by our experimental analysis, unsuitable mechanisms could easily be submerged by an abundance of useless alerts. Even with the use of methods that achieve a high fusion rate and low false positives, attack is also possible. To find the solution, we carried out analysis on a series of alerts generated by well-known datasets as well as realistic alerts from the Australian Honey-Pot. One important finding is that one alert has more than an 85% chance of being fused in the following 5 alerts. Of particular importance is our design of a novel lightweight Cache-based Alert Fusion Scheme, called CAFS. CAFS has the capacity to not only reduce the quantity of useless alerts generated by IDS (Intrusion Detection System), but also enhance the accuracy of alerts, therefore greatly reducing the cost of fusion processing. We also present reasonable and practical specifications for the target-oriented fusion policy that provides a quality guarantee on alert fusion, and as a result seamlessly satisfies the process of successive correlation. Our experimental results showed that the CAFS easily attained the desired level of survivable, inescapable alert fusion design. Furthermore, as a lightweight scheme, CAFS can easily be deployed and excel in a large amount of alert fusions, which go towards improving the usability of system resources. To the best of our knowledge, our work is a novel exploration in addressing these problems from a survivable, inescapable and deployable point of view
Network Anomaly Detection System with Optimized DS Evidence Theory
Network anomaly detection has been focused on by more people with the fast development of computer network. Some researchers utilized fusion method and DS evidence theory to do network anomaly detection but with low performance, and they did not consider features of networkâcomplicated and varied. To achieve high detection rate, we present a novel network anomaly detection system with optimized Dempster-Shafer evidence theory (ODS) and regression basic probability assignment (RBPA) function. In this model, we add weights for each senor to optimize DS evidence theory according to its previous predict accuracy. And RBPA employs sensorâs regression ability to address complex network. By four kinds of experiments, we find that our novel network anomaly detection model has a better detection rate, and RBPA as well as ODS optimization methods can improve system performance significantly
Using metrics from multiple layers to detect attacks in wireless networks
The IEEE 802.11 networks are vulnerable to numerous wireless-specific attacks. Attackers can implement MAC address spoofing techniques to launch these attacks, while masquerading themselves behind a false MAC address. The implementation of Intrusion Detection Systems has become fundamental in the development of security infrastructures for wireless networks. This thesis proposes the designing a novel security system that makes use of metrics from multiple layers of observation to produce a collective decision on whether an attack is taking place.
The Dempster-Shafer Theory of Evidence is the data fusion technique used to combine the evidences from the different layers. A novel, unsupervised and self- adaptive Basic Probability Assignment (BPA) approach able to automatically adapt its beliefs assignment to the current characteristics of the wireless network is proposed. This BPA approach is composed of three different and independent statistical techniques, which are capable to identify the presence of attacks in real time. Despite the lightweight processing requirements, the proposed security system produces outstanding detection results, generating high intrusion detection accuracy and very low number of false alarms. A thorough description of the generated results, for all the considered datasets is presented in this thesis. The effectiveness of the proposed system is evaluated using different types of injection attacks. Regarding one of these attacks, to the best of the author knowledge, the security system presented in this thesis is the first one able to efficiently identify the Airpwn attack
Intrusion Detection from Heterogenous Sensors
RĂSUMĂ
De nos jours, la protection des systĂšmes et rĂ©seaux informatiques contre diffĂ©rentes attaques avancĂ©es et distribuĂ©es constitue un dĂ©fi vital pour leurs propriĂ©taires. Lâune des menaces critiques Ă la sĂ©curitĂ© de ces infrastructures informatiques sont les attaques rĂ©alisĂ©es par des individus dont les intentions sont malveillantes, quâils soient situĂ©s Ă lâintĂ©rieur et Ă lâextĂ©rieur de lâenvironnement du systĂšme, afin dâabuser des services disponibles, ou de rĂ©vĂ©ler des informations confidentielles. Par consĂ©quent, la gestion et la surveillance des systĂšmes informatiques est un dĂ©fi considĂ©rable considĂ©rant que de nouvelles menaces et attaques sont dĂ©couvertes sur une base quotidienne.
Les systĂšmes de dĂ©tection dâintrusion, Intrusion Detection Systems (IDS) en anglais, jouent un rĂŽle clĂ© dans la surveillance et le contrĂŽle des infrastructures de rĂ©seau informatique. Ces systĂšmes inspectent les Ă©vĂ©nements qui se produisent dans les systĂšmes et rĂ©seaux informatiques et en cas de dĂ©tection dâactivitĂ© malveillante, ces derniers gĂ©nĂšrent des alertes afin de fournir les dĂ©tails des attaques survenues. Cependant, ces systĂšmes prĂ©sentent certaines limitations qui mĂ©ritent dâĂȘtre adressĂ©es si nous souhaitons les rendre suffisamment fiables pour rĂ©pondre aux besoins rĂ©els. Lâun des principaux dĂ©fis qui caractĂ©rise les IDS est le grand nombre dâalertes redondantes et non pertinentes ainsi que le taux de faux-positif gĂ©nĂ©rĂ©s, faisant de leur analyse une tĂąche difficile pour les administrateurs de sĂ©curitĂ© qui tentent de dĂ©terminer et dâidentifier les alertes qui sont rĂ©ellement importantes. Une partie du problĂšme rĂ©side dans le fait que la plupart des IDS ne prennent pas compte les informations contextuelles (type de systĂšmes, applications, utilisateurs, rĂ©seaux, etc.) reliĂ©es Ă lâattaque. Ainsi, une grande partie des alertes gĂ©nĂ©rĂ©es par les IDS sont non pertinentes en ce sens quâelles ne permettent de comprendre lâattaque dans son contexte et ce, malgrĂ© le fait que le systĂšme ait rĂ©ussi Ă correctement dĂ©tecter une intrusion. De plus, plusieurs IDS limitent leur dĂ©tection Ă un seul type de capteur, ce qui les rend inefficaces pour dĂ©tecter de nouvelles attaques complexes. Or, ceci est particuliĂšrement important dans le cas des attaques ciblĂ©es qui tentent dâĂ©viter la dĂ©tection par IDS conventionnels et par dâautres produits de sĂ©curitĂ©. Bien que de nombreux administrateurs systĂšme incorporent avec succĂšs des informations de contexte ainsi que diffĂ©rents types de capteurs et journaux dans leurs analyses, un problĂšme important avec cette approche reste le manque dâautomatisation, tant au niveau du stockage que de lâanalyse.
Afin de rĂ©soudre ces problĂšmes dâapplicabilitĂ©, divers types dâIDS ont Ă©tĂ© proposĂ©s dans les derniĂšres annĂ©es, dont les IDS de type composant pris sur Ă©tagĂšre, commercial off-the-shelf (COTS) en anglais, qui sont maintenant largement utilisĂ©s dans les centres dâopĂ©rations de sĂ©curitĂ©, Security Operations Center (SOC) en anglais, de plusieurs grandes organisations. Dâun point de vue plus gĂ©nĂ©ral, les diffĂ©rentes approches proposĂ©es peuvent ĂȘtre classĂ©es en diffĂ©rentes catĂ©gories : les mĂ©thodes basĂ©es sur lâapprentissage machine, tel que les rĂ©seaux bayĂ©siens, les mĂ©thodes dâextraction de donnĂ©es, les arbres de dĂ©cision, les rĂ©seaux de neurones, etc., les mĂ©thodes impliquant la corrĂ©lation dâalertes et les approches fondĂ©es sur la fusion dâalertes, les systĂšmes de dĂ©tection dâintrusion sensibles au contexte, les IDS dit distribuĂ©s et les IDS qui reposent sur la notion dâontologie de base. Ătant donnĂ© que ces diffĂ©rentes approches se concentrent uniquement sur un ou quelques-uns des dĂ©fis courants reliĂ©s aux IDS, au meilleure de notre connaissance, le problĂšme dans son ensemble nâa pas Ă©tĂ© rĂ©solu. Par consĂ©quent, il nâexiste aucune approche permettant de couvrir tous les dĂ©fis des IDS modernes prĂ©cĂ©demment mentionnĂ©s. Par exemple, les systĂšmes qui reposent sur des mĂ©thodes dâapprentissage machine classent les Ă©vĂ©nements sur la base de certaines caractĂ©ristiques en fonction du comportement observĂ© pour un type dâĂ©vĂ©nements, mais ils ne prennent pas en compte les informations reliĂ©es au contexte et les relations pouvant exister entre plusieurs Ă©vĂ©nements. La plupart des techniques de corrĂ©lation dâalerte proposĂ©es ne considĂšrent que la corrĂ©lation entre plusieurs capteurs du mĂȘme type ayant un Ă©vĂ©nement commun et une sĂ©mantique dâalerte similaire (corrĂ©lation homogĂšne), laissant aux administrateurs de sĂ©curitĂ© la tĂąche dâeffectuer la corrĂ©lation entre les diffĂ©rents types de capteurs hĂ©tĂ©rogĂšnes. Pour leur part, les approches sensibles au contexte nâemploient que des aspects limitĂ©s du contexte sous-jacent. Une autre limitation majeure des diffĂ©rentes approches proposĂ©es est lâabsence dâĂ©valuation prĂ©cise basĂ©e sur des ensembles de donnĂ©es qui contiennent des scĂ©narios dâattaque complexes et modernes.
Ă cet effet, lâobjectif de cette thĂšse est de concevoir un systĂšme de corrĂ©lation dâĂ©vĂ©nements qui peut prendre en considĂ©ration plusieurs types hĂ©tĂ©rogĂšnes de capteurs ainsi que les journaux de plusieurs applications (par exemple, IDS/IPS, pare-feu, base de donnĂ©es, systĂšme dâexploitation, antivirus, proxy web, routeurs, etc.). Cette mĂ©thode permettra de dĂ©tecter des attaques complexes qui laissent des traces dans les diffĂ©rents systĂšmes, et dâincorporer les informations de contexte dans lâanalyse afin de rĂ©duire les faux-positifs. Nos contributions peuvent ĂȘtre divisĂ©es en quatre parties principales : 1) Nous proposons la Pasargadae, une solution complĂšte sensible au contexte et reposant sur une ontologie de corrĂ©lation des Ă©vĂ©nements, laquelle effectue automatiquement la corrĂ©lation des Ă©vĂ©nements par lâanalyse des informations recueillies auprĂšs de diverses sources. Pasargadae utilise le concept dâontologie pour reprĂ©senter et stocker des informations sur les Ă©vĂ©nements, le contexte et les vulnĂ©rabilitĂ©s, les scĂ©narios dâattaques, et utilise des rĂšgles dâontologie de logique simple Ă©crites en Semantic Query-Enhance Web Rule Language (SQWRL) afin de corrĂ©ler diverse informations et de filtrer les alertes non pertinentes, en double, et les faux-positifs. 2) Nous proposons une approche basĂ©e sur, mĂ©ta-Ă©vĂ©nement , tri topologique et lâapproche corrĂ©lation dâĂ©vĂ©nement basĂ©e sur sĂ©mantique qui emploie Pasargadae pour effectuer la corrĂ©lation dâĂ©vĂ©nements Ă travers les Ă©vĂ©nements collectĂ©s de plusieurs capteurs rĂ©partis dans un rĂ©seau informatique. 3) Nous proposons une approche alerte de fusion basĂ©e sur sĂ©mantique, contexte sensible, qui sâappuie sur certains des sous-composantes de Pasargadae pour effectuer une alerte fusion hĂ©tĂ©rogĂšne recueillies auprĂšs IDS hĂ©tĂ©rogĂšnes. 4) Dans le but de montrer le niveau de flexibilitĂ© de Pasargadae, nous lâutilisons pour mettre en oeuvre dâautres approches proposĂ©es dâalertes et de corrĂ©lation dâĂ©vĂ©nements. La somme de ces contributions reprĂ©sente une amĂ©lioration significative de lâapplicabilitĂ© et la fiabilitĂ© des IDS dans des situations du monde rĂ©el.
Afin de tester la performance et la flexibilitĂ© de lâapproche de corrĂ©lation dâĂ©vĂ©nements proposĂ©s, nous devons aborder le manque dâinfrastructures expĂ©rimental adĂ©quat pour la sĂ©curitĂ© du rĂ©seau. Une Ă©tude de littĂ©rature montre que les approches expĂ©rimentales actuelles ne sont pas adaptĂ©es pour gĂ©nĂ©rer des donnĂ©es de rĂ©seau de grande fidĂ©litĂ©. Par consĂ©quent, afin dâaccomplir une Ă©valuation complĂšte, dâabord, nous menons nos expĂ©riences sur deux scĂ©narios dâĂ©tude dâanalyse de cas distincts, inspirĂ©s des ensembles de donnĂ©es dâĂ©valuation DARPA 2000 et UNB ISCX IDS. Ensuite, comme une Ă©tude dĂ©posĂ©e complĂšte, nous employons Pasargadae dans un vrai rĂ©seau informatique pour une pĂ©riode de deux semaines pour inspecter ses capacitĂ©s de dĂ©tection sur un vrai terrain trafic de rĂ©seau. Les rĂ©sultats obtenus montrent que, par rapport Ă dâautres amĂ©liorations IDS existants, les contributions proposĂ©es amĂ©liorent considĂ©rablement les performances IDS (taux de dĂ©tection) tout en rĂ©duisant les faux positifs, non pertinents et alertes en double.----------ABSTRACT
Nowadays, protecting computer systems and networks against various distributed and multi-steps attack has been a vital challenge for their owners. One of the essential threats to the security of such computer infrastructures is attacks by malicious individuals from inside and outside of the system environment to abuse available services, or reveal their confidential information. Consequently, managing and supervising computer systems is a considerable challenge, as new threats and attacks are discovered on a daily basis.
Intrusion Detection Systems (IDSs) play a key role in the surveillance and monitoring of computer network infrastructures. These systems inspect events occurred in computer systems and networks and in case of any malicious behavior they generate appropriate alerts describing the attacksâ details. However, there are a number of shortcomings that need to be addressed to make them reliable enough in the real-world situations. One of the fundamental challenges in real-world IDS is the large number of redundant, non-relevant, and false positive alerts that they generate, making it a difficult task for security administrators to determine and identify real and important alerts. Part of the problem is that most of the IDS do not take into account contextual information (type of systems, applications, users, networks, etc.), and therefore a large portion of the alerts are non-relevant in that even though they correctly recognize an intrusion, the intrusion fails to reach its objectives. Additionally, to detect newer and complicated attacks, relying on only one detection sensor type is not adequate, and as a result many of the current IDS are unable to detect them. This is especially important with respect to targeted attacks that try to avoid detection by conventional IDS and by other security products. While many system administrators are known to successfully incorporate context information and many different types of sensors and logs into their analysis, an important problem with this approach is the lack of automation in both storage and analysis. In order to address these problems in IDS applicability, various IDS types have been proposed in the recent years and commercial off-the-shelf (COTS) IDS products have found their way into Security Operations Centers (SOC) of many large organizations. From a general perspective, these works can be categorized into: machine learning based approaches including Bayesian networks, data mining methods, decision trees, neural networks, etc., alert correlation and alert fusion based approaches, context-aware intrusion detection systems, distributed intrusion detection systems, and ontology based intrusion detection systems. To the best of our knowledge, since these works only focus on one or few of the IDS challenges, the problem as a whole has not been resolved. Hence, there is no comprehensive work addressing all the mentioned challenges of modern intrusion detection systems. For example, works that utilize machine learning approaches only classify events based on some features depending on behavior observed with one type of events, and they do not take into account contextual information and event interrelationships. Most of the proposed alert correlation techniques consider correlation only across multiple sensors of the same type having a common event and alert semantics (homogeneous correlation), leaving it to security administrators to perform correlation across heterogeneous types of sensors. Context-aware approaches only employ limited aspects of the underlying context. The lack of accurate evaluation based on the data sets that encompass modern complex attack scenarios is another major shortcoming of most of the proposed approaches.
The goal of this thesis is to design an event correlation system that can correlate across several heterogeneous types of sensors and logs (e.g. IDS/IPS, firewall, database, operating system, anti-virus, web proxy, routers, etc.) in order to hope to detect complex attacks that leave traces in various systems, and incorporate context information into the analysis, in order to reduce false positives. To this end, our contributions can be split into 4 main parts: 1) we propose the Pasargadae comprehensive context-aware and ontology-based event correlation framework that automatically performs event correlation by reasoning on the information collected from various information resources. Pasargadae uses ontologies to represent and store information on events, context and vulnerability information, and attack scenarios, and uses simple ontology logic rules written in Semantic Query-Enhance Web Rule Language (SQWRL) to correlate various information and filter out non-relevant alerts and duplicate alerts, and false positives. 2) We propose a meta-event based, topological sort based and semantic-based event correlation approach that employs Pasargadae to perform event correlation across events collected form several sensors distributed in a computer network. 3) We propose a semantic-based context-aware alert fusion approach that relies on some of the subcomponents of Pasargadae to perform heterogeneous alert fusion collected from heterogeneous IDS. 4) In order to show the level of flexibility of Pasargadae, we use it to implement some other proposed alert and event correlation approaches. The sum of these contributions represent a significant improvement in the applicability and reliability of IDS in real-world situations.
In order to test the performance and flexibility of the proposed event correlation approach, we need to address the lack of experimental infrastructure suitable for network security. A study of the literature shows that current experimental approaches are not appropriate to generate high fidelity network data. Consequently, in order to accomplish a comprehensive evaluation, first, we conduct our experiments on two separate analysis case study scenarios, inspired from the DARPA 2000 and UNB ISCX IDS evaluation data sets. Next, as a complete field study, we employ Pasargadae in a real computer network for a two weeks period to inspect its detection capabilities on a ground truth network traffic. The results obtained show that compared to other existing IDS improvements, the proposed contributions significantly improve IDS performance (detection rate) while reducing false positives, non-relevant and duplicate alerts