CloudMon: a resource-efficient IaaS cloud monitoring system based on networked intrusion detection system virtual appliances

The networked intrusion detection system virtual appliance (NIDS-VA), also known as virtualized NIDS, plays an important role in the protection and safeguard of IaaS cloud environments. However, it is nontrivial to guarantee both of the performance of NIDS-VA and the resource efficiency of cloud applications because both are sharing computing resources in the same cloud environment. To overcome this challenge and trade-off, we propose a novel system, named CloudMon, which enables dynamic resource provision and live placement for NIDS-VAs in IaaS cloud environments. CloudMon provides two techniques to maintain high resource efficiency of IaaS cloud environments without degrading the performance of NIDS-VAs and other virtual machines (VMs). The first technique is a virtual machine monitor based resource provision mechanism, which can minimize the resource usage of a NIDS-VA with given performance guarantee. It uses a fuzzy model to characterize the complex relationship between performance and resource demands of a NIDS-VA and develops an online fuzzy controller to adaptively control the resource allocation for NIDS-VAs under varying network traffic. The second one is a global resource scheduling approach for optimizing the resource efficiency of the entire cloud environments. It leverages VM migration to dynamically place NIDS-VAs and VMs. An online VM mapping algorithm is designed to maximize the resource utilization of the entire cloud environment. Our virtual machine monitor based resource provision mechanism has been evaluated by conducting comprehensive experiments based on Xen hypervisor and Snort NIDS in a real cloud environment. The results show that the proposed mechanism can allocate resources for a NIDS-VA on demand while still satisfying its performance requirements. We also verify the effectiveness of our global resource scheduling approach by comparing it with two classic vector packing algorithms, and the results show that our approach improved the resource utilization of cloud environments and reduced the number of in-use NIDS-VAs and physical hosts.The authors gratefully acknowledge the anonymous reviewers for their helpful suggestions and insightful comments to improve the quality of the paper. The work reported in this paper has been partially supported by National Nature Science Foundation of China (No. 61202424, 61272165, 91118008), China 863 program (No. 2011AA01A202), Natural Science Foundation of Jiangsu Province of China (BK20130528) and China 973 Fundamental R&D Program (2011CB302600)

CHID : conditional hybrid intrusion detection system for reducing false positives and resource consumption on malicous datasets

Inspecting packets to detect intrusions faces challenges when coping with a high volume of network traffic. Packet-based detection processes every payload on the wire, which degrades the performance of network intrusion detection system (NIDS). This issue requires an introduction of a flow-based NIDS that reduces the amount of data to be processed by examining aggregated information of related packets. However, flow-based detection still suffers from the generation of the false positive alerts due to incomplete data input. This study proposed a Conditional Hybrid Intrusion Detection (CHID) by combining the flow-based with packet-based detection. In addition, it is also aimed to improve the resource consumption of the packet-based detection approach. CHID applied attribute wrapper features evaluation algorithms that marked malicious flows for further analysis by the packet-based detection. Input Framework approach was employed for triggering packet flows between the packetbased and flow-based detections. A controlled testbed experiment was conducted to evaluate the performance of detection mechanism’s CHID using datasets obtained from on different traffic rates. The result of the evaluation showed that CHID gains a significant performance improvement in terms of resource consumption and packet drop rate, compared to the default packet-based detection implementation. At a 200 Mbps, CHID in IRC-bot scenario, can reduce 50.6% of memory usage and decreases 18.1% of the CPU utilization without packets drop. CHID approach can mitigate the false positive rate of flow-based detection and reduce the resource consumption of packet-based detection while preserving detection accuracy. CHID approach can be considered as generic system to be applied for monitoring of intrusion detection systems

New opportunities for load balancing in network-wide intrusion detection systems

As traffic volumes and the types of analysis grow, network intru-sion detection systems (NIDS) face a continuous scaling challenge. Management realities, however, limit NIDS hardware upgrades to occur typically once every 3-5 years. Given that traffic patterns can change dramatically, this leaves a significant scaling challenge in the interim. This motivates the need for practical solutions that can help administrators better utilize and augment their existing NIDS infrastructure. To this end, we design a general architecture for network-wide NIDS deployment that leverages three scaling op-portunities: on-path distribution to split responsibilities, replicat-ing traffic to NIDS clusters, and aggregating intermediate results to split expensive NIDS processing. The challenge here is to balance both the compute load across the network and the total communica-tion cost incurred via replication and aggregation. We implement a backwards-compatible mechanism to enable existing NIDS infras-tructure to leverage these benefits. Using emulated and trace-driven evaluations on several real-world network topologies, we show that our proposal can substantially reduce the maximum computation load, provide better resilience under traffic variability, and offer improved detection coverage

Scalable and Reliable Middlebox Deployment

Middleboxes are pervasive in modern computer networks providing functionalities beyond mere packet forwarding. Load balancers, intrusion detection systems, and network address translators are typical examples of middleboxes. Despite their benefits, middleboxes come with several challenges with respect to their scalability and reliability. The goal of this thesis is to devise middlebox deployment solutions that are cost effective, scalable, and fault tolerant. The thesis includes three main contributions: First, distributed service function chaining with multiple instances of a middlebox deployed on different physical servers to optimize resource usage; Second, Constellation, a geo-distributed middlebox framework enabling a middlebox application to operate with high performance across wide area networks; Third, a fault tolerant service function chaining system

Vers la sécurité des conteneurs : les comprendre et les sécuriser

To facilitate shorter modern development cycles, as well as the ephemeral nature of cloud computing, many organizations are now running their applications in containers, a form of operating system virtualization. These new environments are often referred to as containerized environments. However, these environments are not without risk. Recent studies have shown that containerized applications are, like all types of applications, prone to various attacks. Another problem for those working in IT security is that containerized applications are often very dynamic and short-lived, which compounds the problem because it is more difficult to audit their activities or even make an investigation. In case of intrusion. In this thesis, we propose an intrusion detection system based on machine learning for containerized environments. Containers provide isolation between the host system and the containerized environment by efficiently grouping applications and their dependencies. In this way, containers become a portable software environment. However, unlike virtual machines, containers share the same kernel as the host operating system. In order to be able to do anomaly detection, our system uses this feature to monitor system calls sent from a container to a host system. Thus, the monitored container does not have to be modified and our system is not required to know the nature of the container to monitor it. The results of our experiments show that it is indeed possible to use system calls to detect abnormal behaviour made by a containerized application without having to modify the container.Afin de faciliter les cycles de développement moderne plus courts, ainsi que la nature éphémère de l’infonuagique, de nombreuses organisations exécutent désormais leurs applications dans des conteneurs, une forme de virtualisation du système d'exploitation. Ces nouveaux environnements sont souvent appelés environnements conteneurisés. Cependant, ces environnements ne sont pas sans risque. Des études récentes ont montré que les applications conteneurisées sont, comme tous les types d’applications, sujettes à diverses attaques. Un autre problème pour ceux qui travaillent dans le domaine de la sécurité informatique est que les applications conteneurisées sont souvent très dynamiques et de courte durée, ce qui aggrave le problème, car il est plus difficile d’auditer leurs activités ou encore de faire une enquête en cas d’intrusion. Dans ce mémoire, nous proposons un système de détection d’intrusion basé sur l’apprentissage machine pour les environnements conteneurisés. Les conteneurs assurent l'isolation entre le système hôte et l'environnement conteneurisé en regroupant efficacement, les applications ainsi que leurs dépendances. De cette façon, les conteneurs deviennent un environnement logiciel portable. Cependant, contrairement aux machines virtuelles, les conteneurs partagent le même noyau que le système d'exploitation hôte. Afin de pouvoir faire la détection d'anomalies, notre système utilise cette caractéristique pour surveiller les appels système envoyés d’un conteneur vers un système hôte. Ainsi, le conteneur surveillé n’a pas à être modifié et notre système n'est pas tenu de connaitre la nature du conteneur pour le surveiller. Les résultats de nos expériences montrent qu’il est en effet possible d’utiliser les appels système afin de détecter des comportements anormaux faits par une application conteneurisée et ce sans à avoir à modifier le conteneur

Expression and Composition of Optimization-Based Applications for Software-Defined Networking

Motivated by the adoption of the Software Defined Networking and its increasing focus on applications for resource management, we propose a novel framework for expressing network optimization applications. Named the SDN Optimization Layer (SOL), the framework and its extensions alleviate the burden of constructing optimization applications by abstracting the low-level details of mathematical optimization techniques such as linear programming. SOL utilizes the path abstraction to express a wide variety of network constraints and resource-management logic. We show that the framework is general and efficient enough to support various classes of applications. We extend SOL to support composition of multiple applications in a fair and resource-efficient way. We demonstrate that SOL’s composition produces better resource efficiency than previously available composition approaches and is tolerant to network variations. Finally, as a case study, we develop a new application for load balancing network intrusion prevention systems, called SNIPS. We highlight the challenges in developing the SNIPS optimization from the ground up, show SOL’s (conceptually) simplified version, and verify that both produce nearly identical solutions.Doctor of Philosoph

Security Configuration Management in Intrusion Detection and Prevention Systems

Intrusion Detection and/or Prevention Systems (IDPS) represent an important line of defense against a variety of attacks that can compromise the security and proper functioning of an enterprise information system. IDPSs can be network or host-based and can collaborate in order to provide better detection of malicious traffic. Although several IDPS systems have been proposed, their appropriate con figuration and control for e effective detection/ prevention of attacks and efficient resource consumption is still far from trivial. Another concern is related to the slowing down of system performance when maximum security is applied, hence the need to trade o between security enforcement levels and the performance and usability of an enterprise information system. In this dissertation, we present a security management framework for the configuration and control of the security enforcement mechanisms of an enterprise information system. The approach leverages the dynamic adaptation of security measures based on the assessment of system vulnerability and threat prediction, and provides several levels of attack containment. Furthermore, we study the impact of security enforcement levels on the performance and usability of an enterprise information system. In particular, we analyze the impact of an IDPS con figuration on the resulting security of the network, and on the network performance. We also analyze the performance of the IDPS for different con figurations and under different traffic characteristics. The analysis can then be used to predict the impact of a given security con figuration on the prediction of the impact on network performance

Towards Coordinated, Network-Wide Traffic Monitoring for Early Detection of DDoS Flooding Attacks

DDoS flooding attacks are one of the biggest concerns for security professionals and they are typically explicit attempts to disrupt legitimate users' access to services. Developing a comprehensive defense mechanism against such attacks requires a comprehensive understanding of the problem and the techniques that have been used thus far in preventing, detecting, and responding to various such attacks. In this thesis, we dig into the problem of DDoS flooding attacks from four directions: (1) We study the origin of these attacks, their variations, and various existing defense mechanisms against them. Our literature review gives insight into a list of key required features for the next generation of DDoS flooding defense mechanisms. The most important requirement on this list is to see more distributed DDoS flooding defense mechanisms in near future, (2) In such systems, the success in detecting DDoS flooding attacks earlier and in a distributed fashion is highly dependent on the quality and quantity of the traffic flows that are covered by the employed traffic monitoring mechanisms. This motivates us to study and understand the challenges of existing traffic monitoring mechanisms, (3) We propose a novel distributed, coordinated, network-wide traffic monitoring (DiCoTraM) approach that addresses the key challenges of current traffic monitoring mechanisms. DiCoTraM enhances flow coverage to enable effective, early detection of DDoS flooding attacks. We compare and evaluate the performance of DiCoTraM with various other traffic monitoring mechanisms in terms of their total flow coverage and DDoS flooding attack flow coverage, and (4) We evaluate the effectiveness of DiCoTraM with cSamp, an existing traffic monitoring mechanism that outperforms most of other traffic monitoring mechanisms, with regards to supporting early detection of DDoS flooding attacks (i.e., at the intermediate network) by employing two existing DDoS flooding detection mechanisms over them. We then compare the effectiveness of DiCoTraM with that of cSamp by comparing the detection rates and false positive rates achieved when the selected detection mechanisms are employed over DiCoTraM and cSamp. The results show that DiCoTraM outperforms other traffic monitoring mechanisms in terms of DDoS flooding attack flow coverage

A framework for correlation and aggregation of security alerts in communication networks. A reasoning correlation and aggregation approach to detect multi-stage attack scenarios using elementary alerts generated by Network Intrusion Detection Systems (NIDS) for a global security perspective.

The tremendous increase in usage and complexity of modern communication and network systems connected to the Internet, places demands upon security management to protect organisations¿ sensitive data and resources from malicious intrusion. Malicious attacks by intruders and hackers exploit flaws and weakness points in deployed systems through several sophisticated techniques that cannot be prevented by traditional measures, such as user authentication, access controls and firewalls. Consequently, automated detection and timely response systems are urgently needed to detect abnormal activities by monitoring network traffic and system events. Network Intrusion Detection Systems (NIDS) and Network Intrusion Prevention Systems (NIPS) are technologies that inspect traffic and diagnose system behaviour to provide improved attack protection. The current implementation of intrusion detection systems (commercial and open-source) lacks the scalability to support the massive increase in network speed, the emergence of new protocols and services. Multi-giga networks have become a standard installation posing the NIDS to be susceptible to resource exhaustion attacks. The research focuses on two distinct problems for the NIDS: missing alerts due to packet loss as a result of NIDS performance limitations; and the huge volumes of generated alerts by the NIDS overwhelming the security analyst which makes event observation tedious. A methodology for analysing alerts using a proposed framework for alert correlation has been presented to provide the security operator with a global view of the security perspective. Missed alerts are recovered implicitly using a contextual technique to detect multi-stage attack scenarios. This is based on the assumption that the most serious intrusions consist of relevant steps that temporally ordered. The pre- and post- condition approach is used to identify the logical relations among low level alerts. The alerts are aggregated, verified using vulnerability modelling, and correlated to construct multi-stage attacks. A number of algorithms have been proposed in this research to support the functionality of our framework including: alert correlation, alert aggregation and graph reduction. These algorithms have been implemented in a tool called Multi-stage Attack Recognition System (MARS) consisting of a collection of integrated components. The system has been evaluated using a series of experiments and using different data sets i.e. publicly available datasets and data sets collected using real-life experiments. The results show that our approach can effectively detect multi-stage attacks. The false positive rates are reduced due to implementation of the vulnerability and target host information