Hardware Acceleration of Network Intrusion Detection System Using FPGA

This thesis presents new algorithms and hardware designs for Signature-based Network Intrusion Detection System (SB-NIDS) optimisation exploiting a hybrid hardwaresoftware co-designed embedded processing platform. The work describe concentrates on optimisation of a complete SB-NIDS Snort application software on a FPGA based hardware-software target rather than on the implementation of a single functional unit for hardware acceleration. Pattern Matching Hardware Accelerator (PMHA) based on Bloom filter was designed to optimise SB-NIDS performance for execution on a Xilinx MicroBlaze soft-core processor. The Bloom filter approach enables the potentially large number of network intrusion attack patterns to be efficiently represented and searched primarily using accesses to FPGA on-chip memory. The thesis demonstrates, the viability of hybrid hardware-software co-designed approach for SB-NIDS. Future work is required to investigate the effects of later generation FPGA technology and multi-core processors in order to clearly prove the benefits over conventional processor platforms for SB-NIDS. The strengths and weaknesses of the hardware accelerators and algorithms are analysed, and experimental results are examined to determine the effectiveness of the implementation. Experimental results confirm that the PMHA is capable of performing network packet analysis for gigabit rate network traffic. Experimental test results indicate that our SB-NIDS prototype implementation on relatively low clock rate embedded processing platform performance is approximately 1.7 times better than Snort executing on a general purpose processor on PC when comparing processor cycles rather than wall clock time

Enhancing snort IDs performance using data mining

Intrusion detection systems (IDSs) such as Snort apply deep packet inspection to detect intrusions. Usually, these are rule-based systems, where each incoming packet is matched with a set of rules. Each rule consists of two parts: the rule header and the rule options. The rule header is compared with the packet header. The rule options usually contain a signature string that is matched with packet content using an efficient string matching algorithm. The traditional approach to IDS packet inspection checks a packet against the detection rules by scanning from the first rule in the set and continuing to scan all the rules until a match is found. This approach becomes inefficient if the number of rules is too large and if the majority of the packets match with rules located at the end of the rule set. In this thesis, we propose an intelligent predictive technique for packet inspection based on data mining. We consider each rule in a rule set as a ‘class’. A classifier is first trained with labeled training data. Each such labeled data point contains packet header information, packet content summary information, and the corresponding class label (i.e. the rule number with which the packet matches). Then the classifier is used to classify new incoming packets. The predicted class, i.e. rule, is checked against the packet to see if this packet really matches the predicted rule. If it does, the corresponding action (i.e. alert) of the rule is taken. Otherwise, if the prediction of the classifier is wrong, we go back to the traditional way of matching rules. The advantage of this intelligent predictive packet matching is that it offers much faster rule matching. We have proved, both analytically and empirically, that even with millions of real network traffic packets and hundreds of rules, the classifier can achieve very high accuracy, thereby making the IDS several times faster in making matching decisions

A MULTI-GIGABIT NETWORK PACKET INSPECTION AND ANALYSIS ARCHITECTURE FOR INTRUSION DETECTION AND PREVENTION UTILIZING PIPELINING AND CONTENT-ADDRESSABLE MEMORY

Increases in network traffic volume and transmission speeds have given rise to the need for extremely fast packet processing. Many traditional processor-based network devices are no longer sufficient to handle tasks such as packet analysis and intrusion detection at multi-Gigabit rates. This thesis proposes two novel pipelined hardware architectures to relieve the computational load of a processor within network switches and routers. First, the Embedded Protocol Analyzer Pre-Processor (ePAPP) is capable of taking an unclassified packet byte stream directly off of a network cable at line speed and separating the data into individually classified protocol fields. Second, the CAM-Assisted Signature-Matching Architecture (CASMA) uses ternary content-addressable memory to perform the task of stateless intrusion detection signature-matching. The Snort open-source software network intrusion detection system is used as a model for intrusion detection functionality. Structured ASIC synthesis results show that ePAPP supports speeds of 2.89 Gb/s using less than 1% of available logic cells. CASMA is shown to support 1.25 Gb/s using less than 6% of available logic cells. The CASMA architecture is demonstrated to be able to implement 1729 of 1993 or 86.8% of the attack signatures, or rules, packaged with Snort version 2.1.2

Encrypted mal-ware detection

Mal-ware such as viruses and worms are increasingly proliferating through out all networks. Existing schemes that address these issues either assume that the mal-ware is available in its plain-text format which can be detected directly with its signature or that its exploit-code execution is directly recognizable. Hence much of the development in this area has been focussed on generating more efficient signatures or in coming up with improved anomaly-based detection and pattern matching rules. However with secure data being the watch-word and several efficient encryption schemes being developed to obfuscate data and protect its privacy, encrypted mal-ware is very much a clear and present threat. While securing resources from encrypted threats is the need of the hour, equally critical is the privacy of content that needs to be protected. In this paper we discuss encrypted mal-ware detection and propose an efficient IP-packet level scheme for encrypted mal-ware detection that does not compromise the privacy of the data but at the same time helps detect the presence of hidden mal-ware in it. We also propose a new grammar for a generalized representation of all kinds of malicious-signatures. This signature grammar is inclusive of even polymorphic and metamorphic signatures which do not have a straight-forward one-to-one mapping between the signature string and worm-recognition. In a typical system model consisting of several co-operating hosts which are un-intentional senders of mal-ware traffic, where a centralized network monitor functions as the mal-ware detection entity, we show that for a very small memory and processing overhead and almost negligible time-requirements, we achieve a very high detection rate for even the most advanced multi-keyword polymorphic signatures

Parallelization of a software based intrusion detection system - Snort

Computer networks are already ubiquitous in people’s lives and work and network security is becoming a critical part. A simple firewall, which can only scan the bottom four OSI layers, cannot satisfy all security requirements. An intrusion detection system (IDS) with deep packet inspection, which can filter all seven OSI layers, is becoming necessary for more and more networks. However, the processing throughputs of the IDSs are far behind the current network speed. People have begun to improve the performance of the IDSs by implementing them on different hardware platforms, such as Field-Programmable Gate Array (FPGA) or some special network processors. Nevertheless, all of these options are either less flexible or more expensive to deploy. This research focuses on some possibilities of implementing a parallelized IDS on a general computer environment based on Snort, which is the most popular open-source IDS at the moment. In this thesis, some possible methods have been analyzed for the parallelization of the pattern-matching engine based on a multicore computer. However, owing to the small granularity of the network packets, the pattern-matching engine of Snort is unsuitable for parallelization. In addition, a pipelined structure of Snort has been implemented and analyzed. The universal packet capture API - LibPCAP has been modified for a new feature, which can capture a packet directly to an external buffer. Then, the performance of the pipelined Snort can have an improvement up to 60% on an Intel i7 multicore computer for jumbo frames. A primary limitation is on the memory bandwidth. With a higher bandwidth, the performance of the parallelization can be further improved