Ultra-high throughput string matching for deep packet inspection

Deep Packet Inspection (DPI) involves searching a packet's header and payload against thousands of rules to detect possible attacks. The increase in Internet usage and growing number of attacks which must be searched for has meant hardware acceleration has become essential in the prevention of DPI becoming a bottleneck to a network if used on an edge or core router. In this paper we present a new multi-pattern matching algorithm which can search for the fixed strings contained within these rules at a guaranteed rate of one character per cycle independent of the number of strings or their length. Our algorithm is based on the Aho-Corasick string matching algorithm with our modifications resulting in a memory reduction of over 98% on the strings tested from the Snort ruleset. This allows the search structures needed for matching thousands of strings to be small enough to fit in the on-chip memory of an FPGA. Combined with a simple architecture for hardware, this leads to high throughput and low power consumption. Our hardware implementation uses multiple string matching engines working in parallel to search through packets. It can achieve a throughput of over 40 Gbps (OC-768) when implemented on a Stratix 3 FPGA and over 10 Gbps (OC-192) when implemented on the lower power Cyclone 3 FPGA

Packet Inspection on Programmable Hardware

In the network security system, one of the issues that are being discussed is to conduct a quick inspection of all incoming and outgoing packet. In this paper, we make a design packet inspection systems using programmable hardware. We propose the packet inspection system using a Field Programmable Gate Array (FPGA). The system proposed consisting of two important parts. The first part is to scanning packet very fast and the second is for verifying the results of scanning the first part. On the first part, the system based on incoming packet contents, the packet can reduce the number of strings to be matched for each packet and, accordingly, feed the packet to a verifier in the second part to conduct accurate string matching. In this paper a novel multi-threading finite state machine is proposed, which improves the clock frequency and allows multiple packets to be examined by a single state machine simultaneously. Design techniques for high-speed interconnect and interface circuits are also presented. The results of our experiment show that the system performance depend on the string matching algorithm, design on FPGA, and the number of string to be matched. Keywords: Packet inspection, string matching, Field programmable gate array, Traffic classificatio

Scalable NIDS via Negative Pattern Matching and Exclusive Pattern Matching

In this paper, we identify the unique challenges in deploying parallelism on TCAM-based pattern matching for Network Intrusion Detection Systems (NIDSes). We resolve two critical issues when designing scalable parallelism specifically for pattern matching modules: 1) how to enable fine-grained parallelism in pursuit of effective load balancing and desirable speedup simultaneously; and 2) how to reconcile the tension between parallel processing speedup and prohibitive TCAM power consumption. To this end, we first propose the novel concept of Negative Pattern Matching to partition flows, by which the number of TCAM lookups can be significantly reduced, and the resulting (fine-grained) flow segments can be inspected in parallel without incurring false negatives. Then we propose the notion of Exclusive Pattern Matching to divide the entire pattern set into multiple subsets which can later be matched against selectively and independently without affecting the correctness. We show that Exclusive Pattern Matching enables the adoption of smaller and faster TCAM blocks and improves both the pattern matching speed and scalability. Finally, our theoretical and experimental results validate that the above two concepts are inherently complementary, enabling our integrated scheme to provide performance gain in any scenario (with either clean or dirty traffic).Department of ComputingRefereed conference pape

The HSS/SNiC : a conceptual framework for collapsing security down to the physical layer

This work details the concept of a novel network security model called the Super NIC (SNIC) and a Hybrid Super Switch (HSS). The design will ultimately incorporate deep packet inspection (DPI), intrusion detection and prevention (IDS/IPS) functions, as well as network access control technologies therefore making all end-point network devices inherently secure. The SNIC and HSS functions are modelled using a transparent GNU/Linux Bridge with the Netfilter framework

Energy Efficient Hardware Accelerators for Packet Classification and String Matching

This thesis focuses on the design of new algorithms and energy efficient high throughput hardware accelerators that implement packet classification and fixed string matching. These computationally heavy and memory intensive tasks are used by networking equipment to inspect all packets at wire speed. The constant growth in Internet usage has made them increasingly difficult to implement at core network line speeds. Packet classification is used to sort packets into different flows by comparing their headers to a list of rules. A flow is used to decide a packet’s priority and the manner in which it is processed. Fixed string matching is used to inspect a packet’s payload to check if it contains any strings associated with known viruses, attacks or other harmful activities. The contributions of this thesis towards the area of packet classification are hardware accelerators that allow packet classification to be implemented at core network line speeds when classifying packets using rulesets containing tens of thousands of rules. The hardware accelerators use modified versions of the HyperCuts packet classification algorithm. An adaptive clocking unit is also presented that dynamically adjusts the clock speed of a packet classification hardware accelerator so that its processing capacity matches the processing needs of the network traffic. This keeps dynamic power consumption to a minimum. Contributions made towards the area of fixed string matching include a new algorithm that builds a state machine that is used to search for strings with the aid of default transition pointers. The use of default transition pointers keep memory consumption low, allowing state machines capable of searching for thousands of strings to be small enough to fit in the on-chip memory of devices such as FPGAs. A hardware accelerator is also presented that uses these state machines to search through the payloads of packets for strings at core network line speeds