Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates

The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is legitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes computer and network outages, reduces network throughput, and costs governments and companies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a hardware circuit capable of TCP stream processing on multi-gigabit networks for millions of simultaneous network connections. Live Internet traffic is analyzed using this new TCP processing circuit

A formalization and analysis of high-speed stateful signature matching for intrusion detection

The present work is aimed to develop and analyze a novel model for distributed stateful intrusion detection able to scale in order to keep up with the pace of high speed network links. More precisely, in this work we make the following contributions: - We introduce a novel architecture for the distributed matching of stateful network-based signatures. - We present a novel algorithm that allows for the detection of complex, stateful attacks in a distributed fashion. - We provide a precise characterization of the bottlenecks that are inherent to the distributed matching of stateful signatures in the most general case. - We developed optimizing to reduce the impact of these bottlenecks and improve the performance of distributed detection. - We describe a working, yet demonstrative implementation of the system based on the Snort intrusion detection engine - We provide an evaluation of the implemented system on a real-world testbe

On Optimizing Traffic Distribution for Clusters of Network Intrusion Detection and Prevention Systems

To address the overload conditions caused by the increasing network traffic volume, recent literature in the network intrusion detection and prevention field has proposed the use of clusters of network intrusion detection and prevention systems (NIDPSs). We observe that simple traffic distribution schemes are usually used for NIDPS clusters. These schemes have two major drawbacks: (1) the loss of correlation information caused by the traffic distribution because correlated flows are not sent to the same NIDPS and (2) the unbalanced loads of the NIDPSs. The first drawback severely affects the ability to detect intrusions that require analysis of correlated flows. The second drawback greatly increases the chance of overloading an NIDPS even when loads of the others are low. In this thesis, we address these two drawbacks. In particular, we propose two novel traffic distribution systems: the Correlation-Based Load Balancer and the Correlation-Based Load Manager as two different solutions to the NIDPS traffic distribution problem. On the one hand, the Load Balancer and the Load Manager both consider the current loads of the NIDPSs while distributing traffic to provide fine-grained load balancing and dynamic load distribution, respectively. On the other hand, both systems take into account traffic correlation in their distributions, thereby significantly reducing the loss of correlation information during their distribution of traffic. We have implemented prototypes of both systems and evaluated them using extensive simulations and real traffic traces. Overall, the evaluation results show that both systems have low overhead in terms of the delays introduced to the packets. More importantly, compared to the naive hash-based distribution, the Load Balancer significantly improves the anomaly-based detection accuracy of DDoS attacks and port scans -- the two major attacks that require the analysis of correlated flows -- meanwhile, the Load Manager successfully maintains the anomaly-based detection accuracy of these two major attacks of the NIDPSs

All-optical header processing in a 42.6Gb/s optoelectronic firewall

A novel architecture to enable future network security systems to provide effective protection in the context of continued traffic growth and the need to minimise energy consumption is proposed. It makes use of an all-optical pre-filtering stage operating at the line rate under software control to distribute incoming packets to specialised electronic processors. An experimental system that integrates software controls and electronic interfaces with an all-optical pattern recognition system has demonstrated the key functions required by the new architecture. As an example, the ability to sort packets arriving in a 42.6Gb/s data stream according to their service type was shown experimentally

Hardware Acceleration of Network Intrusion Detection System Using FPGA

This thesis presents new algorithms and hardware designs for Signature-based Network Intrusion Detection System (SB-NIDS) optimisation exploiting a hybrid hardwaresoftware co-designed embedded processing platform. The work describe concentrates on optimisation of a complete SB-NIDS Snort application software on a FPGA based hardware-software target rather than on the implementation of a single functional unit for hardware acceleration. Pattern Matching Hardware Accelerator (PMHA) based on Bloom filter was designed to optimise SB-NIDS performance for execution on a Xilinx MicroBlaze soft-core processor. The Bloom filter approach enables the potentially large number of network intrusion attack patterns to be efficiently represented and searched primarily using accesses to FPGA on-chip memory. The thesis demonstrates, the viability of hybrid hardware-software co-designed approach for SB-NIDS. Future work is required to investigate the effects of later generation FPGA technology and multi-core processors in order to clearly prove the benefits over conventional processor platforms for SB-NIDS. The strengths and weaknesses of the hardware accelerators and algorithms are analysed, and experimental results are examined to determine the effectiveness of the implementation. Experimental results confirm that the PMHA is capable of performing network packet analysis for gigabit rate network traffic. Experimental test results indicate that our SB-NIDS prototype implementation on relatively low clock rate embedded processing platform performance is approximately 1.7 times better than Snort executing on a general purpose processor on PC when comparing processor cycles rather than wall clock time

INSecS: An Intelligent Network Security System

There are new challenges in network security, introduced by the nature of modern networks like IoT systems, Cloud systems, and other distributed systems. System resource limitations in IoT, delays in processing the large stream of data from Cloud and distributed system, incapability to handle multi-step attacks due to delay in updates, limited datasets used for Intrusion Detection System (IDS) training which impacts the system performance are some of the pressing issues. To address these challenges, the author proposes Intelligent Network Security Systems, a framework that can handle these issues and also be as accurate as a commercial grade IDS. The proposed framework consists of three components: a Dataset Creation Software (DCS), an Intrusion Detection System and a Learning module. This thesis presents implementation details and validation results for DCS and IDS. The first component is a highly customizable software framework capable of generating labeled network intrusion datasets on demand. This software is able to collect data from a live network as well as from a pre-recorded packet capture file. The output can be either Raw packet capture (PCAP) with selected attributes per packet or a processed dataset with customized attributes related to both individual packet features and overall traffic behavior within a time window. The abilities of this component are compared with a state-of-the-art dataset creation system through a feature comparison. The proposed Intrusion Detection System is a novel, distributed IDS that is able to perform in real-time in a distributed system. Hierarchical decision making is used to reduce traffic overhead on the IDS and allow faster Intrusion Detection. The IDS also detects multi-step attacks faster by updating the system rules when a reconnaissance attack is detected, without any human intervention. Internal attacks are also detected easily because of the distributed nature of the IDS. The performance tests show that the IDS performs 8 times faster on averages with the hierarchical decision-making structure and still maintains the same level of accuracy as Snort

Intrusion detection and response model for mobile ad hoc networks.

This dissertation presents a research whose objective is to design and develop an intrusion detection and response model for Mobile Ad hoc NETworks (MANET). Mobile ad hoc networks are infrastructure-free, pervasive and ubiquitous in nature, without any centralized authority. These unique MANET characteristics present several changes to secure them. The proposed security model is called the Intrusion Detection and Response for Mobile Ad hoc Networks (IDRMAN). The goal of the proposed model is to provide a security framework that will detect various attacks and take appropriate measures to control the attack automatically. This model is based on identifying critical system parameters of a MANET that are affected by various types of attacks, and continuously monitoring the values of these parameters to detect and respond to attacks. This dissertation explains the design and development of the detection framework and the response framework of the IDRMAN. The main aspects of the detection framework are data mining using CART to identify attack sensitive network parameters from the wealth of raw network data, statistical processing using six sigma to identify the thresholds for the attack sensitive parameters and quantification of the MANET node state through a measure called the Threat Index (TI) using fuzzy logic methodology. The main aspects of the response framework are intruder identification and intruder isolation through response action plans. The effectiveness of the detection and response framework is mathematically analyzed using probability techniques. The detection framework is also evaluated by performance comparison experiments with related models, and through performance evaluation experiments from scalability perspective. Performance metrics used for assessing the detection aspect of the proposed model are detection rate and false positive rate at different node mobility speed. Performance evaluation experiments for scalability are with respect to the size of the MANET, where more and more mobile nodes are added into the MANET at varied mobility speed. The results of both the mathematical analysis and the performance evaluation experiments demonstrate that the IDRMAN model is an effective and viable security model for MANET

Multi-wavelength, multi-beam, photonic based sensor for object discrimination and positioning

Over the last decade, substantial research efforts have been dedicated towards the development of advanced laser scanning systems for discrimination in perimeter security, defence, agriculture, transportation, surveying and geosciences. Military forces, in particular, have already started employing laser scanning technologies for projectile guidance, surveillance, satellite and missile tracking; and target discrimination and recognition. However, laser scanning is relatively a new security technology. It has previously been utilized for a wide variety of civil and military applications. Terrestrial laser scanning has found new use as an active optical sensor for indoors and outdoors perimeter security. A laser scanning technique with moving parts was tested in the British Home Office - Police Scientific Development Branch (PSDB) in 2004. It was found that laser scanning has the capability to detect humans in 30m range and vehicles in 80m range with low false alarm rates. However, laser scanning with moving parts is much more sensitive to vibrations than a multi-beam stationary optic approach. Mirror device scanners are slow, bulky and expensive and being inherently mechanical they wear out as a result of acceleration, cause deflection errors and require regular calibration. Multi-wavelength laser scanning represent a potential evolution from object detection to object identification and classification, where detailed features of objects and materials are discriminated by measuring their reflectance characteristics at specific wavelengths and matching them with their spectral reflectance curves. With the recent advances in the development of high-speed sensors and high-speed data processors, the implementation of multi-wavelength laser scanners for object identification has now become feasible. A two-wavelength photonic-based sensor for object discrimination has recently been reported, based on the use of an optical cavity for generating a laser spot array and maintaining adequate overlapping between tapped collimated laser beams of different wavelengths over a long optical path. While this approach is capable of discriminating between objects of different colours, its main drawback is the limited number of security-related objects that can be discriminated. This thesis proposes and demonstrates the concept of a novel photonic based multi-wavelength sensor for object identification and position finding. The sensor employs a laser combination module for input wavelength signal multiplexing and beam overlapping, a custom-made curved optical cavity for multi-beam spot generation through internal beam reflection and transmission and a high-speed imager for scattered reflectance spectral measurements. Experimental results show that five different laser wavelengths, namely 473nm, 532nm, 635nm, 670nm and 785nm, are necessary for discriminating various intruding objects of interest through spectral reflectance and slope measurements. Various objects were selected to demonstrate the proof of concept. We also demonstrate that the object position (coordinates) is determined using the triangulation method, which is based on the projection of laser spots along determined angles onto intruding objects and the measurement of their reflectance spectra using an image sensor. Experimental results demonstrate the ability of the multi-wavelength spectral reflectance sensor to simultaneously discriminate between different objects and predict their positions over a 6m range with an accuracy exceeding 92%. A novel optical design is used to provide additional transverse laser beam scanning for the identification of camouflage materials. A camouflage material is chosen to illustrate the discrimination capability of the sensor, which has complex patterns within a single sample, and is successfully detected and discriminated from other objects over a 6m range by scanning the laser beam spots along the transverse direction. By using more wavelengths at optimised points in the spectrum where different objects show different optical characteristics, better discrimination can be accomplished