Anomaly Detection for Science DMZs Using System Performance Data

Science DMZs are specialized networks that enable large-scale distributed scientific research, providing efficient and guaranteed performance while transferring large amounts of data at high rates. The high-speed performance of a Science DMZ is made viable via data transfer nodes (DTNs), therefore they are a critical point of failure. DTNs are usually monitored with network intrusion detection systems (NIDS). However, NIDS do not consider system performance data, such as network I/O interrupts and context switches, which can also be useful in revealing anomalous system performance potentially arising due to external network based attacks or insider attacks. In this paper, we demonstrate how system performance metrics can be applied towards securing a DTN in a Science DMZ network. Specifically, we evaluate the effectiveness of system performance data in detecting TCP-SYN flood attacks on a DTN using DBSCAN (a density-based clustering algorithm) for anomaly detection. Our results demonstrate that system interrupts and context switches can be used to successfully detect TCP-SYN floods, suggesting that system performance data could be effective in detecting a variety of attacks not easily detected through network monitoring alone

WiFi Miner: An online apriori and sensor based wireless network Intrusion Detection System

This thesis proposes an Intrusion Detection System, WiFi Miner, which applies an infrequent pattern association rule mining Apriori technique to wireless network packets captured through hardware sensors for purposes of real time detection of intrusive or anomalous packets. Contributions of the proposed system includes effectively adapting an efficient data mining association rule technique to important problem of intrusion detection in a wireless network environment using hardware sensors, providing a solution that eliminates the need for hard-to-obtain training data in this environment, providing increased intrusion detection rate and reduction of false alarms. The proposed system, WiFi Miner, solution approach is to find frequent and infrequent patterns on pre-processed wireless connection records using infrequent pattern finding Apriori algorithm also proposed by this thesis. The proposed Online Apriori-Infrequent algorithm improves the join and prune step of the traditional Apriori algorithm with a rule that avoids joining itemsets not likely to produce frequent itemsets as their results, thereby improving efficiency and run times significantly. A positive anomaly score is assigned to each packet (record) for each infrequent pattern found while a negative anomaly score is assigned for each frequent pattern found. So, a record with final positive anomaly score is considered as anomaly based on the presence of more infrequent patterns than frequent patterns found

Modeling the Abnormality: Machine Learning-based Anomaly and Intrusion Detection in Software-defined Networks

Modern software-defined networks (SDN) provide additional control and optimal functionality over large-scale computer networks. Due to the rise in networking applications, cyber attacks have also increased progressively. Modern cyber attacks wreak havoc on large-scale SDNs, many of which are part of critical national infrastructures. Artifacts of these attacks may present as network anomalies within the core network or edge anomalies in the SDN edge. As protection, intrusion and anomaly detection must be implemented in both the edge and core. In this dissertation, we investigate and create novel network intrusion and anomaly detection techniques that can handle the next generation of network attacks. We collect and use new network metrics and statistics to perform network intrusion detection. We demonstrated that machine learning models like Random Forest classifiers effectively use network port statistics to differentiate between normal and attack traffic with up to 98% accuracy. These collected metrics are augmented to create a new open-sourced dataset that improves upon class imbalance. The developed dataset outperforms other contemporary datasets with an Fμ score of 94% and a minimum F score of 86%. We also propose SDN intrusion detection approaches that provide high confidence scores and explainability to provide additional insights and be implemented in a real-time environment. Through this, we observed that network byte and packet transmissions and their robust statistics can be significant indicators for the prevalence of any attack. Additionally, we propose an anomaly detection technique for time-series SDN edge devices. We observe precision and recall scores inversely correlate as ε increases, and ε = 6.0 yielded the best F score. Results also highlight that the best performance was achieved from data that had been moderately smoothed (0.8 ≤ α ≤ 0.4), compared to intensely smoothed or non-smoothed data. In addition, we investigated and analyzed the impact that adversarial attacks can have on machine learning-based network intrusion detection systems for SDN. Results show that the proposed attacks provide substantial deterioration of classifier performance in single SDNs, and some classifiers deteriorate up to ≈60. Finally, we proposed an adversarial attack detection framework for multi-controller SDN setups that uses inherent network architecture features to make decisions. Results indicate efficient detection performance achieved by the framework in determining and localizing the presence of adversarial attacks. However, the performance begins to deteriorate when more than 30% of the SDN controllers have become compromised. The work performed in this dissertation has provided multiple contributions to the network security research community like providing equitable open-sourced SDN datasets, promoting the usage of core network statistics for intrusion detection, proposing robust anomaly detection techniques for time-series data, and analyzing how adversarial attacks can compromise the machine learning algorithms that protect our SDNs. The results of this dissertation can catalyze future developments in network security

Exploring a Service-Based Normal Behaviour Profiling System for Botnet Detection

Effective detection of botnet traffic becomes difficult as the attackers use encrypted payload and dynamically changing port numbers (protocols) to bypass signature based detection and deep packet inspection. In this paper, we build a normal profiling-based botnet detection system using three unsupervised learning algorithms on service-based flow-based data, including self-organizing map, local outlier, and k-NN outlier factors. Evaluations on publicly available botnet data sets show that the proposed system could reach up to 91% detection rate with a false alarm rate of 5%

NeuDetect: A neural network data mining system for wireless network intrusion detection

This thesis proposes an Intrusion Detection System, NeuDetect, which applies Neural Network technique to wireless network packets captured through hardware sensors for purposes of real time detection of anomalous packets. To address the problem of high false alarm rate confronted by the current wireless intrusion detection systems, this thesis presents a method of applying the artificial neural networks technique to the wireless network intrusion detection system. The proposed system solution approach is to find normal and anomalous patterns on preprocessed wireless packet records by comparing them with training data using Back-propagation algorithm. An anomaly score is assigned to each packet by calculating the difference between the output error and threshold. If the anomaly score is positive then the wireless packet is flagged as anomalous and is negative then the packet is flagged as normal. If the anomaly score is zero or close to zero it will be flagged as an unknown attack and will be sent back to training process for re-evaluation

Unsupervised Intrusion Detection with Cross-Domain Artificial Intelligence Methods

Cybercrime is a major concern for corporations, business owners, governments and citizens, and it continues to grow in spite of increasing investments in security and fraud prevention. The main challenges in this research field are: being able to detect unknown attacks, and reducing the false positive ratio. The aim of this research work was to target both problems by leveraging four artificial intelligence techniques. The first technique is a novel unsupervised learning method based on skip-gram modeling. It was designed, developed and tested against a public dataset with popular intrusion patterns. A high accuracy and a low false positive rate were achieved without prior knowledge of attack patterns. The second technique is a novel unsupervised learning method based on topic modeling. It was applied to three related domains (network attacks, payments fraud, IoT malware traffic). A high accuracy was achieved in the three scenarios, even though the malicious activity significantly differs from one domain to the other. The third technique is a novel unsupervised learning method based on deep autoencoders, with feature selection performed by a supervised method, random forest. Obtained results showed that this technique can outperform other similar techniques. The fourth technique is based on an MLP neural network, and is applied to alert reduction in fraud prevention. This method automates manual reviews previously done by human experts, without significantly impacting accuracy