Modeling the Abnormality: Machine Learning-based Anomaly and Intrusion Detection in Software-defined Networks

Modern software-defined networks (SDN) provide additional control and optimal functionality over large-scale computer networks. Due to the rise in networking applications, cyber attacks have also increased progressively. Modern cyber attacks wreak havoc on large-scale SDNs, many of which are part of critical national infrastructures. Artifacts of these attacks may present as network anomalies within the core network or edge anomalies in the SDN edge. As protection, intrusion and anomaly detection must be implemented in both the edge and core. In this dissertation, we investigate and create novel network intrusion and anomaly detection techniques that can handle the next generation of network attacks. We collect and use new network metrics and statistics to perform network intrusion detection. We demonstrated that machine learning models like Random Forest classifiers effectively use network port statistics to differentiate between normal and attack traffic with up to 98% accuracy. These collected metrics are augmented to create a new open-sourced dataset that improves upon class imbalance. The developed dataset outperforms other contemporary datasets with an Fμ score of 94% and a minimum F score of 86%. We also propose SDN intrusion detection approaches that provide high confidence scores and explainability to provide additional insights and be implemented in a real-time environment. Through this, we observed that network byte and packet transmissions and their robust statistics can be significant indicators for the prevalence of any attack. Additionally, we propose an anomaly detection technique for time-series SDN edge devices. We observe precision and recall scores inversely correlate as ε increases, and ε = 6.0 yielded the best F score. Results also highlight that the best performance was achieved from data that had been moderately smoothed (0.8 ≤ α ≤ 0.4), compared to intensely smoothed or non-smoothed data. In addition, we investigated and analyzed the impact that adversarial attacks can have on machine learning-based network intrusion detection systems for SDN. Results show that the proposed attacks provide substantial deterioration of classifier performance in single SDNs, and some classifiers deteriorate up to ≈60. Finally, we proposed an adversarial attack detection framework for multi-controller SDN setups that uses inherent network architecture features to make decisions. Results indicate efficient detection performance achieved by the framework in determining and localizing the presence of adversarial attacks. However, the performance begins to deteriorate when more than 30% of the SDN controllers have become compromised. The work performed in this dissertation has provided multiple contributions to the network security research community like providing equitable open-sourced SDN datasets, promoting the usage of core network statistics for intrusion detection, proposing robust anomaly detection techniques for time-series data, and analyzing how adversarial attacks can compromise the machine learning algorithms that protect our SDNs. The results of this dissertation can catalyze future developments in network security

Detection of Denial of Service (DoS) Attacks in Local Area Networks Based on Outgoing Packets

Denial of Service (DoS) is a security threat which compromises the confidentiality of information stored in Local Area Networks (LANs) due to unauthorized access by spoofed IP addresses. DoS is harmful to LANs as the flooding of packets may delay other users from accessing the server and in severe cases, the server may need to be shut down, wasting valuable resources, especially in critical real-time services such as in e-commerce and the medical field. The objective of this project is to propose a new DoS detection system to protect organizations from unauthenticated access to important information which may jeopardize the confidentiality, privacy and integrity of information in Local Area Networks. The new DoS detection system monitors the traffic flow of packets and filters the packets based on their IP addresses to determine whether they are genuine requests for network services or DoS attacks. Results obtained demonstrate that the detection accuracy of the new DoS detection system was in good agreement with the detection accuracy from the network protocol analyzer, Wireshark. For high-rate DoS attacks, the accuracy was 100% whereas for low-rate DoS attacks, the accuracy was 67%

Integrating complex event processing and machine learning: An intelligent architecture for detecting IoT security attacks

The Internet of Things (IoT) is growing globally at a fast pace: people now find themselves surrounded by a variety of IoT devices such as smartphones and wearables in their everyday lives. Additionally, smart environments, such as smart healthcare systems, smart industries and smart cities, benefit from sensors and actuators interconnected through the IoT. However, the increase in IoT devices has brought with it the challenge of promptly detecting and combating the cybersecurity attacks and threats that target them, including malware, privacy breaches and denial of service attacks, among others. To tackle this challenge, this paper proposes an intelligent architecture that integrates Complex Event Processing (CEP) technology and the Machine Learning (ML) paradigm in order to detect different types of IoT security attacks in real time. In particular, such an architecture is capable of easily managing event patterns whose conditions depend on values obtained by ML algorithms. Additionally, a model-driven graphical tool for security attack pattern definition and automatic code generation is provided, hiding all the complexity derived from implementation details from domain experts. The proposed architecture has been applied in the case of a healthcare IoT network to validate its ability to detect attacks made by malicious devices. The results obtained demonstrate that this architecture satisfactorily fulfils its objectives.El Internet de las Cosas (IoT) está creciendo a nivel global a un ritmo acelerado: las personas ahora se encuentran rodeadas de una variedad de dispositivos IoT como smartphones y wearables en su vida cotidiana. Además, los entornos inteligentes, como los sistemas de atención médica inteligentes, las industrias inteligentes y las ciudades inteligentes, se benefician de sensores y actuadores interconectados a través del IoT. Sin embargo, el aumento de los dispositivos IoT ha traído consigo el desafío de detectar y combatir rápidamente los ataques y amenazas de ciberseguridad que los tienen como objetivo, incluyendo malware, violaciones de privacidad y ataques de denegación de servicio, entre otros. Para abordar este desafío, este documento propone una arquitectura inteligente que integra la tecnología de Procesamiento de Eventos Complejos (CEP) y el paradigma de Aprendizaje Automático (ML) con el fin de detectar diferentes tipos de ataques de seguridad en IoT en tiempo real. En particular, dicha arquitectura es capaz de gestionar fácilmente patrones de eventos cuyas condiciones dependen de los valores obtenidos por los algoritmos de ML. Además, se proporciona una herramienta gráfica impulsada por modelos para la definición de patrones de ataque de seguridad y la generación automática de código, ocultando toda la complejidad derivada de los detalles de implementación a los expertos del dominio. La arquitectura propuesta ha sido aplicada en el caso de una red de IoT de atención médica para validar su capacidad para detectar ataques realizados por dispositivos maliciosos. Los resultados obtenidos demuestran que esta arquitectura cumple satisfactoriamente sus objetivos.This work was supported by the Spanish Ministry of Science, Innovation and Universities and the European Union FEDER Funds [grant numbers FPU 17/02007, RTI2018-093608-B-C33, RTI2018- 098156-B-C52 and RED2018-102654-T]. This work was also sup- ported by the JCCM [grant number SB-PLY/17/180501/ 0 0 0353

A hybrid and cross-protocol architecture with semantics and syntax awareness to improve intrusion detection efficiency in Voice over IP environments

Includes abstract.Includes bibliographical references (leaves 134-140).Voice and data have been traditionally carried on different types of networks based on different technologies, namely, circuit switching and packet switching respectively. Convergence in networks enables carrying voice, video, and other data on the same packet-switched infrastructure, and provides various services related to these kinds of data in a unified way. Voice over Internet Protocol (VoIP) stands out as the standard that benefits from convergence by carrying voice calls over the packet-switched infrastructure of the Internet. Although sharing the same physical infrastructure with data networks makes convergence attractive in terms of cost and management, it also makes VoIP environments inherit all the security weaknesses of Internet Protocol (IP). In addition, VoIP networks come with their own set of security concerns. Voice traffic on converged networks is packet-switched and vulnerable to interception with the same techniques used to sniff other traffic on a Local Area Network (LAN) or Wide Area Network (WAN). Denial of Service attacks (DoS) are among the most critical threats to VoIP due to the disruption of service and loss of revenue they cause. VoIP systems are supposed to provide the same level of security provided by traditional Public Switched Telephone Networks (PSTNs), although more functionality and intelligence are distributed to the endpoints, and more protocols are involved to provide better service. A new design taking into consideration all the above factors with better techniques in Intrusion Detection are therefore needed. This thesis describes the design and implementation of a host-based Intrusion Detection System (IDS) that targets VoIP environments. Our intrusion detection system combines two types of modules for better detection capabilities, namely, a specification-based and a signaturebased module. Our specification-based module takes the specifications of VoIP applications and protocols as the detection baseline. Any deviation from the protocol’s proper behavior described by its specifications is considered anomaly. The Communicating Extended Finite State Machines model (CEFSMs) is used to trace the behavior of the protocols involved in VoIP, and to help exchange detection results among protocols in a stateful and cross-protocol manner. The signature-based module is built in part upon State Transition Analysis Techniques which are used to model and detect computer penetrations. Both detection modules allow for protocol-syntax and protocol-semantics awareness. Our intrusion detection uses the aforementioned techniques to cover the threats propagated via low-level protocols such as IP, ICMP, UDP, and TCP

Emulator for Distributed DDoS Datasets (EDDD)

With the rapid escalation in prevalence and severity of Distributed Denial of Service (DDoS) attacks, the need for robust and effective countermeasures has become paramount. This thesis presents a unique approach to tackling this issue through the development of an emulator tool that generates distributed DDoS datasets. Addressing the limitations of existing, predominantly centralized DDoS datasets, this tool provides a distributed perspective, offering critical insights into the dynamics of these attacks. Built upon the open-source flexibility of Network Simulator 3 (NS3), the emulator is capable of modeling SYN flood traffic, ICMP flood traffic, and legitimate traffic, each one based on pre-existing datasets, thereby increasing the richness and realism of simulated DDoS scenarios. The tool’s architectural design allows for comprehensive configuration of network structures that can realistically span multiple countries, significantly enhancing the range of attack scenarios that can be explored. Providing outputs in the widely used PCAP format and featuring a straightforward command-line interface, the tool is designed to be highly accessible for both research and deployed applications. In essence, this tool constitutes a significant step forward in DDoS research, laying a solid foundation for future enhancements. It stands as a testament to the potential for improving our understanding and mitigation strategies in the face of increasingly complex and destructive DDoS attacks. The insights it offers into attack dynamics mark a valuable addition to the ongoing efforts in network security

Transfer Learning for Detecting Unknown Network Attacks

Network attacks are serious concerns in today’s increasingly interconnected society. Recent studies have applied conventional machine learning to network attack detection by learning the patterns of the network behaviors and training a classification model. These models usually require large labeled datasets; however, the rapid pace and unpredictability of cyber attacks make this labeling impossible in real time. To address these problems, we proposed utilizing transfer learning for detecting new and unseen attacks by transferring the knowledge of the known attacks. In our previous work, we have proposed a transfer learning-enabled framework and approach, called HeTL, which can find the common latent subspace of two different attacks and learn an optimized representation, which was invariant to attack behaviors’ changes. However, HeTL relied on manual pre-settings of hyper-parameters such as relativeness between the source and target attacks. In this paper, we extended this study by proposing a clustering-enhanced transfer learning approach, called CeHTL, which can automatically find the relation between the new attack and known attack. We evaluated these approaches by stimulating scenarios where the testing dataset contains different attack types or subtypes from the training set. We chose several conventional classification models such as decision trees, random forests, KNN, and other novel transfer learning approaches as strong baselines. Results showed that proposed HeTL and CeHTL improved the performance remarkably. CeHTL performed best, demonstrating the effectiveness of transfer learning in detecting new network attacks

Modeling Cyber Situational Awareness through Data Fusion

Cyber attacks are compromising networks faster than administrators can respond. Network defenders are unable to become oriented with these attacks, determine the potential impacts, and assess the damages in a timely manner. Since the observations of network sensors are normally disjointed, analysis of the data is overwhelming and time is not spent efficiently. Automation in defending cyber networks requires a level of reasoning for adequate response. Current automated systems are mostly limited to scripted responses. Better defense tools are required. This research develops a framework that aggregates data from heterogeneous network sensors. The collected data is correlated into a single model that is easily interpreted by decision-making entities. This research proposes and tests an impact rating system that estimates the feasibility of an attack and its potential level of impact against the targeted network host as well the other hosts that reside on the network. The impact assessments would allow decision makers to prioritize attacks in real-time and attempt to mitigate the attacks in order of their estimated impact to the network. The ultimate goal of this system is to provide computer network defense tools the situational awareness required to make the right decisions to mitigate cyber attacks in real-time

Denial-of-service attack modelling and detection for HTTP/2 services

Businesses and society alike have been heavily dependent on Internet-based services, albeit with experiences of constant and annoying disruptions caused by the adversary class. A malicious attack that can prevent establishment of Internet connections to web servers, initiated from legitimate client machines, is termed as a Denial of Service (DoS) attack; volume and intensity of which is rapidly growing thanks to the readily available attack tools and the ever-increasing network bandwidths. A majority of contemporary web servers are built on the HTTP/1.1 communication protocol. As a consequence, all literature found on DoS attack modelling and appertaining detection techniques, addresses only HTTP/1.x network traffic. This thesis presents a model of DoS attack traffic against servers employing the new communication protocol, namely HTTP/2. The HTTP/2 protocol significantly differs from its predecessor and introduces new messaging formats and data exchange mechanisms. This creates an urgent need to understand how malicious attacks including Denial of Service, can be launched against HTTP/2 services. Moreover, the ability of attackers to vary the network traffic models to stealthy affects web services, thereby requires extensive research and modelling. This research work not only provides a novel model for DoS attacks against HTTP/2 services, but also provides a model of stealthy variants of such attacks, that can disrupt routine web services. Specifically, HTTP/2 traffic patterns that consume computing resources of a server, such as CPU utilisation and memory consumption, were thoroughly explored and examined. The study presents four HTTP/2 attack models. The first being a flooding-based attack model, the second being a distributed model, the third and fourth are variant DoS attack models. The attack traffic analysis conducted in this study employed four machine learning techniques, namely Naïve Bayes, Decision Tree, JRip and Support Vector Machines. The HTTP/2 normal traffic model portrays online activities of human users. The model thus formulated was employed to also generate flash-crowd traffic, i.e. a large volume of normal traffic that incapacitates a web server, similar in fashion to a DoS attack, albeit with non-malicious intent. Flash-crowd traffic generated based on the defined model was used to populate the dataset of legitimate network traffic, to fuzz the machine learning-based attack detection process. The two variants of DoS attack traffic differed in terms of the traffic intensities and the inter-packet arrival delays introduced to better analyse the type and quality of DoS attacks that can be launched against HTTP/2 services. A detailed analysis of HTTP/2 features is also presented to rank relevant network traffic features for all four traffic models presented. These features were ranked based on legitimate as well as attack traffic observations conducted in this study. The study shows that machine learning-based analysis yields better classification performance, i.e. lower percentage of incorrectly classified instances, when the proposed HTTP/2 features are employed compared to when HTTP/1.1 features alone are used. The study shows how HTTP/2 DoS attack can be modelled, and how future work can extend the proposed model to create variant attack traffic models that can bypass intrusion-detection systems. Likewise, as the Internet traffic and the heterogeneity of Internet-connected devices are projected to increase significantly, legitimate traffic can yield varying traffic patterns, demanding further analysis. The significance of having current legitimate traffic datasets, together with the scope to extend the DoS attack models presented herewith, suggest that research in the DoS attack analysis and detection area will benefit from the work presented in this thesis