Network Threat Detection Using Machine/Deep Learning in SDN-Based Platforms: A Comprehensive Analysis of State-of-the-Art Solutions, Discussion, Challenges, and Future Research Direction

A revolution in network technology has been ushered in by software defined networking (SDN), which makes it possible to control the network from a central location and provides an overview of the network’s security. Despite this, SDN has a single point of failure that increases the risk of potential threats. Network intrusion detection systems (NIDS) prevent intrusions into a network and preserve the network’s integrity, availability, and confidentiality. Much work has been done on NIDS but there are still improvements needed in reducing false alarms and increasing threat detection accuracy. Recently advanced approaches such as deep learning (DL) and machine learning (ML) have been implemented in SDN-based NIDS to overcome the security issues within a network. In the first part of this survey paper, we offer an introduction to the NIDS theory, as well as recent research that has been conducted on the topic. After that, we conduct a thorough analysis of the most recent ML- and DL-based NIDS approaches to ensure reliable identification of potential security risks. Finally, we focus on the opportunities and difficulties that lie ahead for future research on SDN-based ML and DL for NIDS.publishedVersio

Towards Scalable Network Traffic Measurement With Sketches

Driven by the ever-increasing data volume through the Internet, the per-port speed of network devices reached 400 Gbps, and high-end switches are capable of processing 25.6 Tbps of network traffic. To improve the efficiency and security of the network, network traffic measurement becomes more important than ever. For fast and accurate traffic measurement, managing an accurate working set of active flows (WSAF) at line rates is a key challenge. WSAF is usually located in high-speed but expensive memories, such as TCAM or SRAM, and thus their capacity is quite limited. To scale up the per-flow measurement, we pursue three thrusts. In the first thrust, we propose to use In-DRAM WSAF and put a compact data structure (i.e., sketch) called FlowRegulator before WSAF to compensate for DRAM\u27s slow access time. Per our results, FlowRegulator can substantially reduce massive influxes to WSAF without compromising measurement accuracy. In the second thrust, we integrate our sketch into a network system and propose an SDN-based WLAN monitoring and management framework called RFlow+, which can overcome the limitations of existing traffic measurement solutions (e.g., OpenFlow and sFlow), such as a limited view, incomplete flow statistics, and poor trade-off between measurement accuracy and CPU/network overheads. In the third thrust, we introduce a novel sampling scheme to deal with the poor trade-off that is provided by the standard simple random sampling (SRS). Even though SRS has been widely used in practice because of its simplicity, it provides non-uniform sampling rates for different flows, because it samples packets over an aggregated data flow. Starting with a simple idea that independent per-flow packet sampling provides the most accurate estimation of each flow, we introduce a new concept of per-flow systematic sampling, aiming to provide the same sampling rate across all flows. In addition, we provide a concrete sampling method called SketchFlow, which approximates the idea of the per-flow systematic sampling using a sketch saturation event

Realising the Network Service Federation vision

In Press / En PrensaThe 5G-TRANSFORMER project proposes an NFV/SDN-based architecture to manage the end-to-end deployment of composite NFV network services, which may involve multiple administrative domains, hence, requiring network service federation capabilities. At the architectural level, this article presents the service federation functionality of the 5G-TRANSFORMER service orchestrator. It covers the gaps identified in ETSI NFV reports and specifications (e.g., IFA028). Some recommendations are also presented based on this experience, particularly on the relevance of multi-domain resource orchestration. Experimental results show that the federated service under evaluation is deployed in less than 5 minutes. Time profiling of the various processing federation-related operation shows its reduced impact in the experienced deployment time. A comparison of service deployments of increasing complexity also offers valuable insights.This work has been partially funded by the EC H2020 5G-Transformer Project (grant no. 761536), by MINECO grant TEC2017-88373-R (5G-REFINE) and Generalitat de Catalunya grant 2017 SGR 1195

Intrusion Detection System against Denial of Service attack in Software-Defined Networking

Das exponentielle Wachstum der Online-Dienste und des über die Kommunikationsnetze übertragenen Datenvolumens macht es erforderlich, die Struktur traditioneller Netzwerke durch ein neues Paradigma zu ersetzen, das sich den aktuellen Anforderungen anpasst. Software-Defined Networking (SDN) ist hierfür eine fortschrittliche Netzwerkarchitektur, die darauf abzielt, das traditionelle Netzwerk in ein flexibleres Netzwerk umzuwandeln, das sich an die wachsenden Anforderungen anpasst. Im Gegensatz zum traditionellen Netzwerk ermöglicht SDN die Entkopplung von Steuer- und Datenebene, um Netzwerkressourcen effizient zu überwachen, zu konfigurieren und zu optimieren. Es verfügt über einen zentralisierten Controller mit einer globalen Netzwerksicht, der seine Ressourcen über programmierbare Schnittstellen verwaltet. Die zentrale Steuerung bringt jedoch neue Sicherheitsschwachstellen mit sich und fungiert als Single Point of Failure, den ein böswilliger Benutzer ausnutzen kann, um die normale Netzwerkfunktionalität zu stören. So startet der Angreifer einen massiven Datenverkehr, der als Distributed-Denial-of-Service Angriff (DDoSAngriff) von der SDN-Infrastrukturebene in Richtung des Controllers bekannt ist. Dieser DDoS-Angriff führt zu einer Sättigung der Steuerkanal-Bandbreite und belegt die Ressourcen des Controllers. Darüber hinaus erbt die SDN-Architektur einige Angriffsarten aus den traditionellen Netzwerken. Der Angreifer fälscht beispielweise die Pakete, um gutartig zu erscheinen, und zielt dann auf die traditionellen DDoS-Ziele wie Hosts, Server, Anwendungen und Router ab. In dieser Arbeit wird das Verhalten von böswilligen Benutzern untersucht. Anschließend wird ein Intrusion Detection System (IDS) zum Schutz der SDN-Umgebung vor DDoS-Angriffen vorgestellt. Das IDS berücksichtigt dabei drei Ansätze, um ausreichendes Feedback über den laufenden Verkehr durch die SDN-Architektur zu erhalten: die Informationen von einem externen Gerät, den OpenFlow-Kanal und die Flow-Tabelle. Daher besteht das vorgeschlagene IDS aus drei Komponenten. Das Inspector Device verhindert, dass böswillige Benutzer einen Sättigungsangriff auf den SDN-Controller starten. Die Komponente Convolutional Neural Network (CNN) verwendet eindimensionale neuronale Faltungsnetzwerke (1D-CNN), um den Verkehr des Controllers über den OpenFlow-Kanal zu analysieren. Die Komponente Deep Learning Algorithm(DLA) verwendet Recurrent Neural Networks (RNN), um die vererbten DDoS-Angriffe zu erkennen. Sie unterstützt auch die Unterscheidung zwischen bösartigen und gutartigen Benutzern als neue Gegenmaßnahme. Am Ende dieser Arbeit werden alle vorgeschlagenen Komponenten mit dem Netzwerkemulator Mininet und der Programmiersprache Python modelliert, um ihre Machbarkeit zu testen. Die Simulationsergebnisse zeigen hierbei, dass das vorgeschlagene IDS im Vergleich zu mehreren Benchmarking- und State-of-the-Art-Vorschlägen überdurchschnittliche Leistungen erbringt.The exponential growth of online services and the data volume transferred over the communication networks raises the need to change the structure of traditional networks to a new paradigm that adapts to the development’s demands. Software- Defined Networking (SDN) is an advanced network architecture aiming to evolve and transform the traditional network into a more flexible network that responds to the new requirements. In contrast to the traditional network, SDN allows decoupling of the control and data planes functionalities to monitor, configure, and optimize network resources efficiently. It has a centralized controller with a global network view to manage its resources using programmable interfaces. The central control brings new security vulnerabilities and acts as a single point of failure, which the malicious user might exploit to disrupt the network functionality. Thus, the attacker launches massive traffic known as Distributed Denial of Service (DDoS) attack from the SDN infrastructure layer towards the controller. This DDoS attack leads to saturation of control channel bandwidth and destroys the controller resources. Furthermore, the SDN architecture inherits some attacks types from the traditional networks. Therefore, the attacker forges the packets to appear benign and then targets the traditional DDoS objectives such as hosts, servers, applications, routers. This work observes the behavior of malicious users. It then presents an Intrusion Detection System (IDS) to safeguard the SDN environment against DDoS attacks. The IDS considers three approaches to obtain sufficient feedback about the ongoing traffic through the SDN architecture: the information from an external device, the OpenFlow channel, and the flow table. Therefore, the proposed IDS consists of three components; Inspector Device prevents the malicious users from launching the saturation attack towards the SDN controller. Convolutional Neural Network (CNN) Component employs the One- Dimensional Convolutional Neural Networks (1D-CNN) to analyze the controller’s traffic through the OpenFlow Channel. The Deep Learning Algorithm (DLA) component employs Recurrent Neural Networks (RNN) to detect the inherited DDoS attacks. The IDS also supports distinguishing between malicious and benign users as a new countermeasure. At the end of this work, the network emulator Mininet and the programming language python model all the proposed components to test their feasibility. The simulation results demonstrate that the proposed IDS outperforms compared several benchmarking and state-of-the-art suggestions

Software Defined Application Delivery Networking

In this thesis we present the architecture, design, and prototype implementation details of AppFabric. AppFabric is a next generation application delivery platform for easily creating, managing and controlling massively distributed and very dynamic application deployments that may span multiple datacenters. Over the last few years, the need for more flexibility, finer control, and automatic management of large (and messy) datacenters has stimulated technologies for virtualizing the infrastructure components and placing them under software-based management and control; generically called Software-defined Infrastructure (SDI). However, current applications are not designed to leverage this dynamism and flexibility offered by SDI and they mostly depend on a mix of different techniques including manual configuration, specialized appliances (middleboxes), and (mostly) proprietary middleware solutions together with a team of extremely conscientious and talented system engineers to get their applications deployed and running. AppFabric, 1) automates the whole control and management stack of application deployment and delivery, 2) allows application architects to define logical workflows consisting of application servers, message-level middleboxes, packet-level middleboxes and network services (both, local and wide-area) composed over application-level routing policies, and 3) provides the abstraction of an application cloud that allows the application to dynamically (and automatically) expand and shrink its distributed footprint across multiple geographically distributed datacenters operated by different cloud providers. The architecture consists of a hierarchical control plane system called Lighthouse and a fully distributed data plane design (with no special hardware components such as service orchestrators, load balancers, message brokers, etc.) called OpenADN . The current implementation (under active development) consists of ~10000 lines of python and C code. AppFabric will allow applications to fully leverage the opportunities provided by modern virtualized Software-Defined Infrastructures. It will serve as the platform for deploying massively distributed, and extremely dynamic next generation application use-cases, including: Internet-of-Things/Cyber-Physical Systems: Through support for managing distributed gather-aggregate topologies common to most Internet-of-Things(IoT) and Cyber-Physical Systems(CPS) use-cases. By their very nature, IoT and CPS use cases are massively distributed and have different levels of computation and storage requirements at different locations. Also, they have variable latency requirements for their different distributed sites. Some services, such as device controllers, in an Iot/CPS application workflow may need to gather, process and forward data under near-real time constraints and hence need to be as close to the device as possible. Other services may need more computation to process aggregated data to drive long term business intelligence functions. AppFabric has been designed to provide support for such very dynamic, highly diversified and massively distributed application use-cases. Network Function Virtualization: Through support for heterogeneous workflows, application-aware networking, and network-aware application deployments, AppFabric will enable new partnerships between Application Service Providers (ASPs) and Network Service Providers (NSPs). An application workflow in AppFabric may comprise of application services, packet and message-level middleboxes, and network transport services chained together over an application-level routing substrate. The Application-level routing substrate allows policy-based service chaining where the application may specify policies for routing their application traffic over different services based on application-level content or context. Virtual worlds/multiplayer games: Through support for creating, managing and controlling dynamic and distributed application clouds needed by these applications. AppFabric allows the application to easily specify policies to dynamically grow and shrink the application\u27s footprint over different geographical sites, on-demand. Mobile Apps: Through support for extremely diversified and very dynamic application contexts typical of such applications. Also, AppFabric provides support for automatically managing massively distributed service deployment and controlling application traffic based on application-level policies. This allows mobile applications to provide the best Quality-of-Experience to its users without This thesis is the first to handle and provide a complete solution for such a complex and relevant architectural problem that is expected to touch each of our lives by enabling exciting new application use-cases that are not possible today. Also, AppFabric is a non-proprietary platform that is expected to spawn lots of innovations both in the design of the platform itself and the features it provides to applications. AppFabric still needs many iterations, both in terms of design and implementation maturity. This thesis is not the end of journey for AppFabric but rather just the beginning

Industrial Internet of Things Driven by SDN Platform for Smart Grid Resiliency

Software defined networking (SDN) is a key enabling technology of industrial Internet of things (IIoT) that provides dynamic reconfiguration to improve data network robustness. In the context of smart grid infrastructure, the strong demand of seamless data transmission during critical events (e.g. failures or natural disturbances) seems to be fundamentally shifting energy attitude towards emerging technology. Therefore, SDN will play a vital role on energy revolution to enable flexible interfacing between smart utility domains and facilitate the integration of mix renewable energy resources to deliver efficient power of sustainable grid. In this regard, we propose a new SDN platform based on IIoT technology to support resiliency by reacting immediately whenever a failure occurs to recover smart grid networks using real-time monitoring techniques. We employ SDN controller to achieve multi-functionality control and optimization challenge by providing operators with real-time data monitoring to manage demand, resources and increasing system reliability. Data processing will be used to manage resources at local network level by employing SDN switch segment, which is connected to SDN controller though IIoT aggregation node. Furthermore, we address different scenarios to control packet flows between switches on hub-to-hub basis using traffic indicators of the infrastructure layer, in addition to any other data from the application layer. Extensive experimental simulation is conducted to demonstrate the validation of the proposed platform model. The experimental results prove the innovative SDN based IIoT solutions can improve grid reliability for enhancing smart grid resilience

Flexible architecture for the future internet scalability of SDN control plane

Software-Defined Networking (SDN) separates the control plane from the data plane. The initial SDN approach involves a single centralized controller, which may not scale properly as a network grows in size. Distributed controllers have emerged to address the disadvantages of a single centralized controller. The control architecture needs to be distributed with traffic control between switches and controllers and among the controllers in order to allow SDNs for several thousand switches. One of the most significant research challenges for distributed controller architectures is to effectively manage controllers, which includes allocating enough controllers to appropriate network locations. To address these daunting issues, we make the following major contributions: This thesis expands the method of solving the Control Placement Problem (CPP) based on the K-means and K-center algorithms to include a Hierarchical Controller Placement Problem (HCPP), located at a high level of Super Controller (SC), a middle level of Master Controllers (MCs), and the lowest level of domain controllers (DCs). The optimization metric addresses latency between the controller and the switches assigned to it.. The proposed architecture and methodology are implemented using the topology of Western European NRENs from the Internet Topology Zoo. The entire network topology is divided into clusters, and the optimal number of controllers (DCs) and their placement are determined for each cluster. MC placement optimization determines the optimal number of MCs and their optimal placement. As a second contribution, an accumulated latency is defined to solve CPP, which takes into account both the latency between the controller and its associated switches and the latency between controllers. Under the constraint of latency, an optimization problem is formulated as per mixed-integer linear programming (MILP). The goal of the research is to reduce accumulated latency while also reducing the number of network controllers and optimizing their placement to achieve an optimal balance. The performance of the developed method is evaluated on Internet2 OS3E real network topology. To achieve the third objective, a metric was developed that includes reliability. The communication latency between controllers should also be considered because a low controller-switch delay does not always imply a short controller-controller delay for a particular controller placement. As the third contribution, we propose a novel metric for CPP to improve the reliability of controllers that takes into account both communication latency and communication reliability between switches and controllers, as well as between controllers. When a single link fails, reliability is taken into account. This aspect concluded by identifying the optimal controller placement to achieve low latencies in control plane traffic. The goal of this project is to reduce the average latency. As the fourth contribution, this study evaluates the Joint Latency and Reliability-aware Controller Placement (LRCP) optimization model. As the evaluation metric, control plane latency (CPL) is defined as the sum of the average switch-to-controller latency and average inter-controller latency. The latency of the control plane, utilizing the actual latencies of the real network topology, is calculated for every optimum placement in the network. In the case of a failure of the single link, the actual CPL for LRCP placements is calculated and evaluated to determine how good LRCP placements are. CPL metrics are used to compare latency and reliability metrics with other models. This study provides proof that the developed methodologies for large-scale networks are highly powerful in terms of searching for all feasible controller placements while assessing the outcomes. In addition, compared to previous work including latency among controllers and reliability for an event of single-link failure.La xarxa definida per programari (SDN) separa el pla de control del pla de dades. L’enfocament SDN inicial implica un únic controlador centralitzat, que pot no escalar correctament a mesura que la xarxa creixi de mida. Els controladors distribuïts han sorgit per abordar els inconvenients d’un únic controlador centralitzat. . Un dels reptes de recerca més importants per a les arquitectures de controladors distribuïts és gestionar de manera eficaç els controladors, que inclou l’assignació de controladors suficients a les ubicacions de xarxa adequades. Per abordar aquests problemes, fem les següents contribucions. Aquesta tesi amplia el mètode de resolució del Problema de Col·locació de Control (CPP) basat en els algorismes de K-means i K-center per incloure un Problema de Col·locació de Controladors Jeràrquics (HCPP), situat a un nivell alt de Super Controller (SC), un nivell de controladors mestres (MC) i el nivell més baix de controladors de domini (DC). La mètrica d’optimització és la latència entre el controlador i els commutadors assignats a aquest. L’arquitectura i la metodologia proposades s’implementen utilitzant la topologia de NREN d’Europa occidental de l’Internet Topology Zoo. La topologia de la xarxa es divideix en clústers i es determina el nombre òptim de controladors de domini (DC) i la seva ubicació per a cada clúster. L’optimització de la ubicació de MC determina el nombre òptim de MC i la seva col·locació òptima. Com a segona contribució, es defineix una latència acumulada per resoldre el CPP, que té en compte tant la latència entre el controlador i els seus commutadors associats com la latència entre controladors. Sota la restricció de la latència, es formula un problema d’optimització segons la programació lineal de nombres enters mixts (MILP). L’objectiu de la investigació és reduir la latència acumulada alhora que es redueix el nombre de controladors de xarxa i optimitza la seva col·locació per aconseguir un equilibri òptim. El rendiment del mètode desenvolupat s’avalua en la topologia de xarxa real d’Internet2 OS3E. Per aconseguir el tercer objectiu, es va desenvolupar una mètrica que inclou la fiabilitat. També s’ha de tenir en compte la latència de comunicació entre controladors perquè un retard baix entre el commutador i el controlador no sempre implica un retard curt del controladorcontrolador per a una ubicació concreta dels controladors. Com a tercera contribució, proposem una nova mètrica per al CPP per millorar la fiabilitat dels controladors que tingui en compte tant la latència de la comunicació com la fiabilitat de la comunicació entre commutadors i controladors, així com entre controladors. La fiabilitat es té en compte quan falla un únic enllaç identificant la col·locació òptima dels controladors per aconseguir baixes latències en el trànsit del pla de control. L’objectiu d’aquest projecte és reduir la latència mitjana. Com a quarta contribució, aquest estudi avalua el model d’optimització Joint Latency and Reliability-aware Controller Placement (LRCP). Com a mètrica d’avaluació, la latència del pla de control (CPL) es defineix com la suma de la latència mitjana de commutador a controlador i la latència mitjana entre controladors. La latència del pla de control, utilitzant les latències reals de la topologia de xarxa real, es calcula per a cada col·locació òptima a la xarxa. En el cas d’una fallida en un únicenllaç, es calcula i s’avalua el CPL real de les ubicacions LRCP per determinar com de bones són les ubicacions LRCP. Les mètriques CPL s’utilitzen per comparar les mètriques de latència i fiabilitat amb altres models. Aquest estudi proporciona la prova que les metodologies desenvolupades per a xarxes a gran escala són molt potents pel que fa a la recerca de totes les ubicacions de controladors factibles mentre s’avaluen els resultats. A més, en comparació amb el treball anterior, inclou la latència entre els controladors i la fiabilitat per a un esdeveniment de fallada d’un enllaç únic.Las redes definidas por software (SDN) separan el plano de control del plano de datos. El enfoque inicial de SDN implica un único controlador centralizado, que puede no escalar adecuadamente a medida que una red crece en tamaño. Los controladores distribuidos han surgido para abordar las desventajas de un único controlador centralizado. Uno de los retos de investigación más importantes para las arquitecturas de controladores distribuidos es la gestión eficaz de los controladores, que incluye la asignación de suficientes controladores en las ubicaciones adecuadas. Para hacer frente a estos problemas, realizamos las siguientes contribuciones principales: Esta tesis amplía el método de resolución del Problema de Colocación de Controles (CPP) basado en los algoritmos K-means y K-center para incluir un Problema de Colocación de Controladores Jerárquicos (HCPP), situado en un nivel alto de Super-controladores (SC), un nivel medio de Controladores Maestros (MC), y el nivel más bajo de controladores de dominio (DC). La métrica de optimización es la latencia entre el controlador y los conmutadores asignados al mismo. . La arquitectura y la metodología propuestas se implementan utilizando la topología de las NREN de Europa Occidental del TopologyZoo. La topología completa de la red se divide en clústeres, y se determina el número óptimo de controladores de dominio (CD) y su colocación para cada clúster. La optimización de la colocación de los MC determina el número óptimo de MC y su colocación óptima. Como segunda contribución, se define una latencia acumulada para resolver el CPP, que tiene en cuenta tanto la latencia entre el controlador y sus conmutadores asociados como la latencia entre los controladores. Bajo la restricción de la latencia, se formula un problema de optimización según la programación lineal de enteros mixtos (MILP). El objetivo es reducir la latencia acumulada al tiempo que se reduce el número de controladores de la red y se optimiza su ubicación para lograr un equilibrio óptimo. El rendimiento del método desarrollado se evalúa en la topología de Internet2 OS3E. Para lograr el tercer objetivo, se desarrolló una métrica que incluye la fiabilidad. La latencia de la comunicación entre controladores también debe tenerse en cuenta, ya que un bajo retardo entre controladores y conmutadores no siempre implica un corto retardo entre controladores para una determinada ubicación de los mismos. Como tercera contribución proponemos una nueva métrica para el CPP para mejorar la fiabilidad de los controladores que tiene en cuenta tanto la latencia de la comunicación como la fiabilidad de la comunicación entre los conmutadores y los controladores, así como entre los controladores. Se tiene en cuenta la fiabilidad cuando falla un solo enlace. Este aspecto concluye con la identificación de la ubicación óptima de los controladores para lograr bajas latencias en el tráfico del plano de control. El objetivo es reducir la latencia media. Como cuarta contribución, este estudio evalúa el modelo de optimización Joint Latency and Reliability-aware Controller Placement (LRCP). Como métrica de evaluación, la latencia del plano de control (CPL) se define como la suma de la latencia media entre conmutadores y controladores y la latencia media entre controladores. La latencia del plano de control, utilizando las latencias reales de la topología de la red, se calcula para cada ubicación óptima en la red. En el caso de un fallo de un enlace, se calcula y evalúa la CPL real para las colocaciones de LRCP con el fin de determinar lo buenas que son las colocaciones de LRCP. Las métricas CPL se utilizan para comparar las métricas de latencia y fiabilidad con otros modelos. Este estudio demuestra que las metodologías desarrolladas para redes a gran escala son muy potentes en cuanto a la búsqueda de todas las ubicaciones factibles de los controladores mientras se evalúan los resultados. Además, en comparación con los trabajos anteriores, que incluyen la latencia entre controladores y la fiabilidad para un caso de fallo de un solo enlacePostprint (published version