DeepRoute: Herding Elephant and Mice Flows with Reinforcement Learning

International audienceWide area networks are built to have enough resilience and flexibility, such as offering many paths between multiple pairs of end-hosts. To prevent congestion, current practices involve numerous tweaking of routing tables to optimize path computation, such as flow diversion to alternate paths or load balancing. However, this process is slow, costly and require difficult online decision-making to learn appropriate settings, such as flow arrival rate, workload, and current network environment. Inspired by recent advances in AI to manage resources, we present DeepRoute, a model-less reinforcement learning approach that translates the path computation problem to a learning problem. Learning from the network environment, DeepRoute learns strategies to manage arriving elephant and mice flows to improve the average path utilization in the network. Comparing to other strategies such as prioritizing certain flows and random decisions, DeepRoute is shown to improve average network path utilization to 30% and potentially reduce possible congestion across the whole network. This paper presents results in simulation and also how DeepRoute can be demonstrated by a Mininet implementation

데이터 센터 내의 다중경로 전송을 위한 동적 부하 균형 기법

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 권태경 .Various applications require the data center networks to carry their traffic efficiently. The data center networks usually have a hierarchical topology and exhibit distinct traffic patterns, which is different from the traditional Internet. These features have driven the data center networks to reduce the flow completion time (FCT) and to achieve high throughput. One of the possible solutions is balancing network loads across multiple paths by leveraging transport mechanisms like Equal-Cost MultiPath (ECMP) routing. ECMP allows flows to exploit multiple paths by hashing the metadata of the flows. However, due to the random nature of hash functions, ECMP often distributes the traffic unevenly, which makes it hard to utilize the links' full capacity. Thus, we propose an adaptive load balancing mechanism for multiple paths in data centers, called MaxPass, to complement ECMP. A sender adaptively selects and dynamically changes multiple paths depending on the current network status like congestion. To monitor the network status, the corresponding receiver transmits a probe packet periodically to the senderits loss indicates a traffic congestion. We implemented MaxPass using commodity switches and carry out the quantitative analysis on the ns-2 simulator to show that MaxPass can improve the FCT and the throughput.데이터 센터 내에서 동작하는 다양한 어플리케이션들은 네트워크 트래픽을 보다 효율적으로 사용할 것을 요구한다. 데이터 센터 네트워크는 기존의 인터넷과는 다른 다중 루트 계층적 토폴로지로 구성되어 있으며, 다양한 트래픽 패턴을 가지고 있다. 이러한 특징으로 인해 데이터 센터 네트워크는 짧은 플로우 처리 완료 시간 (Flow Completion Time)과 높은 처리량 (Throughput)을 요구한다. 데이터 센터가 요구하는 조건들을 만족시키기 위한 방법 중 하나로는 등가 다중 경로 (Equal-Cost Multi-Path)와 같은 라우팅 기법을 활용하여 네트워크 부하를 서로 다른 링크에 분산시키는 것이 있다. 등가 다중 경로 라우팅 기법은 플로우의 메타 데이터를 해싱하여 플로우가 여러 경로를 이용할 수 있도록 한다. 그러나 랜덤한 결과를 도출하는 해시 함수의 특성에 따라 등가 다중 경로 라우팅 기법은 종종 트래픽을 고르게 분배하지 못함으로써, 링크의 전체 용량을 활용하기에는 한계가 있다. 따라서 본 논문에서는 데이터 센터 네트워크 환경에 맞는 새로운 부하 균형 배분 기법인 맥스패스 (MaxPass)를 제안한다. 맥스패스 내에서 데이터 송신자는 현재 네트워크 상태에 따라 경로를 동적으로 선택하고 변경한다. 데이터 수신자는 현재 네트워크 상태를 파악하기 위해 탐색 패킷을 주기적으로 송신자에게 보내고, 탐색 패킷 드랍 여부에 따라 혼잡도를 파악한다. 본 논문은 실제 스위치에서 맥스패스를 구현하였으며, ns-2 시뮬레이션을 기반한 실험을 통해 제안한 기법에 관하여 정량적 수치 분석을 수행하고, 플로우 처리 완료 시간과 링크 처리량의 성능 향상이 있음을 보여 준다.Chapter 1 Introduction 1 Chapter 2 Background 5 2.1 Data Center Network Topology . . . . . . . . . . . . . . . . . 5 2.2 Multipath Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Multipath Transport Protocol . . . . . . . . . . . . . . . . . . . 7 2.4 Credit-based Congestion Control . . . . . . . . . . . . . . . . 8 Chapter 3 MaxPass 10 3.1 Design Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.2 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.3 Path Probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.4 Adaptive Path Selection . . . . . . . . . . . . . . . . . . . . . . .15 3.5 Feedback Control Algorithm . . . . . . . . . . . . . . . . . . . 15 3.6 Credit Stop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 Chapter 4 Evaluation 20 4.1 Ns-2 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.1.1 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1.2 Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 4.1.3 Flow Completion Time (FCT) . . . . . . . . . . . . . . . .25 4.2 Testbed Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 25 Chapter 5 Related Work 28 5.1 Centralized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28 5.2 Decentralized/Distributed . . . . . . . . . . . . . . . . . . . . . .30 Chapter 6 Conclusion 32 초록 38Maste

VNARs: An Ancient and Unique Repertoire of Molecules That Deliver Small, Soluble, Stable and High Affinity Binders of Proteins

Peer reviewedPublisher PD

Traffic Optimization in Data Center and Software-Defined Programmable Networks

L'abstract è presente nell'allegato / the abstract is in the attachmen

Optimizing Flow Routing Using Network Performance Analysis

Relevant conferences were attended at which work was often presented and several papers were published in the course of this project. • Muna Al-Saadi, Bogdan V Ghita, Stavros Shiaeles, Panagiotis Sarigiannidis. A novel approach for performance-based clustering and management of network traffic flows, IWCMC, ©2019 IEEE. • M. Al-Saadi, A. Khan, V. Kelefouras, D. J. Walker, and B. Al-Saadi: Unsupervised Machine Learning-Based Elephant and Mice Flow Identification, Computing Conference 2021. • M. Al-Saadi, A. Khan, V. Kelefouras, D. J. Walker, and B. Al-Saadi: SDN-Based Routing Framework for Elephant and Mice Flows Using Unsupervised Machine Learning, Network, 3(1), pp.218-238, 2023.The main task of a network is to hold and transfer data between its nodes. To achieve this task, the network needs to find the optimal route for data to travel by employing a particular routing system. This system has a specific job that examines each possible path for data and chooses the suitable one and transmit the data packets where it needs to go as fast as possible. In addition, it contributes to enhance the performance of network as optimal routing algorithm helps to run network efficiently. The clear performance advantage that provides by routing procedures is the faster data access. For example, the routing algorithm take a decision that determine the best route based on the location where the data is stored and the destination device that is asking for it. On the other hand, a network can handle many types of traffic simultaneously, but it cannot exceed the bandwidth allowed as the maximum data rate that the network can transmit. However, the overloading problem are real and still exist. To avoid this problem, the network chooses the route based on the available bandwidth space. One serious problem in the network is network link congestion and disparate load caused by elephant flows. Through forwarding elephant flows, network links will be congested with data packets causing transmission collision, congestion network, and delay in transmission. Consequently, there is not enough bandwidth for mice flows, which causes the problem of transmission delay. Traffic engineering (TE) is a network application that concerns with measuring and managing network traffic and designing feasible routing mechanisms to guide the traffic of the network for improving the utilization of network resources. The main function of traffic engineering is finding an obvious route to achieve the bandwidth requirements of the network consequently optimizing the network performance [1]. Routing optimization has a key role in traffic engineering by finding efficient routes to achieve the desired performance of the network [2]. Furthermore, routing optimization can be considered as one of the primary goals in the field of networks. In particular, this goal is directly related to traffic engineering, as it is based on one particular idea: to achieve that traffic is routed according to accurate traffic requirements [3]. Therefore, we can say that traffic engineering is one of the applications of multiple improvements to routing; routing can also be optimized based on other factors (not just on traffic requirements). In addition, these traffic requirements are variable depending on analyzed dataset that considered if it is data or traffic control. In this regard, the logical central view of the Software Defined Network (SDN) controller facilitates many aspects compared to traditional routing. The main challenge in all network types is performance optimization, but the situation is different in SDN because the technique is changed from distributed approach to a centralized one. The characteristics of SDN such as centralized control and programmability make the possibility of performing not only routing in traditional distributed manner but also routing in centralized manner. The first advantage of centralized routing using SDN is the existence of a path to exchange information between the controller and infrastructure devices. Consequently, the controller has the information for the entire network, flexible routing can be achieved. The second advantage is related to dynamical control of routing due to the capability of each device to change its configuration based on the controller commands [4]. This thesis begins with a wide review of the importance of network performance analysis and its role for understanding network behavior, and how it contributes to improve the performance of the network. Furthermore, it clarifies the existing solutions of network performance optimization using machine learning (ML) techniques in traditional networks and SDN environment. In addition, it highlights recent and ongoing studies of the problem of unfair use of network resources by a particular flow (elephant flow) and the possible solutions to solve this problem. Existing solutions are predominantly, flow routing-based and do not consider the relationship between network performance analysis and flow characterization and how to take advantage of it to optimize flow routing by finding the convenient path for each type of flow. Therefore, attention is given to find a method that may describe the flow based on network performance analysis and how to utilize this method for managing network performance efficiently and find the possible integration for the traffic controlling in SDN. To this purpose, characteristics of network flows is identified as a mechanism which may give insight into the diversity in flow features based on performance metrics and provide the possibility of traffic engineering enhancement using SDN environment. Two different feature sets with respect to network performance metrics are employed to characterize network traffic. Applying unsupervised machine learning techniques including Principal Component Analysis (PCA) and k-means cluster analysis to derive a traffic performance-based clustering model. Afterward, thresholding-based flow identification paradigm has been built using pre-defined parameters and thresholds. Finally, the resulting data clusters are integrated within a unified SDN architectural solution, which improves network management by finding the best flow routing based on the type of flow, to be evaluated against a number of traffic data sources and different performance experiments. The validation process of the novel framework performance has been done by making a performance comparison between SDN-Ryu controller and the proposed SDN-external application based on three factors: throughput, bandwidth,and data transfer rate by conducting two experiments. Furthermore, the proposed method has been validated by using different Data Centre Network (DCN) topologies to demonstrate the effectiveness of the network traffic management solution. The overall validation metrics shows real gains, the results show that 70% of the time, it has high performance with different flows. The proposed routing SDN traffic-engineering paradigm for a particular flow therefore, dynamically provisions network resources among different flow types

SDN-Based Routing Framework for Elephant and Mice Flows Using Unsupervised Machine Learning

Software-defined networks (SDNs) have the capabilities of controlling the efficient movement of data flows through a network to fulfill sufficient flow management and effective usage of network resources. Currently, most data center networks (DCNs) suffer from the exploitation of network resources by large packets (elephant flow) that enter the network at any time, which affects a particular flow (mice flow). Therefore, it is crucial to find a solution for identifying and finding an appropriate routing path in order to improve the network management system. This work proposes a SDN application to find the best path based on the type of flow using network performance metrics. These metrics are used to characterize and identify flows as elephant and mice by utilizing unsupervised machine learning (ML) and the thresholding method. A developed routing algorithm was proposed to select the path based on the type of flow. A validation test was performed by testing the proposed framework using different topologies of the DCN and comparing the performance of a SDN-Ryu controller with that of the proposed framework based on three factors: throughput, bandwidth, and data transfer rate. The results show that 70% of the time, the proposed framework has higher performance for different types of flows.</jats:p