Controlling Network Latency in Mixed Hadoop Clusters: Do We Need Active Queue Management?

With the advent of big data, data center applications are processing vast amounts of unstructured and semi-structured data, in parallel on large clusters, across hundreds to thousands of nodes. The highest performance for these batch big data workloads is achieved using expensive network equipment with large buffers, which accommodate bursts in network traffic and allocate bandwidth fairly even when the network is congested. Throughput-sensitive big data applications are, however, often executed in the same data center as latency-sensitive workloads. For both workloads to be supported well, the network must provide both maximum throughput and low latency. Progress has been made in this direction, as modern network switches support Active Queue Management (AQM) and Explicit Congestion Notifications (ECN), both mechanisms to control the level of queue occupancy, reducing the total network latency. This paper is the first study of the effect of Active Queue Management on both throughput and latency, in the context of Hadoop and the MapReduce programming model. We give a quantitative comparison of four different approaches for controlling buffer occupancy and latency: RED and CoDel, both standalone and also combined with ECN and DCTCP network protocol, and identify the AQM configurations that maintain Hadoop execution time gains from larger buffers within 5%, while reducing network packet latency caused by bufferbloat by up to 85%. Finally, we provide recommendations to administrators of Hadoop clusters as to how to improve latency without degrading the throughput of batch big data workloads.The research leading to these results has received funding from the European Unions Seventh Framework Programme (FP7/2007–2013) under grant agreement number 610456 (Euroserver). The research was also supported by the Ministry of Economy and Competitiveness of Spain under the contracts TIN2012-34557 and TIN2015-65316-P, Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), HiPEAC-3 Network of Excellence (ICT- 287759), and the Severo Ochoa Program (SEV-2011-00067) of the Spanish Government.Peer ReviewedPostprint (author's final draft

Load-Balancing in Local and Metro-Area networks with MPTCP and OpenFlow

In this thesis, a novel load-balancing technique for local or metro-area traffic is proposed in mesh-style topologies. The technique uses Software Defined Networking (SDN) architecture with virtual local area network (VLAN) setups typically seen in a campus or small-to-medium enterprise environment. This was done to provide a possible solution or at least a platform to expand on for the load-balancing dilemma that network administrators face today. The transport layer protocol Multi-Path TCP (MPTCP) coupled with IP aliasing is also used. The trait of MPTCP of forming multiple subflows from sender to receiver depending on the availability of IP addresses at either the sender or receiver helps to divert traffic in the subflows across all available paths. The combination of MPTCP subflows with IP aliasing enables spreading out of the traffic load across greater number of links in the network, and thereby achieving load balancing and better network utilization. The traffic formed of each subflow would be forwarded across the network based on Hamiltonian \u27paths\u27 which are created in association with each switch in the topology which are directly connected to hosts. The amount of \u27paths\u27 in the topology would also depend on the number of VLANs setup for the hosts in the topology. This segregation would allow for network administrators to monitor network utilization across VLANs and give the ability to balance load across VLANs. We have devised several experiments in Mininet, and the experimentation showed promising results with significantly better throughput and network utilization compared to cases where normal TCP was used to send traffic from source to destination. Our study clearly shows the advantages of using MPTCP for load balancing purposes in SDN type architectures and provides a platform for future research on using VLANs, SDN, and MPTCP for network traffic management

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

Multipath TCP (MPTCP) has been widely used as an efficient way for communication in many applications. Data centers, smartphones, and network operators use MPTCP to balance the traffic in a network efficiently. MPTCP is an extension of TCP (Transmission Control Protocol), which provides multiple paths, leading to higher throughput and low latency. Although MPTCP has shown better performance than TCP in many applications, it has its own challenges. The network can become congested due to heavy traffic in the multiple paths (subflows) if the subflow rates are not determined correctly. Moreover, communication latency can occur if the packets are not scheduled correctly between the subflows. This paper reviews techniques to solve the above-mentioned problems based on two main approaches; non data-driven (classical) and data-driven (Machine Learning) approaches. This paper compares these two approaches and highlights their strengths and weaknesses with a view to motivating future researchers in this exciting area of machine learning for communications. This paper also provides details on the simulation of MPTCP and its implementations in real environments.Comment: 13 pages, 7 figure

TCP performance enhancement over wireless mesh networks by means of the combination of multi-RAT devices and the MPTCP protocol

The last trends at communications realms, in particular, wireless technologies, where it is more and more usual that devices carry more than one interface (i.e. multi-RAT, Radio Access Technology), to get access to the Internet, question the classic single-path paradigm, imposed by the mainstream transport protocol, TCP. In this work we assess the behavior of Multipath TCP (MPTCP), which allows the transparent breakdown of a single TCP session into multiple simultaneous subflows. This straightforward feature might lead to remarkable performance enhancements, yielding as well a stronger resilience against failures within any of the routes. Moreover, we evaluate three different routing algorithms (link, node and zone disjoint) that aim to discover the optimal route configuration of disjoint paths over a Wireless Mesh Network (WMN), exploiting the possibilities arisen by this brand new protocol. We use the obtained results to evaluate, by means of simulation, the behavior of the MPTCP protocol, showing that the aggregated performance is significatively higher than that of achieved by the traditional single-path and single-flow TCP.The authors would like to express their gratitude to the Spanish government for its funding in the project “Connectivity as a Service: Access for the Internet of the Future”, COSAIF (TEC2012-38574-C02-01)

Traffic-aware adaptive server load balancing for software defined networks

Servers in data center networks handle heterogenous bulk loads. Load balancing, therefore, plays an important role in optimizing network bandwidth and minimizing response time. A complete knowledge of the current network status is needed to provide a stable load in the network. The process of network status catalog in a traditional network needs additional processing which increases complexity, whereas, in software defined networking, the control plane monitors the overall working of the network continuously. Hence it is decided to propose an efficient load balancing algorithm that adapts SDN. This paper proposes an efficient algorithm TA-ASLB-traffic-aware adaptive server load balancing to balance the flows to the servers in a data center network. It works based on two parameters, residual bandwidth, and server capacity. It detects the elephant flows and forwards them towards the optimal server where it can be processed quickly. It has been tested with the Mininet simulator and gave considerably better results compared to the existing server load balancing algorithms in the floodlight controller. After experimentation and analysis, it is understood that the method provides comparatively better results than the existing load balancing algorithms