Controlling Network Latency in Mixed Hadoop Clusters: Do We Need Active Queue Management?

With the advent of big data, data center applications are processing vast amounts of unstructured and semi-structured data, in parallel on large clusters, across hundreds to thousands of nodes. The highest performance for these batch big data workloads is achieved using expensive network equipment with large buffers, which accommodate bursts in network traffic and allocate bandwidth fairly even when the network is congested. Throughput-sensitive big data applications are, however, often executed in the same data center as latency-sensitive workloads. For both workloads to be supported well, the network must provide both maximum throughput and low latency. Progress has been made in this direction, as modern network switches support Active Queue Management (AQM) and Explicit Congestion Notifications (ECN), both mechanisms to control the level of queue occupancy, reducing the total network latency. This paper is the first study of the effect of Active Queue Management on both throughput and latency, in the context of Hadoop and the MapReduce programming model. We give a quantitative comparison of four different approaches for controlling buffer occupancy and latency: RED and CoDel, both standalone and also combined with ECN and DCTCP network protocol, and identify the AQM configurations that maintain Hadoop execution time gains from larger buffers within 5%, while reducing network packet latency caused by bufferbloat by up to 85%. Finally, we provide recommendations to administrators of Hadoop clusters as to how to improve latency without degrading the throughput of batch big data workloads.The research leading to these results has received funding from the European Unions Seventh Framework Programme (FP7/2007–2013) under grant agreement number 610456 (Euroserver). The research was also supported by the Ministry of Economy and Competitiveness of Spain under the contracts TIN2012-34557 and TIN2015-65316-P, Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), HiPEAC-3 Network of Excellence (ICT- 287759), and the Severo Ochoa Program (SEV-2011-00067) of the Spanish Government.Peer ReviewedPostprint (author's final draft

X-TCP: A Cross Layer Approach for TCP Uplink Flows in mmWave Networks

Millimeter wave frequencies will likely be part of the fifth generation of mobile networks and of the 3GPP New Radio (NR) standard. MmWave communication indeed provides a very large bandwidth, thus an increased cell throughput, but how to exploit these resources at the higher layers is still an open research question. A very relevant issue is the high variability of the channel, caused by the blockage from obstacles and the human body. This affects the design of congestion control mechanisms at the transport layer, and state-of-the-art TCP schemes such as TCP CUBIC present suboptimal performance. In this paper, we present a cross layer approach for uplink flows that adjusts the congestion window of TCP at the mobile equipment side using an estimation of the available data rate at the mmWave physical layer, based on the actual resource allocation and on the Signal to Interference plus Noise Ratio. We show that this approach reduces the latency, avoiding to fill the buffers in the cellular stack, and has a quicker recovery time after RTO events than several other TCP congestion control algorithms.Comment: 6 pages, 5 figures, accepted for presentation at the 2017 16th Annual Mediterranean Ad Hoc Networking Workshop (MED-HOC-NET

Evaluation of the Impact of Packet Drops due to AQM over Capacity Limited Paths

Postprin

milliProxy: a TCP Proxy Architecture for 5G mmWave Cellular Systems

TCP is the most widely used transport protocol in the internet. However, it offers suboptimal performance when operating over high bandwidth mmWave links. The main issues introduced by communications at such high frequencies are (i) the sensitivity to blockage and (ii) the high bandwidth fluctuations due to Line of Sight (LOS) to Non Line of Sight (NLOS) transitions and vice versa. In particular, TCP has an abstract view of the end-to-end connection, which does not properly capture the dynamics of the wireless mmWave link. The consequence is a suboptimal utilization of the available resources. In this paper we propose a TCP proxy architecture that improves the performance of TCP flows without any modification at the remote sender side. The proxy is installed in the Radio Access Network, and exploits information available at the gNB in order to maximize throughput and minimize latency.Comment: 7 pages, 6 figures, 2 tables, presented at the 2017 51st Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 201

Will TCP work in mmWave 5G Cellular Networks?

The vast available spectrum in the millimeter wave (mmWave) bands offers the possibility of multi-Gbps data rates for fifth generation (5G) cellular networks. However, mmWave capacity can be highly intermittent due to the vulnerability of mmWave signals to blockages and delays in directional searching. Such highly variable links present unique challenges for adaptive control mechanisms in transport layer protocols and end-to-end applications. This paper considers the fundamental question of whether TCP - the most widely used transport protocol - will work in mmWave cellular systems. The paper provides a comprehensive simulation study of TCP considering various factors such as the congestion control algorithm, including the recently proposed TCP BBR, edge vs. remote servers, handover and multi- connectivity, TCP packet size and 3GPP-stack parameters. We show that the performance of TCP on mmWave links is highly dependent on different combinations of these parameters, and identify the open challenges in this area.Comment: 7 pages, 4 figures, 2 tables. To be published in the IEEE Communication Magazin

Revisiting Old Friends: Is CoDel Really Achieving What RED Cannot?

We use ns-2 simulations to compare RED's gentle mode to CoDel in terms of their ability to reduce the latency for various TCP variants. We use a common dumbbell topology with Pareto background traffic, and measure the packet delays and transmission time of a 10MB FTP transfer. In our scenarios, we find that CoDel reduces the latency by 87%, but RED still manages to reduce it by 75%. However, the use of CoDel results in a transmission time 42% longer than when using RED. In light of its maturity, we therefore argue that RED could be considered as a good candidate to tackle Bufferbloat

Advanced Congestion Control Mechanisms for Internet of Things

The number of IoT devices is growing at high speed, around 18 billion devices are forecast by 2022. Many of these devices are implemented with simple hardware, with low specifications and low resources. Taking into account the limited hardware resources and the huge network formed by IoT devices, CoAP was born as a lighter application protocol than HTTP. One important task for this scenario is the congestion control of huge networks using simple hardware devices. CoAP implements a simple congestion control solution, but many research articles show that this solution is not very efficient and it could be improved using other congestion control algorithms. CoCoA was born with the aim of being the standard congestion control algorithm for CoAP and has been proven through many studies, that it improves CoAP default performance in several scenarios. However, some research articles show that CoCoA offers low performance in bufferbloat scenarios. This thesis evaluates CoCoA in bufferbloat scenarios and introduces changes on CoCoA algorithm, achieving an improvement on its performance