Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

ATP: a Datacenter Approximate Transmission Protocol

Many datacenter applications such as machine learning and streaming systems do not need the complete set of data to perform their computation. Current approximate applications in datacenters run on a reliable network layer like TCP. To improve performance, they either let sender select a subset of data and transmit them to the receiver or transmit all the data and let receiver drop some of them. These approaches are network oblivious and unnecessarily transmit more data, affecting both application runtime and network bandwidth usage. On the other hand, running approximate application on a lossy network with UDP cannot guarantee the accuracy of application computation. We propose to run approximate applications on a lossy network and to allow packet loss in a controlled manner. Specifically, we designed a new network protocol called Approximate Transmission Protocol, or ATP, for datacenter approximate applications. ATP opportunistically exploits available network bandwidth as much as possible, while performing a loss-based rate control algorithm to avoid bandwidth waste and re-transmission. It also ensures bandwidth fair sharing across flows and improves accurate applications' performance by leaving more switch buffer space to accurate flows. We evaluated ATP with both simulation and real implementation using two macro-benchmarks and two real applications, Apache Kafka and Flink. Our evaluation results show that ATP reduces application runtime by 13.9% to 74.6% compared to a TCP-based solution that drops packets at sender, and it improves accuracy by up to 94.0% compared to UDP

데이터 센터 내의 다중경로 전송을 위한 동적 부하 균형 기법

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 권태경 .Various applications require the data center networks to carry their traffic efficiently. The data center networks usually have a hierarchical topology and exhibit distinct traffic patterns, which is different from the traditional Internet. These features have driven the data center networks to reduce the flow completion time (FCT) and to achieve high throughput. One of the possible solutions is balancing network loads across multiple paths by leveraging transport mechanisms like Equal-Cost MultiPath (ECMP) routing. ECMP allows flows to exploit multiple paths by hashing the metadata of the flows. However, due to the random nature of hash functions, ECMP often distributes the traffic unevenly, which makes it hard to utilize the links' full capacity. Thus, we propose an adaptive load balancing mechanism for multiple paths in data centers, called MaxPass, to complement ECMP. A sender adaptively selects and dynamically changes multiple paths depending on the current network status like congestion. To monitor the network status, the corresponding receiver transmits a probe packet periodically to the senderits loss indicates a traffic congestion. We implemented MaxPass using commodity switches and carry out the quantitative analysis on the ns-2 simulator to show that MaxPass can improve the FCT and the throughput.데이터 센터 내에서 동작하는 다양한 어플리케이션들은 네트워크 트래픽을 보다 효율적으로 사용할 것을 요구한다. 데이터 센터 네트워크는 기존의 인터넷과는 다른 다중 루트 계층적 토폴로지로 구성되어 있으며, 다양한 트래픽 패턴을 가지고 있다. 이러한 특징으로 인해 데이터 센터 네트워크는 짧은 플로우 처리 완료 시간 (Flow Completion Time)과 높은 처리량 (Throughput)을 요구한다. 데이터 센터가 요구하는 조건들을 만족시키기 위한 방법 중 하나로는 등가 다중 경로 (Equal-Cost Multi-Path)와 같은 라우팅 기법을 활용하여 네트워크 부하를 서로 다른 링크에 분산시키는 것이 있다. 등가 다중 경로 라우팅 기법은 플로우의 메타 데이터를 해싱하여 플로우가 여러 경로를 이용할 수 있도록 한다. 그러나 랜덤한 결과를 도출하는 해시 함수의 특성에 따라 등가 다중 경로 라우팅 기법은 종종 트래픽을 고르게 분배하지 못함으로써, 링크의 전체 용량을 활용하기에는 한계가 있다. 따라서 본 논문에서는 데이터 센터 네트워크 환경에 맞는 새로운 부하 균형 배분 기법인 맥스패스 (MaxPass)를 제안한다. 맥스패스 내에서 데이터 송신자는 현재 네트워크 상태에 따라 경로를 동적으로 선택하고 변경한다. 데이터 수신자는 현재 네트워크 상태를 파악하기 위해 탐색 패킷을 주기적으로 송신자에게 보내고, 탐색 패킷 드랍 여부에 따라 혼잡도를 파악한다. 본 논문은 실제 스위치에서 맥스패스를 구현하였으며, ns-2 시뮬레이션을 기반한 실험을 통해 제안한 기법에 관하여 정량적 수치 분석을 수행하고, 플로우 처리 완료 시간과 링크 처리량의 성능 향상이 있음을 보여 준다.Chapter 1 Introduction 1 Chapter 2 Background 5 2.1 Data Center Network Topology . . . . . . . . . . . . . . . . . 5 2.2 Multipath Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Multipath Transport Protocol . . . . . . . . . . . . . . . . . . . 7 2.4 Credit-based Congestion Control . . . . . . . . . . . . . . . . 8 Chapter 3 MaxPass 10 3.1 Design Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.2 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.3 Path Probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.4 Adaptive Path Selection . . . . . . . . . . . . . . . . . . . . . . .15 3.5 Feedback Control Algorithm . . . . . . . . . . . . . . . . . . . 15 3.6 Credit Stop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 Chapter 4 Evaluation 20 4.1 Ns-2 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.1.1 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1.2 Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 4.1.3 Flow Completion Time (FCT) . . . . . . . . . . . . . . . .25 4.2 Testbed Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 25 Chapter 5 Related Work 28 5.1 Centralized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28 5.2 Decentralized/Distributed . . . . . . . . . . . . . . . . . . . . . .30 Chapter 6 Conclusion 32 초록 38Maste

Fastpass: A Centralized “Zero-Queue” Datacenter Network

An ideal datacenter network should provide several properties, including low median and tail latency, high utilization (throughput), fair allocation of network resources between users or applications, deadline-aware scheduling, and congestion (loss) avoidance. Current datacenter networks inherit the principles that went into the design of the Internet, where packet transmission and path selection decisions are distributed among the endpoints and routers. Instead, we propose that each sender should delegate control—to a centralized arbiter—of when each packet should be transmitted and what path it should follow. This paper describes Fastpass, a datacenter network architecture built using this principle. Fastpass incorporates two fast algorithms: the first determines the time at which each packet should be transmitted, while the second determines the path to use for that packet. In addition, Fastpass uses an efficient protocol between the endpoints and the arbiter and an arbiter replication strategy for fault-tolerant failover. We deployed and evaluated Fastpass in a portion of Facebook’s datacenter network. Our results show that Fastpass achieves high throughput comparable to current networks at a 240 reduction is queue lengths (4.35 Mbytes reducing to 18 Kbytes), achieves much fairer and consistent flow throughputs than the baseline TCP (5200 reduction in the standard deviation of per-flow throughput with five concurrent connections), scalability from 1 to 8 cores in the arbiter implementation with the ability to schedule 2.21 Terabits/s of traffic in software on eight cores, and a 2.5 reduction in the number of TCP retransmissions in a latency-sensitive service at Facebook.National Science Foundation (U.S.) (grant IIS-1065219)Irwin Mark Jacobs and Joan Klein Jacobs Presidential FellowshipHertz Foundation (Fellowship

Homa: A Receiver-Driven Low-Latency Transport Protocol Using Network Priorities (Complete Version)

Homa is a new transport protocol for datacenter networks. It provides exceptionally low latency, especially for workloads with a high volume of very short messages, and it also supports large messages and high network utilization. Homa uses in-network priority queues to ensure low latency for short messages; priority allocation is managed dynamically by each receiver and integrated with a receiver-driven flow control mechanism. Homa also uses controlled overcommitment of receiver downlinks to ensure efficient bandwidth utilization at high load. Our implementation of Homa delivers 99th percentile round-trip times less than 15{\mu}s for short messages on a 10 Gbps network running at 80% load. These latencies are almost 100x lower than the best published measurements of an implementation. In simulations, Homa's latency is roughly equal to pFabric and significantly better than pHost, PIAS, and NDP for almost all message sizes and workloads. Homa can also sustain higher network loads than pFabric, pHost, or PIAS.Comment: This paper is an extended version of the paper on Homa that was published in ACM SIGCOMM 2018. Material had to be removed from Sections 5.1 and 5.2 to meet the SIGCOMM page restrictions; this version restores the missing material. This paper is 18 pages, plus two pages of reference