MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions

Effective network congestion control strategies are key to keeping the Internet (or any large computer network) operational. Network congestion control has been dominated by hand-crafted heuristics for decades. Recently, ReinforcementLearning (RL) has emerged as an alternative to automatically optimize such control strategies. Research so far has primarily considered RL interfaces which block the sender while an agent considers its next action. This is largely an artifact of building on top of frameworks designed for RL in games (e.g. OpenAI Gym). However, this does not translate to real-world networking environments, where a network sender waiting on a policy without sending data leads to under-utilization of bandwidth. We instead propose to formulate congestion control with an asynchronous RL agent that handles delayed actions. We present MVFST-RL, a scalable framework for congestion control in the QUIC transport protocol that leverages state-of-the-art in asynchronous RL training with off-policy correction. We analyze modeling improvements to mitigate the deviation from Markovian dynamics, and evaluate our method on emulated networks from the Pantheon benchmark platform. The source code is publicly available at https://github.com/facebookresearch/mvfst-rl

Beyond socket options: making the Linux TCP stack truly extensible

The Transmission Control Protocol (TCP) is one of the most important protocols in today's Internet. Its specification and implementations have been refined for almost forty years. The Linux TCP stack is one of the most widely used TCP stacks given its utilisation on servers and Android smartphones and tablets. However, TCP and its implementations evolve very slowly. In this paper, we demonstrate how to leverage the eBPF virtual machine that is part of the recent versions of the Linux kernel to make the TCP stack easier to extend. We demonstrate a variety of use cases where the eBPF code is injected inside a running kernel to update or tune the TCP implementation. We first implement the TCP User Timeout Option. Then we propose a new option that enables a client to request a server to use a specific congestion control scheme. Our third extension is a TCP option that sets the initial congestion window. We then demonstrate how eBPF code can be used to tune the acknowledgment strategy.Comment: 9 pages, 8 figure

Beyond socket options: making the Linux TCP stack truly extensible

The Transmission Control Protocol (TCP) is one of the most important protocols in today's Internet. Its specification and implementations have been refined for almost forty years. The Linux TCP stack is one of the most widely used TCP stacks given its utilisation on servers and Android smartphones and tablets. However, TCP and its implementations evolve very slowly. In this paper, we demonstrate how to leverage the eBPF virtual machine that is part of the recent versions of the Linux kernel to make the TCP stack easier to extend. We demonstrate a variety of use cases where the eBPF code is injected inside a running kernel to update or tune the TCP implementation. We first implement the TCP User Timeout Option. Then we propose a new option that enables a client to request a server to use a specific congestion control scheme. Our third extension is a TCP option that sets the initial congestion window. We then demonstrate how eBPF code can be used to tune the acknowledgment strategy.Comment: 9 pages, 8 figure

Mystique: a fine-grained and transparent congestion control enforcement scheme

TCP congestion control is a vital component for the latency of Web services. In practice, a single congestion control mechanism is often used to handle all TCP connections on a Web server, e.g., Cubic for Linux by default. Considering complex and ever-changing networking environment, the default congestion control may not always be the most suitable one. Adjusting congestion control to meet different networking scenarios usually requires modification of TCP stacks on a server. This is difficult, if not impossible, due to various operating system and application configurations on production servers. In this paper, we propose Mystique, a light-weight, flexible, and dynamic congestion control switching scheme that allows network or server administrators to deploy any congestion control schemes transparently without modifying existing TCP stacks on servers. We have implemented Mystique in Open vSwitch (OVS) and conducted extensive testbed experiments in both public and private cloud environments. Experiment results have demonstrated that Mystique is able to effectively adapt to varying network conditions, and can always employ the most suitable congestion control for each TCP connection. More specifically, Mystique can significantly reduce latency by 18.13% on average when compared with individual congestion controls

Restructuring endpoint congestion control

© 2018 Copyright held by the owner/author(s). Publication rights licensed to the Association for Computing Machinery. This paper describes the implementation and evaluation of a system to implement complex congestion control functions by placing them in a separate agent outside the datapath. Each datapath'such as the Linux kernel TCP, UDP-based QUIC, or kernel-bypass transports like mTCP-on-DPDK'summarizes information about packet round-trip times, receptions, losses, and ECN via a well-defined interface to algorithms running in the off-datapath Congestion Control Plane (CCP). The algorithms use this information to control the datapath's congestion window or pacing rate. Algorithms written in CCP can run on multiple datapaths. CCP improves both the pace of development and ease of maintenance of congestion control algorithms by providing better, modular abstractions, and supports aggregation capabilities of the Congestion Manager, all with one-time changes to datapaths. CCP also enables new capabilities, such as Copa in Linux TCP, several algorithms running on QUIC and mTCP/DPDK, and the use of signal processing algorithms to detect whether cross-traffic is ACK-clocked. Experiments with our user-level Linux CCP implementation show that CCP algorithms behave similarly to kernel algorithms, and incur modest CPU overhead of a few percent