Low latency via redundancy

Low latency is critical for interactive networked applications. But while we know how to scale systems to increase capacity, reducing latency --- especially the tail of the latency distribution --- can be much more difficult. In this paper, we argue that the use of redundancy is an effective way to convert extra capacity into reduced latency. By initiating redundant operations across diverse resources and using the first result which completes, redundancy improves a system's latency even under exceptional conditions. We study the tradeoff with added system utilization, characterizing the situations in which replicating all tasks reduces mean latency. We then demonstrate empirically that replicating all operations can result in significant mean and tail latency reduction in real-world systems including DNS queries, database servers, and packet forwarding within networks

Analysis of Buffer Starvation with Application to Objective QoE Optimization of Streaming Services

Our purpose in this paper is to characterize buffer starvations for streaming services. The buffer is modeled as an M/M/1 queue, plus the consideration of bursty arrivals. When the buffer is empty, the service restarts after a certain amount of packets are \emph{prefetched}. With this goal, we propose two approaches to obtain the \emph{exact distribution} of the number of buffer starvations, one of which is based on \emph{Ballot theorem}, and the other uses recursive equations. The Ballot theorem approach gives an explicit result. We extend this approach to the scenario with a constant playback rate using T\`{a}kacs Ballot theorem. The recursive approach, though not offering an explicit result, can obtain the distribution of starvations with non-independent and identically distributed (i.i.d.) arrival process in which an ON/OFF bursty arrival process is considered in this work. We further compute the starvation probability as a function of the amount of prefetched packets for a large number of files via a fluid analysis. Among many potential applications of starvation analysis, we show how to apply it to optimize the objective quality of experience (QoE) of media streaming, by exploiting the tradeoff between startup/rebuffering delay and starvations.Comment: 9 pages, 7 figures; IEEE Infocom 201