ASIdE: Using Autocorrelation-Based Size Estimation for Scheduling Bursty Workloads.

Temporal dependence in workloads creates peak congestion that can make service unavailable and reduce system performance. To improve system performability under conditions of temporal dependence, a server should quickly process bursts of requests that may need large service demands. In this paper, we propose and evaluateASIdE, an Autocorrelation-based SIze Estimation, that selectively delays requests which contribute to the workload temporal dependence. ASIdE implicitly approximates the shortest job first (SJF) scheduling policy but without any prior knowledge of job service times. Extensive experiments show that (1) ASIdE achieves good service time estimates from the temporal dependence structure of the workload to implicitly approximate the behavior of SJF; and (2) ASIdE successfully counteracts peak congestion in the workload and improves system performability under a wide variety of settings. Specifically, we show that system capacity under ASIdE is largely increased compared to the first-come first-served (FCFS) scheduling policy and is highly-competitive with SJF. © 2012 IEEE

Max-weight scheduling in networks with heavy-tailed traffic

We consider the problem of packet scheduling in a single-hop network with a mix of heavy-tailed and light-tailed traffic, and analyze the impact of heavy-tailed traffic on the performance of Max-Weight scheduling. As a performance metric we use the delay stability of traffic flows: a traffic flow is delay stable if its expected steady-state delay is finite, and delay unstable otherwise. First, we show that a heavy-tailed traffic flow is delay unstable under any scheduling policy. Then, we focus on the celebrated Max-Weight scheduling policy, and show that a light-tailed flow that conflicts with a heavy-tailed flow is also delay unstable. This is true irrespective of the rate or the tail distribution of the light-tailed flow, or other scheduling constraints in the network. Surprisingly, we show that a light-tailed flow can be delay unstable, even when it does not conflict with heavy-tailed traffic. Furthermore, delay stability in this case may depend on the rate of the light-tailed flow. Finally, we turn our attention to the class of Max-Weight-α scheduling policies; we show that if the α-parameters are chosen suitably, then the sum of the α-moments of the steady-state queue lengths is finite. We provide an explicit upper bound for the latter quantity, from which we derive results related to the delay stability of traffic flows, and the scaling of moments of steady-state queue lengths with traffic intensity

Max-Weight Scheduling in Queueing Networks With Heavy-Tailed Traffic

We consider the problem of scheduling in a single-hop switched network with a mix of heavy-tailed and light-tailed traffic and analyze the impact of heavy-tailed traffic on the performance of Max-Weight scheduling. As a performance metric, we use the delay stability of traffic flows: A traffic flow is delay-stable if its expected steady-state delay is finite, and delay-unstable otherwise. First, we show that a heavy-tailed traffic flow is delay-unstable under any scheduling policy. Then, we focus on the celebrated Max-Weight scheduling policy and show that a light-tailed flow that conflicts with a heavy-tailed flow is also delay-unstable. This is true irrespective of the rate or the tail distribution of the light-tailed flow or other scheduling constraints in the network. Surprisingly, we show that a light-tailed flow can become delay-unstable, even when it does not conflict with heavy-tailed traffic. Delay stability in this case may depend on the rate of the light-tailed flow. Finally, we turn our attention to the class of Max-Weight-α scheduling policies. We show that if the α-parameters are chosen suitably, then the sum of the α-moments of the steady-state queue lengths is finite. We provide an explicit upper bound for the latter quantity, from which we derive results related to the delay stability of traffic flows, and the scaling of moments of steady-state queue lengths with traffic intensity.National Science Foundation (U.S.) (Grant CNS-0915988)National Science Foundation (U.S.) (Grant CCF-0728554)United States. Air Force. Office of Scientific Research. Multidisciplinary University Research Initiative (Grant W911NF-08- 1-0238

Low latency via redundancy

Low latency is critical for interactive networked applications. But while we know how to scale systems to increase capacity, reducing latency --- especially the tail of the latency distribution --- can be much more difficult. In this paper, we argue that the use of redundancy is an effective way to convert extra capacity into reduced latency. By initiating redundant operations across diverse resources and using the first result which completes, redundancy improves a system's latency even under exceptional conditions. We study the tradeoff with added system utilization, characterizing the situations in which replicating all tasks reduces mean latency. We then demonstrate empirically that replicating all operations can result in significant mean and tail latency reduction in real-world systems including DNS queries, database servers, and packet forwarding within networks

Robust Queueing Theory

We propose an alternative approach for studying queues based on robust optimization. We model the uncertainty in the arrivals and services via polyhedral uncertainty sets, which are inspired from the limit laws of probability. Using the generalized central limit theorem, this framework allows us to model heavy-tailed behavior characterized by bursts of rapidly occurring arrivals and long service times. We take a worst-case approach and obtain closed-form upper bounds on the system time in a multi-server queue. These expressions provide qualitative insights that mirror the conclusions obtained in the probabilistic setting for light-tailed arrivals and services and generalize them to the case of heavy-tailed behavior. We also develop a calculus for analyzing a network of queues based on the following key principles: (a) the departure from a queue, (b) the superposition, and (c) the thinning of arrival processes have the same uncertainty set representation as the original arrival processes. The proposed approach (a) yields results with error percentages in single digits relative to simulation, and (b) is to a large extent insensitive to the number of servers per queue, network size, degree of feedback, and traffic intensity; it is somewhat sensitive to the degree of diversity of external arrival distributions in the network