Scheduling for today’s computer systems: bridging theory and practice

Scheduling is a fundamental technique for improving performance in computer systems. From web servers to routers to operating systems, how the bottleneck device is scheduled has an enormous impact on the performance of the system as a whole. Given the immense literature studying scheduling, it is easy to think that we already understand enough about scheduling. But, modern computer system designs have highlighted a number of disconnects between traditional analytic results and the needs of system designers. In particular, the idealized policies, metrics, and models used by analytic researchers do not match the policies, metrics, and scenarios that appear in real systems. The goal of this thesis is to take a step towards modernizing the theory of scheduling in order to provide results that apply to today’s computer systems, and thus ease the burden on system designers. To accomplish this goal, we provide new results that help to bridge each of the disconnects mentioned above. We will move beyond the study of idealized policies by introducing a new analytic framework where the focus is on scheduling heuristics and techniques rather than individual policies. By moving beyond the study of individual policies, our results apply to the complex hybrid policies that are often used in practice. For example, our results enable designers to understand how the policies that favor small job sizes are affected by the fact that real systems only have estimates of job sizes. In addition, we move beyond the study of mean response time and provide results characterizing the distribution of response time and the fairness of scheduling policies. These results allow us to understand how scheduling affects QoS guarantees and whether favoring small job sizes results in large job sizes being treated unfairly. Finally, we move beyond the simplified models traditionally used in scheduling research and provide results characterizing the effectiveness of scheduling in multiserver systems and when users are interactive. These results allow us to answer questions about the how to design multiserver systems and how to choose a workload generator when evaluating new scheduling designs

PSBS: Practical Size-Based Scheduling

Size-based schedulers have very desirable performance properties: optimal or near-optimal response time can be coupled with strong fairness guarantees. Despite this, such systems are very rarely implemented in practical settings, because they require knowing a priori the amount of work needed to complete jobs: this assumption is very difficult to satisfy in concrete systems. It is definitely more likely to inform the system with an estimate of the job sizes, but existing studies point to somewhat pessimistic results if existing scheduler policies are used based on imprecise job size estimations. We take the goal of designing scheduling policies that are explicitly designed to deal with inexact job sizes: first, we show that existing size-based schedulers can have bad performance with inexact job size information when job sizes are heavily skewed; we show that this issue, and the pessimistic results shown in the literature, are due to problematic behavior when large jobs are underestimated. Once the problem is identified, it is possible to amend existing size-based schedulers to solve the issue. We generalize FSP -- a fair and efficient size-based scheduling policy -- in order to solve the problem highlighted above; in addition, our solution deals with different job weights (that can be assigned to a job independently from its size). We provide an efficient implementation of the resulting protocol, which we call Practical Size-Based Scheduler (PSBS). Through simulations evaluated on synthetic and real workloads, we show that PSBS has near-optimal performance in a large variety of cases with inaccurate size information, that it performs fairly and it handles correctly job weights. We believe that this work shows that PSBS is indeed pratical, and we maintain that it could inspire the design of schedulers in a wide array of real-world use cases.Comment: arXiv admin note: substantial text overlap with arXiv:1403.599

On the impact of heterogeneity and back-end scheduling in load balancing designs

Load balancing is a common approach for task assignment in distributed architectures. In this paper, we show that the degree of inefficiency in load balancing designs is highly dependent on the scheduling discipline used at each of the backend servers. Traditionally, the back-end scheduler can be modeled as Processor Sharing (PS), in which case the degree of inefficiency grows linearly with the number of servers. However, if the back-end scheduler is changed to Shortest Remaining Processing Time (SRPT), the degree of inefficiency can be independent of the number of servers, instead depending only on the heterogeneity of the speeds of the servers. Further, switching the back-end scheduler to SRPT can provide significant improvements in the overall mean response time of the system as long as the heterogeneity of the server speeds is small

Incentives for Quality through Endogenous Routing

We study how rework routing together with wage and piece rate compensation can strengthen incentives for quality. Traditionally, rework is assigned back to the agent who generates the defect (in a self routing scheme) or to another agent dedicated to rework (in a dedicated routing scheme). In contrast, a novel cross routing scheme allocates rework to a parallel agent performing both new jobs and rework. The agent who passes quality inspection or completes rework receives the piece rate paid per job. We compare the incentives of these rework allocation schemes in a principal-agent model with embedded quality control and routing in a multi-class queueing network. We show that conventional self routing of rework can never induce first-best effort. Dedicated routing and cross routing, however, strengthen incentives for quality by imposing an implicit punishment for quality failure. In addition, cross routing leads to workload allocation externalities and a prisoner’s dilemma, thereby creating highest incentives for quality. Firm profitability depends on capacity levels, revenues, and quality costs. With ample capacity, dedicated routing and cross routing both achieve first-best profit rate, while self routing does not. With limited capacity, cross routing generates the highest profit rate when appraisal, internal failure, or external failure costs are high, while self routing performs best when gross margins are high. When the number of agents increases, the incentive power of cross routing reduces monotonically and approaches that of dedicated routing.queueing networks; routing; Nash equilibrium; quality control; piece rate; epsilon equilibrium.

Heavy-traffic analysis of a multiple-phase network with discriminatory processor sharing

We analyze a generalization of the Discriminatory Processor Sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue length vector in heavy traffic. The result shows that in the limit, the queue length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta (1996) who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically i.i.d. according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for d_k/E(B_k^fwd) obtain a larger share of the capacity, where d_k is the cost associated to class k, and E(B_k^fwd) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments

Heavy-traffic analysis of a multiple-phase network with discriminatory processor sharing

We analyze a generalization of the Discriminatory Processor Sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue length vector in heavy traffic. The result shows that in the limit, the queue length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta (1996) who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically i.i.d. according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for d_k/E(B_k^fwd) obtain a larger share of the capacity, where d_k is the cost associated to class k, and E(B_k^fwd) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments

Heavy-traffic analysis of a multiple-phase network with discriminatory processor sharing

We analyze a generalization of the discriminatory processor-sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue-length vector in heavy traffic. The result shows that in the limit, the queue-length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta [Rege, K. M., B. Sengupta. 1996. Queue length distribution for the discriminatory processor-sharing queue. Oper. Res. 44(4) 653-657], who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue-length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability-generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue, we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically independent and distributed according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for dk/E(B fwd k) obtain a larger share of the capacity, where dk is the cost associated to class k, and E(B fwd k) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments

A note on comparing response times in the M/GI/1/FB and M/GI/1/PS queues

We compare the overall mean response time (a.k.a. sojourn time) of the processor sharing (PS) and feedback (FB) queues under an M/GI/1 system. We show that FB outperforms PS under service distributions having decreasing failure rates; whereas PS outperforms FB under service distributions having increasing failure rates

A note on comparing response times in the M/GI/1/FB and M/GI/1/PS queues

We compare the overall mean response time (a.k.a. sojourn time) of the processor sharing (PS) and feedback (FB) queues under an M/GI/1 system. We show that FB outperforms PS under service distributions having decreasing failure rates; whereas PS outperforms FB under service distributions having increasing failure rates