Bounds for performance characteristics : a systematic approach via cost structures

In this paper we present a systematic approach to the construction of bounds for the average costs in Markov chains with possibly infinitely many states. The technique used to prove the bounds is based on dynamic programming. Most performance characteristics of Markovian systems can be represented by the average costs for some appropriately chosen cost structure. Therefore, the approach can be used to generate bounds for relevant performance characteristics. The approach is demonstrated for the shortest queue model. It is shown how for this model several bounds for the mean waiting time can be constructed. We include numerical results to demonstrate the quality of these bound

Randomized load balancing in finite regimes

Randomized load balancing is a cost efficient policy for job scheduling in parallel server queueing systems whereby, with every incoming job, a central dispatcher randomly polls some servers and selects the one with the smallest queue. By exactly deriving the jobs' delay distribution in such systems, in explicit and closed form, Mitzenmacher~\cite{Mi03} proved the so-called `power-of-two' result, which states that by randomly polling only two servers yields an exponential improvement in delay over randomly selecting a single server. Such a fundamental result, however, was obtained in an asymptotic regime in the total number of servers, and does do not necessarily provide accurate estimates for practical finite regimes with small or moderate number of servers. In this paper we obtain stochastic lower and upper bounds on the jobs' average delay in non-asymptotic/finite regimes, by borrowing ideas for analyzing the particular case of Join-the-Shortest-Queue (JSQ) policy. Numerical illustrations indicate not only that the obtained (lower) bounds are remarkably accurate, but also that the existing exact but asymptotic results can be largely misleading in finite regimes (e.g., by more than $100\%$ in the case of $12$ servers)

Towards Optimality in Parallel Scheduling

To keep pace with Moore's law, chip designers have focused on increasing the number of cores per chip rather than single core performance. In turn, modern jobs are often designed to run on any number of cores. However, to effectively leverage these multi-core chips, one must address the question of how many cores to assign to each job. Given that jobs receive sublinear speedups from additional cores, there is an obvious tradeoff: allocating more cores to an individual job reduces the job's runtime, but in turn decreases the efficiency of the overall system. We ask how the system should schedule jobs across cores so as to minimize the mean response time over a stream of incoming jobs. To answer this question, we develop an analytical model of jobs running on a multi-core machine. We prove that EQUI, a policy which continuously divides cores evenly across jobs, is optimal when all jobs follow a single speedup curve and have exponentially distributed sizes. EQUI requires jobs to change their level of parallelization while they run. Since this is not possible for all workloads, we consider a class of "fixed-width" policies, which choose a single level of parallelization, k, to use for all jobs. We prove that, surprisingly, it is possible to achieve EQUI's performance without requiring jobs to change their levels of parallelization by using the optimal fixed level of parallelization, k*. We also show how to analytically derive the optimal k* as a function of the system load, the speedup curve, and the job size distribution. In the case where jobs may follow different speedup curves, finding a good scheduling policy is even more challenging. We find that policies like EQUI which performed well in the case of a single speedup function now perform poorly. We propose a very simple policy, GREEDY*, which performs near-optimally when compared to the numerically-derived optimal policy

Analysis of join-the-shortest-queue routing for web server farms

Join the Shortest Queue (JSQ) is a popular routing policy for server farms. However, until now all analysis of JSQ has been limited to First-Come-First-Serve (FCFS) server farms, whereas it is known that web server farms are better modeled as Processor Sharing (PS) server farms. We provide the first approximate analysis of JSQ in the PS server farm model for general job-size distributions, obtaining the distribution of queue length at each queue. To do this, we approximate the queue length of each queue in the server farm by a one-dimensional Markov chain, in a novel fashion. We also discover some interesting insensitivity properties of PS server farms with JSQ routing, and discuss the near-optimality of JSQ

Load Balancing in the Non-Degenerate Slowdown Regime

We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and timescale separation that provides insights into the performance of JSQ. We calculate the price of irrevocably dispatching jobs to servers and prove this to within 15% (in the NDS regime) of the rules that may manoeuvre jobs between servers. We also compare ours results for the JSQ policy with the NDS approximations of many modern load balancing policies such as Idle-Queue-First and Power-of-$d$-choices policies which act as low information proxies for the JSQ policy. Our analysis leads us to construct new rules that have identical performance to JSQ but require less communication overhead than power-of-2-choices.Comment: Revised journal submission versio

Upper and lower bounds for the waiting time in the symmetric shortest queue system

In this paper we compare the exponential symmetric shortest queue system with two related systems: the shortest queue system with threshold jockeying and the shortest queue system with threshold blocking. The latter two systems are easier to analyse and are shown to give tight lower and upper bounds respectively for the mean waiting time in the shortest queue system. The approach also gives bounds for the distribution of the total number of jobs in the system

Upper and lower bounds for the waiting time in the symmetric shortest queue system

In this paper we compare the exponential symmetric shortest queue system with two related systems: the shortest queue system with Threshold Jockeying and the shortest queue system with Threshold Blocking. The latter two systems are easier to analyse and are shown to give tight lower and upper bounds respectively for the mean waiting time in the shortest queue system. The approach also gives bounds for the distribution of the total number of jobs in the system

Upper and lower bounds for the waiting time in the symmetric shortest queue system

In this paper we compare the exponential symmetric shortest queue system with two related systems: the shortest queue system with threshold jockeying and the shortest queue system with threshold blocking. The latter two systems are easier to analyse and are shown to give tight lower and upper bounds respectively for the mean waiting time in the shortest queue system. The approach also gives bounds for the distribution of the total number of jobs in the system