Randomized load balancing in finite regimes

Randomized load balancing is a cost efficient policy for job scheduling in parallel server queueing systems whereby, with every incoming job, a central dispatcher randomly polls some servers and selects the one with the smallest queue. By exactly deriving the jobs' delay distribution in such systems, in explicit and closed form, Mitzenmacher~\cite{Mi03} proved the so-called `power-of-two' result, which states that by randomly polling only two servers yields an exponential improvement in delay over randomly selecting a single server. Such a fundamental result, however, was obtained in an asymptotic regime in the total number of servers, and does do not necessarily provide accurate estimates for practical finite regimes with small or moderate number of servers. In this paper we obtain stochastic lower and upper bounds on the jobs' average delay in non-asymptotic/finite regimes, by borrowing ideas for analyzing the particular case of Join-the-Shortest-Queue (JSQ) policy. Numerical illustrations indicate not only that the obtained (lower) bounds are remarkably accurate, but also that the existing exact but asymptotic results can be largely misleading in finite regimes (e.g., by more than $100\%$ in the case of $12$ servers)

Randomized longest-queue-first scheduling for large-scale buffered systems

We develop diffusion approximations for parallel-queueing systems with the randomized longest-queue-first scheduling algorithm by establishing new mean-field limit theorems as the number of buffers $n\to\infty$. We achieve this by allowing the number of sampled buffers $d=d(n)$ to depend on the number of buffers $n$, which yields an asymptotic `decoupling' of the queue length processes. We show through simulation experiments that the resulting approximation is accurate even for moderate values of $n$ and $d(n)$. To our knowledge, we are the first to derive diffusion approximations for a queueing system in the large-buffer mean-field regime. Another noteworthy feature of our scaling idea is that the randomized longest-queue-first algorithm emulates the longest-queue-first algorithm, yet is computationally more attractive. The analysis of the system performance as a function of $d(n)$ is facilitated by the multi-scale nature in our limit theorems: the various processes we study have different space scalings. This allows us to show the trade-off between performance and complexity of the randomized longest-queue-first scheduling algorithm