Join the Shortest Queue with Many Servers. The Heavy-Traffic Asymptotics

We consider queueing systems with n parallel queues under a Join the Shortest Queue (JSQ) policy in the Halfin-Whitt heavy-traffic regime. We use the martingale method to prove that a scaled process counting the number of idle servers and queues of length exactly two weakly converges to a two-dimensional reflected Ornstein-Uhlenbeck process, while processes counting longer queues converge to a deterministic system decaying to zero in constant time. This limiting system is comparable to that of the traditional Halfin-Whitt model, but there are key differences in the queueing behavior of the JSQ model. In particular, only a vanishing fraction of customers will have to wait, but those who do incur a constant order waiting time. Keywords: queueing theory; parallel queues; diffusion model

Load Balancing in the Non-Degenerate Slowdown Regime

We analyse Join-the-Shortest-Queue in a contemporary scaling regime known as the Non-Degenerate Slowdown regime. Join-the-Shortest-Queue (JSQ) is a classical load balancing policy for queueing systems with multiple parallel servers. Parallel server queueing systems are regularly analysed and dimensioned by diffusion approximations achieved in the Halfin-Whitt scaling regime. However, when jobs must be dispatched to a server upon arrival, we advocate the Non-Degenerate Slowdown regime (NDS) to compare different load-balancing rules. In this paper we identify novel diffusion approximation and timescale separation that provides insights into the performance of JSQ. We calculate the price of irrevocably dispatching jobs to servers and prove this to within 15% (in the NDS regime) of the rules that may manoeuvre jobs between servers. We also compare ours results for the JSQ policy with the NDS approximations of many modern load balancing policies such as Idle-Queue-First and Power-of-$d$-choices policies which act as low information proxies for the JSQ policy. Our analysis leads us to construct new rules that have identical performance to JSQ but require less communication overhead than power-of-2-choices.Comment: Revised journal submission versio

Universality of Load Balancing Schemes on Diffusion Scale

We consider a system of $N$ parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among $d$ randomly selected servers otherwise $(1 \leq d \leq N)$. This load balancing scheme subsumes the so-called Join-the-Idle Queue (JIQ) policy $(d = 1)$ and the celebrated Join-the-Shortest Queue (JSQ) policy $(d = N)$ as two crucial special cases. We develop a stochastic coupling construction to obtain the diffusion limit of the queue process in the Halfin-Whitt heavy-traffic regime, and establish that it does not depend on the value of $d$, implying that assigning tasks to idle servers is sufficient for diffusion level optimality

Exponential Tail Bounds on Queues: A Confluence of Non-Asymptotic Heavy Traffic and Large Deviations

In general, obtaining the exact steady-state distribution of queue lengths is not feasible. Therefore, we establish bounds for the tail probabilities of queue lengths. Specifically, we examine queueing systems under Heavy-Traffic (HT) conditions and provide exponentially decaying bounds for the probability $\mathbb P(\epsilon q > x)$, where $\epsilon$ is the HT parameter denoting how far the load is from the maximum allowed load. Our bounds are not limited to asymptotic cases and are applicable even for finite values of $\epsilon$, and they get sharper as $\epsilon \to 0$. Consequently, we derive non-asymptotic convergence rates for the tail probabilities. Unlike other approaches such as moment bounds based on drift arguments and bounds on Wasserstein distance using Stein's method, our method yields sharper tail bounds. Furthermore, our results offer bounds on the exponential rate of decay of the tail, given by $-\frac{1}{x} \log \mathbb P(\epsilon q > x)$ for any finite value of $x$. These can be interpreted as non-asymptotic versions of Large Deviation (LD) results. We demonstrate our approach by presenting tail bounds for: (i) a continuous time Join-the-shortest queue (JSQ) load balancing system, (ii) a discrete time single-server queue and (iii) an $M/M/n$ queue. We not only bridge the gap between classical-HT and LD regimes but also explore the large system HT regimes for JSQ and $M/M/n$ systems. In these regimes, both the system size and the system load increase simultaneously. Our results also close a gap in the existing literature on the limiting distribution of JSQ in the super-NDS (a.k.a. super slowdown) regime. This contribution is of an independent interest. Here, a key ingredient is a more refined characterization of state space collapse for JSQ system, achieved by using an exponential Lyapunov function designed to approximate the $\ell_{\infty}$ norm.Comment: 37 pages, 1 figur