Load Balancing in Large-Scale Systems with Multiple Dispatchers

Load balancing algorithms play a crucial role in delivering robust application performance in data centers and cloud networks. Recently, strong interest has emerged in Join-the-Idle-Queue (JIQ) algorithms, which rely on tokens issued by idle servers in dispatching tasks and outperform power-of-$d$ policies. Specifically, JIQ strategies involve minimal information exchange, and yet achieve zero blocking and wait in the many-server limit. The latter property prevails in a multiple-dispatcher scenario when the loads are strictly equal among dispatchers. For various reasons it is not uncommon however for skewed load patterns to occur. We leverage product-form representations and fluid limits to establish that the blocking and wait then no longer vanish, even for arbitrarily low overall load. Remarkably, it is the least-loaded dispatcher that throttles tokens and leaves idle servers stranded, thus acting as bottleneck. Motivated by the above issues, we introduce two enhancements of the ordinary JIQ scheme where tokens are either distributed non-uniformly or occasionally exchanged among the various dispatchers. We prove that these extensions can achieve zero blocking and wait in the many-server limit, for any subcritical overall load and arbitrarily skewed load profiles. Extensive simulation experiments demonstrate that the asymptotic results are highly accurate, even for moderately sized systems

A Fixed-Point Algorithm for Closed Queueing Networks

In this paper we propose a new efficient iterative scheme for solving closed queueing networks with phase-type service time distributions. The method is especially efficient and accurate in case of large numbers of nodes and large customer populations. We present the method, put it in perspective, and validate it through a large number of test scenarios. In most cases, the method provides accuracies within 5% relative error (in comparison to discrete-event simulation)

Monotonicity and error bounds for networks of Erlang loss queues

Networks of Erlang loss queues naturally arise when modelling finite communication systems without delays, among which, most notably are (i) classical circuit switch telephone networks (loss networks) and (ii) present-day wireless mobile networks. Performance measures of interest such as loss probabilities or throughputs can be obtained from the steady state distribution. However, while this steady state distribution has a closed product form expression in the first case (loss networks), it does not have one in the second case due to blocked (and lost) handovers. Product form approximations are therefore suggested. These approximations are obtained by a combined modification of both the state space (by a hypercubic expansion) and the transition rates (by extra redial rates). It will be shown that these product form approximations lead to (1) upper bounds for loss probabilities and \ud (2) analytic error bounds for the accuracy of the approximation for various performance measures.\ud The proofs of these results rely upon both monotonicity results and an analytic error bound method as based on Markov reward theory. This combination and its technicalities are of interest by themselves. The technical conditions are worked out and verified for two specific applications:\ud (1)• pure loss networks as under (2)• GSM networks with fixed channel allocation as under.\ud The results are of practical interest for computational simplifications and, particularly, to guarantee that blocking probabilities do not exceed a given threshold such as for network dimensioning

A tight bound on the throughput of queueing networks with blocking

In this paper, we present a bounding methodology that allows to compute a tight lower bound on the cycle time of fork--join queueing networks with blocking and with general service time distributions. The methodology relies on two ideas. First, probability masses fitting (PMF) discretizes the service time distributions so that the evolution of the modified network can be modelled by a Markov chain. The PMF discretization is simple: the probability masses on regular intervals are computed and aggregated on a single value in the orresponding interval. Second, we take advantage of the concept of critical path, i.e. the sequence of jobs that covers a sample run. We show that the critical path can be computed with the discretized distributions and that the same sequence of jobs offers a lower bound on the original cycle time. The tightness of the bound is shown on computational experiments. Finally, we discuss the extension to split--and--merge networks and approximate estimations of the cycle time.queueing networks, blocking, throughput, bound, probability masses fitting, critical path.

Capacity planning of prisons in the Netherlands

In this paper we describe a decision support system developed to help in assessing the need for various type of prison cells. In particular we predict the probability that a criminal has to be sent home because of a shortage of cells. The problem is modelled through a queueing network with blocking after service. We focus in particular on the new analytical method to solve this network

Monotonicity and error bounds for networks of Erlang loss queues

Networks of Erlang loss queues naturally arise when modelling finite communication systems without delays, among which, most notably\ud (i) classical circuit switch telephone networks (loss networks) and\ud (ii) present-day wireless mobile networks.\ud \ud Performance measures of interest such as loss probabilities or throughputs can be obtained from the steady state distribution. However, while this steady state distribution has a closed product form expression in the first case (loss networks), it has not in the second case due to blocked (and lost) handovers. Product form approximations are therefore suggested. These approximations are obtained by a combined modification of both the state space (by a hyper cubic expansion) and the transition rates (by extra redial rates). It will be shown that these product form approximations lead to\ud \ud - secure upper bounds for loss probabilities and\ud - analytic error bounds for the accuracy of the approximation for various performance measures.\ud \ud The proofs of these results rely upon both monotonicity results and an analytic error bound method as based on Markov reward theory. This combination and its technicalities are of interest by themselves. The technical conditions are worked out and verified for two specific applications:\ud \ud - pure loss networks as under (i)\ud - GSM-networks with fixed channel allocation as under (ii).\ud \ud The results are of practical interest for computational simplifications and, particularly, to guarantee blocking probabilities not to exceed a given threshold such as for network dimensioning.\u

Approximate Analysis of an Unreliable M/M/2 Retrial Queue

This thesis considers the performance evaluation of an M/M/2 retrial queue for which both servers are subject to active and idle breakdowns. Customers may abandon service requests if they are blocked from service upon arrival, or if their service is interrupted by a server failure. Customers choosing to remain in the system enter a retrial orbit for a random amount of time before attempting to re-access an available server. We assume that each server has its own dedicated repair person, and repairs begin immediately following a failure. Interfailure times, repair times and times between retrials are exponentially distributed, and all processes are assumed to be mutually independent. Modeling the number of customers in the orbit and status of the servers as a continuous-time Markov chain, we employ a phase-merging algorithm to approximately analyze the limiting behavior. Subsequently, we derive approximate expressions for several congestion and delay measures. Using a benchmark simulation model, we assess the accuracy of the approximations and show that, when the algorithm assumptions are met, the approximation procedure yields favorable results. However, as the rate of abandonment for blocked arrivals decreases, the performance declines while the results are insensitive to the rate of abandonment of customers preempted by a server failure