The upper bound of packet transmission capacity in local static routings

We propose a universal analysis for static routings on networks and describe the congestion characteristics by the theory. The relation between average transmission time and transmission capacity is described by inequality T0Rc0<=1. For large scale sparse networks, the non-trivial upper bond of transmission capacity Rc0 is limited by Rc0 in some approximate conditions. the theoretical results agree with simulations on BA Networks.Comment: 7pages,4figure

Stationary analysis of the Shortest Queue First service policy

We analyze the so-called Shortest Queue First (SQF) queueing discipline whereby a unique server addresses queues in parallel by serving at any time that queue with the smallest workload. Considering a stationary system composed of two parallel queues and assuming Poisson arrivals and general service time distributions, we first establish the functional equations satisfied by the Laplace transforms of the workloads in each queue. We further specialize these equations to the so-called "symmetric case", with same arrival rates and identical exponential service time distributions at each queue; we then obtain a functional equation $$M(z) = q(z) \cdot M \circ h(z) + L(z)$$ for unknown function $M$, where given functions $q$, $L$ and $h$ are related to one branch of a cubic polynomial equation. We study the analyticity domain of function $M$ and express it by a series expansion involving all iterates of function $h$. This allows us to determine empty queue probabilities along with the tail of the workload distribution in each queue. This tail appears to be identical to that of the Head-of-Line preemptive priority system, which is the key feature desired for the SQF discipline

The stability of join-the-shortest-queue models with general input and output processes

The paper establishes necessary and sufficient conditions for stability of different join-the-shortest-queue models including load-balanced networks with general input and output processes. It is shown, that the necessary and sufficient condition for stability of load-balanced networks is associated with the solution of a linear programming problem precisely formulated in the paper. It is proved, that if the minimum of the objective function of that linear programming problem is less than 1, then the associated load-balanced network is stable.Comment: 22 pages, typos cor

Load balancing with heterogeneous schedulers

Load balancing is a common approach in web server farms or inventory routing problems. An important issue in such systems is to determine the server to which an incoming request should be routed to optimize a given performance criteria. In this paper, we assume the server's scheduling disciplines to be heterogeneous. More precisely, a server implements a scheduling discipline which belongs to the class of limited processor sharing (LPS-$d$) scheduling disciplines. Under LPS-$d$, up to $d$ jobs can be served simultaneously, and hence, includes as special cases First Come First Served ($d=1$) and Processor Sharing ($d=\infty$). In order to obtain efficient heuristics, we model the above load-balancing framework as a multi-armed restless bandit problem. Using the relaxation technique, as first developed in the seminal work of Whittle, we derive Whittle's index policy for general cost functions and obtain a closed-form expression for Whittle's index in terms of the steady-state distribution. Through numerical computations, we investigate the performance of Whittle's index with two different performance criteria: linear cost criterion and a cost criterion that depends on the first and second moment of the throughput. Our results show that \emph{(i)} the structure of Whittle's index policy can strongly depend on the scheduling discipline implemented in the server, i.e., on $d$, and that \emph{(ii)} Whittle's index policy significantly outperforms standard dispatching rules such as Join the Shortest Queue (JSQ), Join the Shortest Expected Workload (JSEW), and Random Server allocation (RSA)

Stability of Parallel Server Systems

The fundamental problem in the study of parallel-server systems is that of finding and analyzing `good' routing policies of arriving jobs to the servers. It is well known that, if full information regarding the workload process is available to a central dispatcher, then the {\em join the shortest workload} (JSW) policy, which assigns jobs to the server with the least workload, is the optimal assignment policy, in that it maximizes server utilization, and thus minimizes sojourn times. The {\em join the shortest queue} (JSQ) policy is an efficient dispatching policy when information is available only on the number of jobs with each of the servers, but not on their service requirements. If information on the state of the system is not available, other dispatching policies need to be employed, such as the power-of-$d$ routing policy, in which each arriving job joins the shortest among $d \ge 1$ queues sampled uniformly at random. (Under this latter policy, the system is known as {\em the supermarket model}.) In this paper we study the stability question of parallel server systems assuming that routing errors occur, so that arrivals may be routed to the `wrong' (not to the smallest) queue with a positive probability. We show that, even if a `non-idling' dispatching policy is employed, under which new arrivals are always routed to an idle server, if any is available, the performance of the system can be much worse than under the policy that chooses one of the servers uniformly at random. More specifically, we prove that the usual traffic intensity $\rho < 1$ does not guarantee that the system is stable

Blocking Avoidance in Transportation Systems

The blocking problem naturally arises in transportation systems as multiple vehicles with different itineraries share available resources. In this paper, we investigate the impact of the blocking problem to the waiting time at the intersections of transportation systems. We assume that different vehicles, depending on their Internet connection capabilities, may communicate their intentions (e.g., whether they will turn left or right or continue straight) to intersections (specifically to devices attached to traffic lights). We consider that information collected by these devices are transmitted to and processed in a cloud-based traffic control system. Thus, a cloud-based system, based on the intention information, can calculate average waiting times at intersections. We consider this problem as a queuing model, and we characterize average waiting times by taking into account (i) blocking probability, and (ii) vehicles' ability to communicate their intentions. Then, by using average waiting times at intersection, we develop a shortest delay algorithm that calculates the routes with shortest delays between two points in a transportation network. Our simulation results confirm our analysis, and demonstrate that our shortest delay algorithm significantly improves over baselines that are unaware of the blocking problem

Mean Field Approximations to a Queueing System with Threshold-Based Workload Control Scheme

In this paper, motivated by considerations of server utilization and energy consumptions in cloud computing, we investigate a homogeneous queueing system with a threshold-based workload control scheme. In this system, a virtual machine will be turned off when there are no tasks in its buffer upon the completion of a service by the machine, and turned on when the number of tasks in its buffer reaches a pre-set threshold value. Due to complexity of this system, we propose approximations to system performance measures by mean field limits. An iterative algorithm is suggested for the solution to the mean field limit equations. In addition, numerical and simulation results are presented to justify the proposed approximation method and to provide a numerical analysis on the impact of the system performances by system parameters.Comment: Revised version, 26 pages, 5 figure

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

We present an overview of scalable load balancing algorithms which provide favorable delay performance in large-scale systems, and yet only require minimal implementation overhead. Aimed at a broad audience, the paper starts with an introduction to the basic load balancing scenario, consisting of a single dispatcher where tasks arrive that must immediately be forwarded to one of $N$ single-server queues. A popular class of load balancing algorithms are so-called power-of-$d$ or JSQ($d$) policies, where an incoming task is assigned to a server with the shortest queue among $d$ servers selected uniformly at random. This class includes the Join-the-Shortest-Queue (JSQ) policy as a special case ($d = N$), which has strong stochastic optimality properties and yields a mean waiting time that vanishes as $N$ grows large for any fixed subcritical load. However, a nominal implementation of the JSQ policy involves a prohibitive communication burden in large-scale deployments. In contrast, a random assignment policy ($d = 1$) does not entail any communication overhead, but the mean waiting time remains constant as $N$ grows large for any fixed positive load. In order to examine the fundamental trade-off between performance and implementation overhead, we consider an asymptotic regime where $d(N)$ depends on $N$. We investigate what growth rate of $d(N)$ is required to match the performance of the JSQ policy on fluid and diffusion scale. The results demonstrate that the asymptotics for the JSQ($d(N)$) policy are insensitive to the exact growth rate of $d(N)$, as long as the latter is sufficiently fast, implying that the optimality of the JSQ policy can asymptotically be preserved while dramatically reducing the communication overhead. We additionally show how the communication overhead can be reduced yet further by the so-called Join-the-Idle-Queue scheme, leveraging memory at the dispatcher.Comment: Survey paper. Contribution to the Proceedings of the ICM 201

Transform Methods for Heavy-Traffic Analysis

The drift method was recently developed to study queueing systems in steady-state. It was successfully used to obtain bounds on the moments of the scaled queue lengths, that are asymptotically tight in heavy-traffic, in a wide variety of systems including generalized switches, input-queued switches, bandwidth sharing networks, etc. In this paper we develop the use of transform techniques for heavy-traffic analysis, with a special focus on the use of moment generating functions. This approach simplifies the proofs of the drift method, and provides a new perspective on the drift method. We present a general framework and then use the MGF method to obtain the stationary distribution of queue lengths in heavy-traffic in queueing systems that satisfy the Complete Resource Pooling condition. In particular, we study load balancing systems and generalized switches under general settings

A Simple Steady-State Analysis of Load Balancing Algorithms in the Sub-Halfin-Whitt Regime

This paper studies a class of load balancing algorithms for many-server ($N$ servers) systems assuming finite buffer with size $b-1$ (i.e. a server can have at most one job in service and $b-1$ jobs in queue). We focus on steady-state performance of load balancing algorithms in the heavy traffic regime such that the load of system is $\lambda = 1 - N^{-\alpha}$ for $0<\alpha<0.5,$ which we call sub-Halfin-Whitt regime ($\alpha=0.5$ is the so-called the Halfin-Whitt regime). We establish a sufficient condition under which the probability that an incoming job is routed to an idle server is one asymptotically. The class of load balancing algorithms that satisfy the condition includes join-the-shortest-queue (JSQ), idle-one-first (I1F), join-the-idle-queue (JIQ), and power-of-$d$-choices (Po$d$) with $d=N^\alpha\log N$. The proof of the main result is based on the framework of Stein's method. A key contribution is to use a simple generator approximation based on state space collapse