Randomized Assignment of Jobs to Servers in Heterogeneous Clusters of Shared Servers for Low Delay

We consider the job assignment problem in a multi-server system consisting of $N$ parallel processor sharing servers, categorized into $M$ ($\ll N$) different types according to their processing capacity or speed. Jobs of random sizes arrive at the system according to a Poisson process with rate $N \lambda$. Upon each arrival, a small number of servers from each type is sampled uniformly at random. The job is then assigned to one of the sampled servers based on a selection rule. We propose two schemes, each corresponding to a specific selection rule that aims at reducing the mean sojourn time of jobs in the system. We first show that both methods achieve the maximal stability region. We then analyze the system operating under the proposed schemes as $N \to \infty$ which corresponds to the mean field. Our results show that asymptotic independence among servers holds even when $M$ is finite and exchangeability holds only within servers of the same type. We further establish the existence and uniqueness of stationary solution of the mean field and show that the tail distribution of server occupancy decays doubly exponentially for each server type. When the estimates of arrival rates are not available, the proposed schemes offer simpler alternatives to achieving lower mean sojourn time of jobs, as shown by our numerical studies

Energy Efficient Scheduling and Routing via Randomized Rounding

We propose a unifying framework based on configuration linear programs and randomized rounding, for different energy optimization problems in the dynamic speed-scaling setting. We apply our framework to various scheduling and routing problems in heterogeneous computing and networking environments. We first consider the energy minimization problem of scheduling a set of jobs on a set of parallel speed scalable processors in a fully heterogeneous setting. For both the preemptive-non-migratory and the preemptive-migratory variants, our approach allows us to obtain solutions of almost the same quality as for the homogeneous environment. By exploiting the result for the preemptive-non-migratory variant, we are able to improve the best known approximation ratio for the single processor non-preemptive problem. Furthermore, we show that our approach allows to obtain a constant-factor approximation algorithm for the power-aware preemptive job shop scheduling problem. Finally, we consider the min-power routing problem where we are given a network modeled by an undirected graph and a set of uniform demands that have to be routed on integral routes from their sources to their destinations so that the energy consumption is minimized. We improve the best known approximation ratio for this problem.Comment: 27 page

Analysis of randomized join-the-shortest-queue (JSQ) schemes in large heterogeneous processor-sharing systems

In this paper, we investigate the stability and performance of randomized dynamic routing schemes for jobs based on the Join-the-Shortest Queue (JSQ) criterion in a heterogeneous system of many parallel servers. In particular, we consider servers that use processor sharing but with different server rates, and jobs are routed to the server with the smallest occupancy among a finite number of randomly sampled servers. We focus on the case of two servers that is often referred to as a Power-of-Two scheme. We first show that in the heterogeneous setting, uniform sampling of servers can cause a loss in the stability region and thus such randomized dynamic schemes need not outperform static randomized schemes in terms of mean delay in opposition to the homogeneous case of equal server speeds where the stability region is maximal and coincides with that of the static randomized routing. We explicitly characterize the stationary distributions of the server occupancies and show that the tail distribution of the server occupancy has a super-exponential behavior as in the homogeneous case as the number of servers goes to infinity. To overcome the stability issue, we show that it is possible to combine the static state-independent scheme with a randomized JSQ scheme that allows us to recover the maximal stability region combined with the benefits of JSQ, and such a scheme is preferable in terms of average delay. The techniques are based on a mean field analysis where we show that the stationary distributions coincide with those obtained under asymptotic independence of the servers and, moreover, the stationary distributions are insensitive to the job-size distribution

Adaptive Dispatching of Tasks in the Cloud

The increasingly wide application of Cloud Computing enables the consolidation of tens of thousands of applications in shared infrastructures. Thus, meeting the quality of service requirements of so many diverse applications in such shared resource environments has become a real challenge, especially since the characteristics and workload of applications differ widely and may change over time. This paper presents an experimental system that can exploit a variety of online quality of service aware adaptive task allocation schemes, and three such schemes are designed and compared. These are a measurement driven algorithm that uses reinforcement learning, secondly a "sensible" allocation algorithm that assigns jobs to sub-systems that are observed to provide a lower response time, and then an algorithm that splits the job arrival stream into sub-streams at rates computed from the hosts' processing capabilities. All of these schemes are compared via measurements among themselves and with a simple round-robin scheduler, on two experimental test-beds with homogeneous and heterogeneous hosts having different processing capacities.Comment: 10 pages, 9 figure

The mean-field behavior of processor sharing systems with general job lengths under the SQ(d) policy

This paper addresses the mean-field behavior of large-scale systems of parallel servers with a processor sharing service discipline when arrivals are Poisson and jobs have general service time distributions when an SQ() routing policy is used. Under this policy, an arrival is routed to the server with the least number of progressing jobs among randomly chosen servers. The limit of the empirical distribution is then used to study the statistical properties of the system. In particular, this shows that in the limit as grows, individual servers are statistically independent of others (propagation of chaos) and more importantly, the equilibrium point of the mean-field is insensitive to the job length distributions that has important engineering relevance for the robustness of such routing policies used in web server farms. We use a framework of measure-valued processes and martingale techniques to obtain our results. We also provide numerical results to support our analysis

Choosing among heterogeneous server clouds

This paper considers a model of interest in cloud computing applications. We consider a multiserver system consisting of N heterogeneous servers. The servers are categorized into M( ≪N ) different types according to their service capabilities. Jobs having specific resource requirements arrive at the system according to a Poisson process with rate Nλ . Upon each arrival, a small number of servers are sampled uniformly at random from each server type. The job is then routed to the sampled server with maximum vacancy per server capacity. If a job cannot obtain the required amount of resources from the server to which it is assigned, then the job is discarded. We analyze the system in the limit as N→∞ . This gives rise to a mean field, which we show has a unique fixed point and is globally attractive. Furthermore, as N→∞ , the servers behave independently. The stationary tail probabilities of server occupancies are obtained from the stationary solution of the mean field. Numerical results suggest that the proposed scheme significantly reduces the average blocking probability compared to static schemes that probabilistically route jobs to servers in proportion to the number of servers of each type. Moreover, the reduction in blocking holds even for systems at high load. For the limiting system in statistical equilibrium, our simulation results indicate that the occupancy distribution is insensitive to the holding time distribution and only depends on its mean

A sweep algorithm for massively parallel simulation of circuit-switched networks

A new massively parallel algorithm is presented for simulating large asymmetric circuit-switched networks, controlled by a randomized-routing policy that includes trunk-reservation. A single instruction multiple data (SIMD) implementation is described, and corresponding experiments on a 16384 processor MasPar parallel computer are reported. A multiple instruction multiple data (MIMD) implementation is also described, and corresponding experiments on an Intel IPSC/860 parallel computer, using 16 processors, are reported. By exploiting parallelism, our algorithm increases the possible execution rate of such complex simulations by as much as an order of magnitude