A QoS-aware workload routing and server speed scaling policy for energy-efficient data centers: a robust queueing theoretic approach

Maintaining energy efficiency in large data centers depends on the ability to manage workload routing and control server speeds according to fluctuating demand. The use of dynamic algorithms often means that management has to install the complicated software or expensive hardware needed to communicate with routers and servers. This paper proposes a static routing and server speed scaling policy that may achieve energy efficiency similar to dynamic algorithms and eliminate the necessity of frequent communications among resources without compromising quality of service (QoS). We use a robust queueing approach to consider the response time constraints, e.g., service level agreements (SLAs). We model each server as a $G/G/1$ processor sharing (PS) queue and use uncertainty sets to define the domain of random variables. A comparison with a dynamic algorithm shows that the proposed static policy provides competitive solutions in terms of energy efficiency and satisfactory QoS

Unbiased time-average estimators for Markov chains

We consider a time-average estimator $f_{k}$ of a functional of a Markov chain. Under a coupling assumption, we show that the expectation of $f_{k}$ has a limit $\mu$ as the number of time-steps goes to infinity. We describe a modification of $f_{k}$ that yields an unbiased estimator $\hat f_{k}$ of $\mu$. It is shown that $\hat f_{k}$ is square-integrable and has finite expected running time. Under certain conditions, $\hat f_{k}$ can be built without any precomputations, and is asymptotically at least as efficient as $f_{k}$, up to a multiplicative constant arbitrarily close to $1$. Our approach provides an unbiased estimator for the bias of $f_{k}$. We study applications to volatility forecasting, queues, and the simulation of high-dimensional Gaussian vectors. Our numerical experiments are consistent with our theoretical findings.Comment: 37 page

Robust Multiclass Queuing Theory for Wait Time Estimation in Resource Allocation Systems

In this paper, we study systems that allocate different types of scarce resources to heterogeneous allocatees based on predetermined priority rules-the U.S. deceased-donor kidney allocation system or the public housing program. We tackle the problem of estimating the wait time of an allocatee who possesses incomplete system information with regard, for example, to his relative priority, other allocatees' preferences, and resource availability. We model such systems as multiclass, multiserver queuing systems that are potentially unstable or in transient regime. We propose a novel robust optimization solution methodology that builds on the assignment problem. For first-come, first-served systems, our approach yields a mixed-integer programming formulation. For the important case where there is a hierarchy in the resource types, we strengthen our formulation through a drastic variable reduction and also propose a highly scalable heuristic, involving only the solution of a convex optimization problem (usually a second-order cone problem).We back the heuristic with an approximation guarantee that becomes tighter for larger problem sizes. We illustrate the generalizability of our approach by studying systems that operate under different priority rules, such as class priority. Numerical studies demonstrate that our approach outperforms simulation. We showcase how our methodology can be applied to assist patients in the U.S. deceased-donor kidney waitlist. We calibrate our model using historical data to estimate patients' wait times based on their kidney quality preferences, blood type, location, and rank in the waitlist

Boundary–crossing probabilities for stochastic processes and their applications

In this thesis, we focus on the problem that a stochastic process crossing (or not crossing) upper and/or lower deterministic boundaries and its application in statistics, inventory management, finance, risk and ruin theory and queueing. In Chapter 2, we provide a fast and accurate method based on fast Fourier transform (FFT), to compute the (complementary) cumulative distribution function (CDF) of the Kolmogorov-Smirnov (KS) statistic when the CDF under the null hypothesis, F(x), is purely discrete, mixed or continuous, and thus obtain exact p values of the KS test. Secondly, we developed a C++ and an R implementation of the proposed method, which fills in the existing gap in statistical software. The numerical performance of the proposed FFT-based method, implemented both in C++ and in the R package KSgeneral, available from https://CRAN.R project.org/package=KSgeneral, is illustrated when F(x) is mixed, purely discrete, and continuous. In Chapter 3, we develop an efficient method based on FFT, for computing the probability that a non-decreasing, pure jump (compound) stochastic process stays between arbitrary upper and lower boundaries (i.e., deterministic functions, possibly discontinuous) within a finite time period. We further demonstrate that our FFT-based method is computationally efficient and can be successfully applied in the context of inventory management (to determine an optimal replenishment policy), ruin theory (to evaluate ruin probabilities and related quantities) and double-barrier option pricing or simply computing non-exit probabilities for Brownian motion with general boundaries. In Chapter 4, we give explicit formulas and a numerically efficient FFT-based method for computing the probability that a non-decreasing, pure jump stochastic process will first exit from above the strip between two deterministic, possibly discontinuous, time-dependent boundaries, within a finite-time interval with an overshoot (not) exceeding a positive value. The stochastic process is a compound process with events of interest arriving according to an arbitrary point process with conditional stationary independent increments (PPCSII), and event severities with any possibly dependent joint distribution. The class of PPCSII is rather rich covering point processes with independent increments (among which non homogeneous Poisson processes and negative binomial processes), doubly stochastic Poisson (i.e., Cox processes) including mixed Poisson processes (among which processes with the order statistics property) and Markov modulated point processes. These assumptions make our framework and results generally applicable for a broad range of models arising in insurance, finance, queueing, economics, physics, astronomy and many other fields. We present examples of such applications in queueing, ruin and inventory management optimization, leading to new results in the latter fields, illustrated also numerically. In Chapter 5, we consider the large class of PPCSII and the family GD of random variables with arbitrary, possibly dependent joint distribution. These families are interchangeably used to model customers arrival and service times in the very general framework of GD/PPCSII/1 and its inverse PPCSII/GD/1 queueing models. The latter cover well known models, e.g. the G/M/1 and M/G/1 queues, but also models incorporating dependence in the arrival times, service times and across, either by directly stating their joint distribution, through a copula and appropriate marginals, or through the PPCSII class. We further introduce a double–boundary crossing (DBC) queueing duality that extends the known Cramér–Lundberg – G/M/1 duality. The DBC–queueing duality is used to establish new results with respect to the joint and marginal distributions of the busy period, idle time and the maximum waiting time, including bounds, approximations and closed form formulas. We present a FFT-based method for efficient computation of the latter distributions. We also formulate and solve novel profit optimization problems, e.g., of determining the optimal capacity of the server so as to maximize the worse-case profit margin jointly with its related probability. Results are illustrated numerically

A Robust Queueing Network Analyzer Based on Indices of Dispersion

In post-industrial economies, modern service systems are dramatically changing the daily lives of many people. Such systems are often complicated by uncertainty: service providers usually cannot predict when a customer will arrive and how long the service will be. Fortunately, useful guidance can often be provided by exploiting stochastic models such as queueing networks. In iterating the design of service systems, decision makers usually favor analytical analysis of the models over simulation methods, due to the prohibitive computation time required to obtain optimal solutions for service operation problems involving multidimensional stochastic networks. However, queueing networks that can be solved analytically require strong assumptions that are rarely satisfied, whereas realistic models that exhibit complicated dependence structure are prohibitively hard to analyze exactly. In this thesis, we continue the effort to develop useful analytical performance approximations for the single-class open queueing network with Markovian routing, unlimited waiting space and the first-come first-served service discipline. We focus on open queueing networks where the external arrival processes are not Poisson and the service times are not exponential. We develop a new non-parametric robust queueing algorithm for the performance approximation in single-server queues. With robust optimization techniques, the underlying stochastic processes are replaced by samples from suitably defined uncertainty sets and the worst-case scenario is analyzed. We show that this worst-case characterization of the performance measure is asymptotically exact for approximating the mean steady-state workload in G/G/1 models in both the light-traffic and heavy-traffic limits, under mild regularity conditions. In our non-parametric Robust Queueing formulation, we focus on the customer flows, defined as the continuous-time processes counting customers in or out of the network, or flowing from one queue to another. Each flow is partially characterized by a continuous function that measures the change of stochastic variability over time. This function is called the index of dispersion for counts. The Robust Queueing algorithm converts the index of dispersion for counts into approximations of the performance measures. We show the advantage of using index of dispersion for counts in queueing approximation by a renewal process characterization theorem and the ordering of the mean steady-state workload in GI/M/1 models. To develop generalized algorithm for open queueing networks, we first establish the heavy-traffic limit theorem for the stationary departure flows from a GI/GI/1 model. We show that the index of dispersion for counts function of the stationary departure flow can be approximately characterized as the convex combination of the arrival index of dispersion for counts and service index of dispersion for counts with a time-dependent weight function, revealing the non-trivial impact of the traffic intensity on the departure processes. This heavy-traffic limit theorem is further generalized into a joint heavy-traffic limit for the stationary customer flows in generalized Jackson networks, where the external arrival are characterized by independent renewal processes and the service times are independent and identically distributed random variables, independent of the external arrival processes. We show how these limiting theorems can be exploited to establish a set of linear equations, whose solution serves as approximations of the index of dispersion for counts of the flows in an open queueing network. We prove that this set of equations is asymptotically exact in approximating the index of dispersion for counts of the stationary flows. With the index of dispersion for counts available, the network is decomposed into single-server queues and the Robust Queueing algorithm can be applied to obtain performance approximation. This algorithm is referred to as the Robust Queueing Network Analyzer. We perform extensive simulation study to validate the effectiveness of our algorithm. We show that our algorithm can be applied not only to models with non-exponential distirbutions but also to models with more complex arrival processes than renewal processes, including those with Markovian arrival processes

Efficient Simulation and Performance Stabilization for Time-Varying Single-Server Queues

This thesis develops techniques to evaluate and to improve the performance of single-server service systems with time-varying arrivals. The performance measures considered are the time-varying expected length of the queue and the expected customer waiting time. Time varying arrival rates are considered because they often occur in service systems. For example, arrival rates often vary significantly over the hours of each day and over the days of each week. Stochastic textbook methods do not apply to models with time-varying arrival rates. Hence new techniques are needed to provide high quality of service when stationary steady-state analysis is not appropriate. In contrast to the extensive recent literature on many-server queues with time-varying arrival rates, we focus on single-server queues with time-varying arrival rates. Single-server queues arise in real applications where there is no flexibility in the number of service facilities (servers). Different analysis techniques are required for single-server queues, because the two kinds of models exhibit very different performance. Many-server models are more tractable because methods for highly tractable infinite-server models can be applied. In contrast, single-server models are more complicated because it takes a long time to respond to a build up of workload when there is only one server. The thesis is divided into two parts: simulation algorithms for performance evaluation and service-rate controls for performance stabilization. The first part of the thesis develops algorithms to efficiently simulate the single-server time-varying queue. For the generality considered, no explicit mathematical formulas are available for calculating performance measures, so simulation experiments are needed to calculate and evaluate system performance. Efficient algorithms for both standard simulation and rare-event simulation are developed. The second part of the thesis develops service-rate controls to stabilize performance in the time-varying single-server queue. The performance stabilization problem aims to minimize fluctuations in mean waiting times for customers coming at different times even though the arrival rate is time-varying. A new service rate control is developed, where the service rate at each time is a function of the arrival rate function. We show that a specific service rate control can be found to stabilize performance. In turn, that service rate control can be used to provide guidance for real applications on optimal changes in staffing, processing speed or machine power status over time. Both the simulation experiments to evaluate performance of alternative service-rate controls and the simulation search algorithm to find the best parameters for a damped time-lag service-rate control are based on efficient performance evaluation algorithms in the first part of the thesis. In Chapter Two, we present an efficient algorithm to simulate a general non-Poisson non-stationary point process. The general point process can be represented as a time transformation of a rate-one base process and by exploiting a table of the inverse cumulative arrival rate function outside of simulation, we can efficiently convert the simulated rate-one process into the simulated general point process. The simulation experiments can be conducted in linear time subject to small error bounds. Then we can apply this efficient algorithm to generate the arrival process, the service process and thus to calculate performance measures for the G_t/G_t/1 queues, which are single-server queues with time-varying arrival rates and service rates. Service models are constructed for this purpose where time-varying service rates are specified separately from the rate-one service requirement process, and service times are determined by equating service requirements with integrals of service rates over a time period equal to the service time. In Chapter Three, we develop rare-event simulation algorithms in periodic GI_t/GI/1 queues and further in GI_t/GI_t/1 queues to estimate probabilities of rare but important events as a sanity check of the system, for example, estimating the probability that the waiting time is very long. Importance sampling, specifically exponential tilting, is required to estimate rare-event probabilities because in standard simulation, the number of experiments may blow up to achieve a targeted relative error and for each experiment, it may take a very long time to determine that the rare event does not happen. To extend the rare-event simulation algorithm to periodic queues, we derive a convenient expression for the periodic steady-state virtual waiting time. We apply this expression to establish bounds between the periodic workload and the steady-state workload in stationary queues, so that we can prove that the exponential tilting algorithm with the same parameter efficient in stationary queues is efficient in the periodic setting as well, which has a bounded relative error. We apply this algorithm to compute the periodic steady-state distribution of reflected periodic Brownian motion with support of a heavy-traffic limit theorem and to calculate the periodic steady-state distribution and moments of the virtual waiting time. This algorithm's advantage in calculating these distributions and moments is that it can directly estimate them at a specific position of the cycle without simulating the whole queueing process until steady state is reached for the whole cycle. In Chapter Four, we conduct simulation experiments to validate performance of four service-rate controls: the rate-matching control, which is directly proportional to the arrival rate, two square-root controls related to the square root staffing formula and the square-root control based on the mean stationary waiting time. Simulations show that the rate-matching control stabilizes the queue length distribution but not the virtual waiting time. This is consistent with established theoretical results, which follow from the observation that with rate-matching control, the queueing process becomes a time transformation of the stationary queueing process with constant arrival rates and service rates. Simulation results also show that the two square-root controls analogous to the server staffing formula are not effective in stabilizing performance. On the other hand, the alternative square-root service rate control based on the mean stationary waiting time approximately stabilizes the virtual waiting time when the cycle is long so that the arrival rate changes slowly enough. In Chapter Five, since we are mostly interested in stabilizing waiting times in more common scenarios when the traffic intensity is not close to one or when the arrival rate does not change slowly, we develop a damped time-lag service-rate control that performs fairly well for this purpose. This control is a modification of the rate-matching control involving a time lag and a damping factor. To find the best parameters for this control, we search over reasonable intervals for the most time-stable performance measures, which are computed by the extended rare-event simulation algorithm in GI_t/GI_t/1 queue. We conduct simulation experiments to validate that this control is effective for stabilizing the expected steady-state virtual waiting time (and its distribution to a large extent). We also establish a heavy-traffic limit with periodicity in the fluid scale to provide theoretical support for this control. We also show that there is a time-varying Little's law in heavy-traffic, which implies that this control cannot stabilize the queue length and the waiting time at the same time