Asymptotic optimality of maximum pressure policies in stochastic processing networks

We consider a class of stochastic processing networks. Assume that the networks satisfy a complete resource pooling condition. We prove that each maximum pressure policy asymptotically minimizes the workload process in a stochastic processing network in heavy traffic. We also show that, under each quadratic holding cost structure, there is a maximum pressure policy that asymptotically minimizes the holding cost. A key to the optimality proofs is to prove a state space collapse result and a heavy traffic limit theorem for the network processes under a maximum pressure policy. We extend a framework of Bramson [Queueing Systems Theory Appl. 30 (1998) 89--148] and Williams [Queueing Systems Theory Appl. 30 (1998b) 5--25] from the multiclass queueing network setting to the stochastic processing network setting to prove the state space collapse result and the heavy traffic limit theorem. The extension can be adapted to other studies of stochastic processing networks.Comment: Published in at http://dx.doi.org/10.1214/08-AAP522 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

State space collapse and diffusion approximation for a network operating under a fair bandwidth sharing policy

We consider a connection-level model of Internet congestion control, introduced by Massouli\'{e} and Roberts [Telecommunication Systems 15 (2000) 185--201], that represents the randomly varying number of flows present in a network. Here, bandwidth is shared fairly among elastic document transfers according to a weighted $\alpha$-fair bandwidth sharing policy introduced by Mo and Walrand [IEEE/ACM Transactions on Networking 8 (2000) 556--567] [$\alpha\in (0,\infty)$]. Assuming Poisson arrivals and exponentially distributed document sizes, we focus on the heavy traffic regime in which the average load placed on each resource is approximately equal to its capacity. A fluid model (or functional law of large numbers approximation) for this stochastic model was derived and analyzed in a prior work [Ann. Appl. Probab. 14 (2004) 1055--1083] by two of the authors. Here, we use the long-time behavior of the solutions of the fluid model established in that paper to derive a property called multiplicative state space collapse, which, loosely speaking, shows that in diffusion scale, the flow count process for the stochastic model can be approximately recovered as a continuous lifting of the workload process.Comment: Published in at http://dx.doi.org/10.1214/08-AAP591 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

ASYMPTOTIC ANALYSIS OF SINGLE-HOP STOCHASTIC PROCESSING NETWORKS USING THE DRIFT METHOD

Today’s era of cloud computing and big data is powered by massive data centers. The focus of my dissertation is on resource allocation problems that arise in the operation of these large-scale data centers. Analyzing these systems exactly is usually intractable, and a usual approach is to study them in various asymptotic regimes with heavy traffic being a popular one. We use the drift method, which is a two-step procedure to obtain bounds that are asymptotically tight. In the first step, one shows state-space collapse, which, intuitively, means that one detects the bottleneck(s) of the system. In the second step, one sets to zero the drift of a carefully chosen test function. Then, using state-space collapse, one can obtain the desired bounds. This dissertation focuses on exploiting the properties of the drift method and providing conditions under which one can completely determine the asymptotic distribution of the queue lengths. In chapter 1 we present the motivation, research background, and main contributions. In chapter 2 we revisit some well-known definitions and results that will be repeatedly used in the following chapters. In chapter 3, chapter 4, and chapter 5 we focus on load-balancing systems, also known as supermarket checkout systems. In the load-balancing system, there are a certain number of servers, and jobs arrive in a single stream. Once they come, they join the queue associated with one of the servers, and they wait in line until the corresponding server processes them. In chapter 3 we introduce the moment generating function (MGF) method. The MGF, also known as two-sided Laplace form, is an invertible transformation of the random variable’s distribution and, hence, it provides the same information as the cumulative distribution function or the density (when it exists). The MGF method is a two-step procedure to compute the MGF of the delay in stochastic processing networks (SPNs) that satisfy the complete resource pooling (CRP) condition. Intuitively, CRP means that the SPN has a single bottleneck in heavy traffic. A popular routing algorithm is power-of-d choices, under which one selects d servers at random and routes the new arrivals to the shortest queue among those d. The power-of-d choices algorithm has been widely studied in load-balancing systems with homogeneous servers. However, it is not well understood when the servers are different. In chapter 4 we study this routing policy under heterogeneous servers. Specifically, we provide necessary and sufficient conditions on the service rates so that the load-balancing system achieves throughput and heavy-traffic optimality. We use the MGF method to show heavy-traffic optimality. In chapter 5 we study the load-balancing system in the many-server heavy-traffic regime, which means that we analyze the limit as the number of servers and the load increase together. Specifically, we are interested in studying how fast the number of servers can grow with respect to the load if we want to observe the same probabilistic behavior of the delay as a system with a fixed number of servers in heavy traffic. We show two approaches to obtain the results: the MGF method and Stein’s method. In chapter 6 we apply the MGF method to a generalized switch, which is one of the most general single-hop SPNs with control on the service process. Many systems, such as ad hoc wireless networks, input-queued switches, and parallel-server systems, can be modeled as special cases of the generalized switch. Most of the literature in SPNs (including the previous chapters of this thesis) focuses on systems that satisfy the CRP condition in heavy traffic, i.e., systems that behave as single-server queues in the limit. In chapter 7 we study systems that do not satisfy this condition and, hence, may have multiple bottlenecks. We specify conditions under which the drift method is sufficient to obtain the distribution function of the delay, and when it can only be used to obtain information about its mean value. Our results are valid for both, the CRP and non-CRP cases and they are immediately applicable to a variety of systems. Additionally, we provide a mathematical proof that shows a limitation of the drift method.Ph.D

Heavy-traffic analysis of a multiple-phase network with discriminatory processor sharing

We analyze a generalization of the Discriminatory Processor Sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue length vector in heavy traffic. The result shows that in the limit, the queue length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta (1996) who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically i.i.d. according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for d_k/E(B_k^fwd) obtain a larger share of the capacity, where d_k is the cost associated to class k, and E(B_k^fwd) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments

Heavy-traffic analysis of a multiple-phase network with discriminatory processor sharing

We analyze a generalization of the Discriminatory Processor Sharing (DPS) queue in a heavy-traffic setting. Customers present in the system are served simultaneously at rates controlled by a vector of weights. We assume that customers have phase-type distributed service requirements and allow that customers have different weights in various phases of their service. In our main result we establish a state-space collapse for the queue length vector in heavy traffic. The result shows that in the limit, the queue length vector is the product of an exponentially distributed random variable and a deterministic vector. This generalizes a previous result by Rege and Sengupta (1996) who considered a DPS queue with exponentially distributed service requirements. Their analysis was based on obtaining all moments of the queue length distributions by solving systems of linear equations. We undertake a more direct approach by showing that the probability generating function satisfies a partial differential equation that allows a closed-form solution after passing to the heavy-traffic limit. Making use of the state-space collapse result, we derive interesting properties in heavy traffic: (i) For the DPS queue we obtain that, conditioned on the number of customers in the system, the residual service requirements are asymptotically i.i.d. according to the forward recurrence times. (ii) We then investigate how the choice for the weights influences the asymptotic performance of the system. In particular, for the DPS queue we show that the scaled holding cost reduces as classes with a higher value for d_k/E(B_k^fwd) obtain a larger share of the capacity, where d_k is the cost associated to class k, and E(B_k^fwd) is the forward recurrence time of the class-k service requirement. The applicability of this result for a moderately loaded system is investigated by numerical experiments

Workload reduction of a generalized Brownian network

We consider a dynamic control problem associated with a generalized Brownian network, the objective being to minimize expected discounted cost over an infinite planning horizon. In this Brownian control problem (BCP), both the system manager's control and the associated cumulative cost process may be locally of unbounded variation. Due to this aspect of the cost process, both the precise statement of the problem and its analysis involve delicate technical issues. We show that the BCP is equivalent, in a certain sense, to a reduced Brownian control problem (RBCP) of lower dimension. The RBCP is a singular stochastic control problem, in which both the controls and the cumulative cost process are locally of bounded variation.Comment: Published at http://dx.doi.org/10.1214/105051605000000458 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org