Bayesian inference for queueing networks and modeling of internet services

Modern Internet services, such as those at Google, Yahoo!, and Amazon, handle billions of requests per day on clusters of thousands of computers. Because these services operate under strict performance requirements, a statistical understanding of their performance is of great practical interest. Such services are modeled by networks of queues, where each queue models one of the computers in the system. A key challenge is that the data are incomplete, because recording detailed information about every request to a heavily used system can require unacceptable overhead. In this paper we develop a Bayesian perspective on queueing models in which the arrival and departure times that are not observed are treated as latent variables. Underlying this viewpoint is the observation that a queueing model defines a deterministic transformation between the data and a set of independent variables called the service times. With this viewpoint in hand, we sample from the posterior distribution over missing data and model parameters using Markov chain Monte Carlo. We evaluate our framework on data from a benchmark Web application. We also present a simple technique for selection among nested queueing models. We are unaware of any previous work that considers inference in networks of queues in the presence of missing data.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS392 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

FCFS Parallel Service Systems and Matching Models

We consider three parallel service models in which customers of several types are served by several types of servers subject to a bipartite compatibility graph, and the service policy is first come first served. Two of the models have a fixed set of servers. The first is a queueing model in which arriving customers are assigned to the longest idling compatible server if available, or else queue up in a single queue, and servers that become available pick the longest waiting compatible customer, as studied by Adan and Weiss, 2014. The second is a redundancy service model where arriving customers split into copies that queue up at all the compatible servers, and are served in each queue on FCFS basis, and leave the system when the first copy completes service, as studied by Gardner et al., 2016. The third model is a matching queueing model with a random stream of arriving servers. Arriving customers queue in a single queue and arriving servers match with the first compatible customer and leave immediately with the customer, or they leave without a customer. The last model is relevant to organ transplants, to housing assignments, to adoptions and many other situations. We study the relations between these models, and show that they are closely related to the FCFS infinite bipartite matching model, in which two infinite sequences of customers and servers of several types are matched FCFS according to a bipartite compatibility graph, as studied by Adan et al., 2017. We also introduce a directed bipartite matching model in which we embed the queueing systems. This leads to a generalization of Burke's theorem to parallel service systems

Multi-resource fairness: Objectives, algorithms and performance

Designing efficient and fair algorithms for sharing multiple resources between heterogeneous demands is becoming increasingly important. Applications include compute clusters shared by multi-task jobs and routers equipped with middleboxes shared by flows of different types. We show that the currently preferred objective of Dominant Resource Fairness has a significantly less favorable efficiency-fairness tradeoff than alternatives like Proportional Fairness and our proposal, Bottleneck Max Fairness. In addition to other desirable properties, these objectives are equally strategyproof in any realistic scenario with dynamic demand

Computable bounds in fork-join queueing systems

In a Fork-Join (FJ) queueing system an upstream fork station splits incoming jobs into N tasks to be further processed by N parallel servers, each with its own queue; the response time of one job is determined, at a downstream join station, by the maximum of the corresponding tasks' response times. This queueing system is useful to the modelling of multi-service systems subject to synchronization constraints, such as MapReduce clusters or multipath routing. Despite their apparent simplicity, FJ systems are hard to analyze. This paper provides the first computable stochastic bounds on the waiting and response time distributions in FJ systems. We consider four practical scenarios by combining 1a) renewal and 1b) non-renewal arrivals, and 2a) non-blocking and 2b) blocking servers. In the case of non blocking servers we prove that delays scale as O(logN), a law which is known for first moments under renewal input only. In the case of blocking servers, we prove that the same factor of log N dictates the stability region of the system. Simulation results indicate that our bounds are tight, especially at high utilizations, in all four scenarios. A remarkable insight gained from our results is that, at moderate to high utilizations, multipath routing 'makes sense' from a queueing perspective for two paths only, i.e., response times drop the most when N = 2; the technical explanation is that the resequencing (delay) price starts to quickly dominate the tempting gain due to multipath transmissions

Controlling the order pool in make-to-order production systems

Voor ‘Make-To-Order’ (MTO, oftewel klantordergestuurde) productiesystemen is de tijd die orders moeten wachten op beschikbare productiecapaciteit cruciaal. Het beheersen van die wachttijd is van groot belang om zowel korte als betrouwbare doorlooptijden te realiseren. Daarom analyseerde en ontwierp Remco Germs regels voor orderacceptatie en ordervrijgave, om daarmee de wachttijden te beheersen. Orderacceptatie en -vrijgave zijn de twee belangrijkste mechanismen om de lengte van wachttijden te beïnvloeden en zodoende de productie te sturen. De logistieke prestatie hangt in grote mate af van specifieke kenmerken van MTO-systemen, zoals routing variabiliteit, beperkte productiecapaciteit, omsteltijden, strikte leveringsvoorwaarden en onzekerheid in het aankomstpatroon van orders. Om een beter begrip te krijgen van de afwegingen die MTO-bedrijven in dit opzicht moeten maken richt het proefschrift zich op de modellering van de belangrijkste kenmerken van MTO-systemen. De inzichten die dat oplevert worden vervolgens gebruikt om orderacceptatie- en ordervrijgaveregels te ontwikkelen die eenvoudig te begrijpen en daarom makkelijk in praktijksituaties te implementeren zijn. Deze relatief eenvoudige beslissingsregels kunnen al leiden tot significante verbeteringen in de logistieke prestaties van MTO-bedrijven. The thesis of Remco Germs analyses and develops order acceptance and order release policies to control queues in make-to-order (MTO) production systems. Controlling the time orders spend waiting in queues is crucial for realizing short and reliable delivery times, two performance measures which are of strategic importance for many MTO com-panies. Order acceptance and order release are the two most important production con-trol mechanisms to influence the length of these queues. Their performance depends on typical characteristics of MTO systems, such as random (batch) order arrival, routing variability, fixed capacities, setup times and (strict) due-dates. To better understand the underlying mechanisms of good order acceptance and order release policies the models in this thesis focus on the main characteristics of MTO systems. The insights obtained from these models are then used to develop order acceptance and order release policies that are easy to understand and thereby easy to implement in practice. The results show that these relatively simple policies may already lead to significant performance improvements for MTO companies.