Bayesian inference for queueing networks and modeling of internet services

Modern Internet services, such as those at Google, Yahoo!, and Amazon, handle billions of requests per day on clusters of thousands of computers. Because these services operate under strict performance requirements, a statistical understanding of their performance is of great practical interest. Such services are modeled by networks of queues, where each queue models one of the computers in the system. A key challenge is that the data are incomplete, because recording detailed information about every request to a heavily used system can require unacceptable overhead. In this paper we develop a Bayesian perspective on queueing models in which the arrival and departure times that are not observed are treated as latent variables. Underlying this viewpoint is the observation that a queueing model defines a deterministic transformation between the data and a set of independent variables called the service times. With this viewpoint in hand, we sample from the posterior distribution over missing data and model parameters using Markov chain Monte Carlo. We evaluate our framework on data from a benchmark Web application. We also present a simple technique for selection among nested queueing models. We are unaware of any previous work that considers inference in networks of queues in the presence of missing data.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS392 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Human activity modeling and Barabasi's queueing systems

It has been shown by A.-L. Barabasi that the priority based scheduling rules in single stage queuing systems (QS) generates fat tail behavior for the tasks waiting time distributions (WTD). Such fat tails are due to the waiting times of very low priority tasks which stay unserved almost forever as the task priority indices (PI) are "frozen in time" (i.e. a task priority is assigned once for all to each incoming task). Relaxing the "frozen in time" assumption, this paper studies the new dynamic behavior expected when the priority of each incoming tasks is time-dependent (i.e. "aging mechanisms" are allowed). For two class of models, namely 1) a population type model with an age structure and 2) a QS with deadlines assigned to the incoming tasks which is operated under the "earliest-deadline-first" policy, we are able to analytically extract some relevant characteristics of the the tasks waiting time distribution. As the aging mechanism ultimately assign high priority to any long waiting tasks, fat tails in the WTD cannot find their origin in the scheduling rule alone thus showing a fundamental difference between the present and the A.-L. Barabasi's class of models.Comment: 16 pages, 2 figure

Partially shared buffers with full or mixed priority

This paper studies a finite-sized discrete-time two-class priority queue. Packets of both classes arrive according to a two-class discrete batch Markovian arrival process (2-DBMAP), taking into account the correlated nature of arrivals in heterogeneous telecommunication networks. The model incorporates time and space priority to provide different types of service to each class. One of both classes receives absolute time priority in order to minimize its delay. Space priority is implemented by the partial buffer sharing acceptance policy and can be provided to the class receiving time priority or to the other class. This choice gives rise to two different queueing models and this paper analyses both these models in a unified manner. Furthermore, the buffer finiteness and the use of space priority raise some issues on the order of arrivals in a slot. This paper does not assume that all arrivals from one class enter the queue before those of the other class. Instead, a string representation for sequences of arriving packets and a probability measure on the set of such strings are introduced. This naturally gives rise to the notion of intra-slot space priority. Performance of these queueing systems is then determined using matrix-analytic techniques. The numerical examples explore the range of service differentiation covered by both models

Asymptotically Optimal Load Balancing Topologies

We consider a system of $N$ servers inter-connected by some underlying graph topology $G_N$. Tasks arrive at the various servers as independent Poisson processes of rate $\lambda$. Each incoming task is irrevocably assigned to whichever server has the smallest number of tasks among the one where it appears and its neighbors in $G_N$. Tasks have unit-mean exponential service times and leave the system upon service completion. The above model has been extensively investigated in the case $G_N$ is a clique. Since the servers are exchangeable in that case, the queue length process is quite tractable, and it has been proved that for any $\lambda < 1$, the fraction of servers with two or more tasks vanishes in the limit as $N \to \infty$. For an arbitrary graph $G_N$, the lack of exchangeability severely complicates the analysis, and the queue length process tends to be worse than for a clique. Accordingly, a graph $G_N$ is said to be $N$-optimal or $\sqrt{N}$-optimal when the occupancy process on $G_N$ is equivalent to that on a clique on an $N$-scale or $\sqrt{N}$-scale, respectively. We prove that if $G_N$ is an Erd\H{o}s-R\'enyi random graph with average degree $d(N)$, then it is with high probability $N$-optimal and $\sqrt{N}$-optimal if $d(N) \to \infty$ and $d(N) / (\sqrt{N} \log(N)) \to \infty$ as $N \to \infty$, respectively. This demonstrates that optimality can be maintained at $N$-scale and $\sqrt{N}$-scale while reducing the number of connections by nearly a factor $N$ and $\sqrt{N} / \log(N)$ compared to a clique, provided the topology is suitably random. It is further shown that if $G_N$ contains $\Theta(N)$ bounded-degree nodes, then it cannot be $N$-optimal. In addition, we establish that an arbitrary graph $G_N$ is $N$-optimal when its minimum degree is $N - o(N)$, and may not be $N$-optimal even when its minimum degree is $c N + o(N)$ for any $0 < c < 1/2$.Comment: A few relevant results from arXiv:1612.00723 are included for convenienc