Bayesian inference for queueing networks and modeling of internet services

Modern Internet services, such as those at Google, Yahoo!, and Amazon, handle billions of requests per day on clusters of thousands of computers. Because these services operate under strict performance requirements, a statistical understanding of their performance is of great practical interest. Such services are modeled by networks of queues, where each queue models one of the computers in the system. A key challenge is that the data are incomplete, because recording detailed information about every request to a heavily used system can require unacceptable overhead. In this paper we develop a Bayesian perspective on queueing models in which the arrival and departure times that are not observed are treated as latent variables. Underlying this viewpoint is the observation that a queueing model defines a deterministic transformation between the data and a set of independent variables called the service times. With this viewpoint in hand, we sample from the posterior distribution over missing data and model parameters using Markov chain Monte Carlo. We evaluate our framework on data from a benchmark Web application. We also present a simple technique for selection among nested queueing models. We are unaware of any previous work that considers inference in networks of queues in the presence of missing data.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS392 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Probabilistic Inference in Queueing Networks

Although queueing models have long been used to model the performance of computer systems, they are out of favor with practitioners, because they have a reputation for requiring unrealistic distributional assumptions. In fact, these distributional assumptions are used mainly to facilitate analytic approximations such as asymptotics and large-deviations bounds. In this paper, we analyze queueing networks from the probabilistic modeling perspective, applying inference methods from graphical models that afford significantly more modeling flexibility. In particular, we present a Gibbs sampler and stochastic EM algorithm for networks of M/M/1 FIFO queues. As an application of this technique, we localize performance problems in distributed systems from incomplete system trace data. On both synthetic networks and an actual distributed Web application, the model accurately recovers the system’s service time using 1 % of the available trace data.

Computationally Efficient Simulation of Queues: The R Package queuecomputer

Large networks of queueing systems model important real-world systems such as MapReduce clusters, web-servers, hospitals, call centers and airport passenger terminals. To model such systems accurately, we must infer queueing parameters from data. Unfortunately, for many queueing networks there is no clear way to proceed with parameter inference from data. Approximate Bayesian computation could offer a straightforward way to infer parameters for such networks if we could simulate data quickly enough. We present a computationally efficient method for simulating from a very general set of queueing networks with the R package queuecomputer. Remarkable speedups of more than 2 orders of magnitude are observed relative to the popular DES packages simmer and simpy. We replicate output from these packages to validate the package. The package is modular and integrates well with the popular R package dplyr. Complex queueing networks with tandem, parallel and fork/join topologies can easily be built with these two packages together. We show how to use this package with two examples: a call center and an airport terminal.Comment: Updated for queuecomputer_0.8.

Inference and Learning in Networks of Queues

Probabilistic models of the performance of computer systems are useful both for predicting system performance in new conditions, and for diagnosing past performance problems. The most popular performance models are networks of queues. However, no current methods exist for parameter estimation or inference in networks of queues with missing data. In this paper, we present a novel viewpoint that combines queueing networks and graphical models, allowing Markov chain Monte Carlo to be applied. We demonstrate the effectiveness of our sampler on real-world data from a benchmark Web application.

New activity pattern in human interactive dynamics

We investigate the response function of human agents as demonstrated by written correspondence, uncovering a new universal pattern for how the reactive dynamics of individuals is distributed across the set of each agent's contacts. In long-term empirical data on email, we find that the set of response times considered separately for the messages to each different correspondent of a given writer, generate a family of heavy-tailed distributions, which have largely the same features for all agents, and whose characteristic times grow exponentially with the rank of each correspondent. We furthermore show that this universal behavioral pattern emerges robustly by considering weighted moving averages of the priority-conditioned response-time probabilities generated by a basic prioritization model. Our findings clarify how the range of priorities in the inputs from one's environment underpin and shape the dynamics of agents embedded in a net of reactive relations. These newly revealed activity patterns might be present in other general interactive environments, and constrain future models of communication and interaction networks, affecting their architecture and evolution.Comment: 15 pages, 7 figure

Performance Modelling and Optimisation of Multi-hop Networks

A major challenge in the design of large-scale networks is to predict and optimise the total time and energy consumption required to deliver a packet from a source node to a destination node. Examples of such complex networks include wireless ad hoc and sensor networks which need to deal with the effects of node mobility, routing inaccuracies, higher packet loss rates, limited or time-varying effective bandwidth, energy constraints, and the computational limitations of the nodes. They also include more reliable communication environments, such as wired networks, that are susceptible to random failures, security threats and malicious behaviours which compromise their quality of service (QoS) guarantees. In such networks, packets traverse a number of hops that cannot be determined in advance and encounter non-homogeneous network conditions that have been largely ignored in the literature. This thesis examines analytical properties of packet travel in large networks and investigates the implications of some packet coding techniques on both QoS and resource utilisation. Specifically, we use a mixed jump and diffusion model to represent packet traversal through large networks. The model accounts for network non-homogeneity regarding routing and the loss rate that a packet experiences as it passes successive segments of a source to destination route. A mixed analytical-numerical method is developed to compute the average packet travel time and the energy it consumes. The model is able to capture the effects of increased loss rate in areas remote from the source and destination, variable rate of advancement towards destination over the route, as well as of defending against malicious packets within a certain distance from the destination. We then consider sending multiple coded packets that follow independent paths to the destination node so as to mitigate the effects of losses and routing inaccuracies. We study a homogeneous medium and obtain the time-dependent properties of the packet’s travel process, allowing us to compare the merits and limitations of coding, both in terms of delivery times and energy efficiency. Finally, we propose models that can assist in the analysis and optimisation of the performance of inter-flow network coding (NC). We analyse two queueing models for a router that carries out NC, in addition to its standard packet routing function. The approach is extended to the study of multiple hops, which leads to an optimisation problem that characterises the optimal time that packets should be held back in a router, waiting for coding opportunities to arise, so that the total packet end-to-end delay is minimised

Optimal Pricing Effect on Equilibrium Behaviors of Delay-Sensitive Users in Cognitive Radio Networks

This paper studies price-based spectrum access control in cognitive radio networks, which characterizes network operators' service provisions to delay-sensitive secondary users (SUs) via pricing strategies. Based on the two paradigms of shared-use and exclusive-use dynamic spectrum access (DSA), we examine three network scenarios corresponding to three types of secondary markets. In the first monopoly market with one operator using opportunistic shared-use DSA, we study the operator's pricing effect on the equilibrium behaviors of self-optimizing SUs in a queueing system. %This queue represents the congestion of the multiple SUs sharing the operator's single \ON-\OFF channel that models the primary users (PUs) traffic. We provide a queueing delay analysis with the general distributions of the SU service time and PU traffic using the renewal theory. In terms of SUs, we show that there exists a unique Nash equilibrium in a non-cooperative game where SUs are players employing individual optimal strategies. We also provide a sufficient condition and iterative algorithms for equilibrium convergence. In terms of operators, two pricing mechanisms are proposed with different goals: revenue maximization and social welfare maximization. In the second monopoly market, an operator exploiting exclusive-use DSA has many channels that will be allocated separately to each entering SU. We also analyze the pricing effect on the equilibrium behaviors of the SUs and the revenue-optimal and socially-optimal pricing strategies of the operator in this market. In the third duopoly market, we study a price competition between two operators employing shared-use and exclusive-use DSA, respectively, as a two-stage Stackelberg game. Using a backward induction method, we show that there exists a unique equilibrium for this game and investigate the equilibrium convergence.Comment: 30 pages, one column, double spac