Low latency via redundancy

Low latency is critical for interactive networked applications. But while we know how to scale systems to increase capacity, reducing latency --- especially the tail of the latency distribution --- can be much more difficult. In this paper, we argue that the use of redundancy is an effective way to convert extra capacity into reduced latency. By initiating redundant operations across diverse resources and using the first result which completes, redundancy improves a system's latency even under exceptional conditions. We study the tradeoff with added system utilization, characterizing the situations in which replicating all tasks reduces mean latency. We then demonstrate empirically that replicating all operations can result in significant mean and tail latency reduction in real-world systems including DNS queries, database servers, and packet forwarding within networks

A numerical approach to cyclic-service queueing models

Queueing Theory;operations research

Performance modelling with adaptive hidden Markov models and discriminatory processor sharing queues

In modern computer systems, workload varies at different times and locations. It is important to model the performance of such systems via workload models that are both representative and efficient. For example, model-generated workloads represent realistic system behaviour, especially during peak times, when it is crucial to predict and address performance bottlenecks. In this thesis, we model performance, namely throughput and delay, using adaptive models and discrete queues. Hidden Markov models (HMMs) parsimoniously capture the correlation and burstiness of workloads with spatiotemporal characteristics. By adapting the batch training of standard HMMs to incremental learning, online HMMs act as benchmarks on workloads obtained from live systems (i.e. storage systems and financial markets) and reduce time complexity of the Baum-Welch algorithm. Similarly, by extending HMM capabilities to train on multiple traces simultaneously it follows that workloads of different types are modelled in parallel by a multi-input HMM. Typically, the HMM-generated traces verify the throughput and burstiness of the real data. Applications of adaptive HMMs include predicting user behaviour in social networks and performance-energy measurements in smartphone applications. Equally important is measuring system delay through response times. For example, workloads such as Internet traffic arriving at routers are affected by queueing delays. To meet quality of service needs, queueing delays must be minimised and, hence, it is important to model and predict such queueing delays in an efficient and cost-effective manner. Therefore, we propose a class of discrete, processor-sharing queues for approximating queueing delay as response time distributions, which represent service level agreements at specific spatiotemporal levels. We adapt discrete queues to model job arrivals with distributions given by a Markov-modulated Poisson process (MMPP) and served under discriminatory processor-sharing scheduling. Further, we propose a dynamic strategy of service allocation to minimise delays in UDP traffic flows whilst maximising a utility function.Open Acces

Priority Auctions and Queue Disciplines that Depend on Processing Time

Lecture on the first SFB/TR 15 meeting, Gummersbach, July, 18 - 20, 2004We analyze the allocation of priority in queues via simple bidding mechanisms. In our model, the stochastically arriving customers are privately informed about their own processing time. They make bids upon arrival at a queue whose length is unobservable. We consider two bidding schemes that differ in the definition of bids (these may reflect either total payments or payments per unit of time) and in the timing of payments (before, or after service). In both schemes, a customer obtains priority over all customers (waiting in the queue or arriving while he is waiting) who make lower bids. Our main results show how the convexity/concavity of the function expressing the costs of delay determines the queue-discipline (i.e., SPT, LPT) arising in a bidding equilibrium

Optimal Pricing Effect on Equilibrium Behaviors of Delay-Sensitive Users in Cognitive Radio Networks

This paper studies price-based spectrum access control in cognitive radio networks, which characterizes network operators' service provisions to delay-sensitive secondary users (SUs) via pricing strategies. Based on the two paradigms of shared-use and exclusive-use dynamic spectrum access (DSA), we examine three network scenarios corresponding to three types of secondary markets. In the first monopoly market with one operator using opportunistic shared-use DSA, we study the operator's pricing effect on the equilibrium behaviors of self-optimizing SUs in a queueing system. %This queue represents the congestion of the multiple SUs sharing the operator's single \ON-\OFF channel that models the primary users (PUs) traffic. We provide a queueing delay analysis with the general distributions of the SU service time and PU traffic using the renewal theory. In terms of SUs, we show that there exists a unique Nash equilibrium in a non-cooperative game where SUs are players employing individual optimal strategies. We also provide a sufficient condition and iterative algorithms for equilibrium convergence. In terms of operators, two pricing mechanisms are proposed with different goals: revenue maximization and social welfare maximization. In the second monopoly market, an operator exploiting exclusive-use DSA has many channels that will be allocated separately to each entering SU. We also analyze the pricing effect on the equilibrium behaviors of the SUs and the revenue-optimal and socially-optimal pricing strategies of the operator in this market. In the third duopoly market, we study a price competition between two operators employing shared-use and exclusive-use DSA, respectively, as a two-stage Stackelberg game. Using a backward induction method, we show that there exists a unique equilibrium for this game and investigate the equilibrium convergence.Comment: 30 pages, one column, double spac