Learning Algorithms for Minimizing Queue Length Regret

We consider a system consisting of a single transmitter/receiver pair and $N$ channels over which they may communicate. Packets randomly arrive to the transmitter's queue and wait to be successfully sent to the receiver. The transmitter may attempt a frame transmission on one channel at a time, where each frame includes a packet if one is in the queue. For each channel, an attempted transmission is successful with an unknown probability. The transmitter's objective is to quickly identify the best channel to minimize the number of packets in the queue over $T$ time slots. To analyze system performance, we introduce queue length regret, which is the expected difference between the total queue length of a learning policy and a controller that knows the rates, a priori. One approach to designing a transmission policy would be to apply algorithms from the literature that solve the closely-related stochastic multi-armed bandit problem. These policies would focus on maximizing the number of successful frame transmissions over time. However, we show that these methods have $\Omega(\log{T})$ queue length regret. On the other hand, we show that there exists a set of queue-length based policies that can obtain order optimal $O(1)$ queue length regret. We use our theoretical analysis to devise heuristic methods that are shown to perform well in simulation.Comment: 28 Pages, 11 figure

Private Schools and Queue‐jumping: A reply to White

John White (2016) defends the UK private school system from the accusation that it allows an unfair form of ‘queue jumping’ in university admissions. He offers two responses to this accusation, one based on considerations of harm, and one based on meritocratic distribution of university places. We will argue that neither response succeeds: the queue-jumping argument remains a powerful case against the private school system in the UK. We begin by briefly outlining the queue-jumping argument (§1), before evaluating White’s no-harm (§2) and meritocracy (§3) arguments

Diffusion approximation for a processor sharing queue in heavy traffic

Consider a single server queue with renewal arrivals and i.i.d. service times in which the server operates under a processor sharing service discipline. To describe the evolution of this system, we use a measure valued process that keeps track of the residual service times of all jobs in the system at any given time. From this measure valued process, one can recover the traditional performance processes, including queue length and workload. We show that under mild assumptions, including standard heavy traffic assumptions, the (suitably rescaled) measure valued processes corresponding to a sequence of processor sharing queues converge in distribution to a measure valued diffusion process. The limiting process is characterized as the image under an appropriate lifting map, of a one-dimensional reflected Brownian motion. As an immediate consequence, one obtains a diffusion approximation for the queue length process of a processor sharing queue

A polling model with an autonomous server

Polling models are used as an analytical performance tool in several application areas. In these models, the focus often is on controlling the operation of the server as to optimize some performance measure. For several applications, controlling the server is not an issue as the server moves independently in the system. We present the analysis for such a polling model with a so-called autonomous server. In this model, the server remains for an exogenous random time at a queue, which also implies that service is preemptive. Moreover, in contrast to most of the previous research on polling models, the server does not immediately switch to a next queue when the current queue becomes empty, but rather remains for an exponentially distributed time at a queue. The analysis is based on considering imbedded Markov chains at specific instants. A system of equations for the queue-length distributions at these instant is given and solved for. Besides, we study to which extent the queues in the polling model are independent and identify parameter settings for which this is indeed the case. These results may be used to approximate performance measures for complex multi-queue models by analyzing a simple single-queue model

Stationary analysis of the Shortest Queue First service policy

We analyze the so-called Shortest Queue First (SQF) queueing discipline whereby a unique server addresses queues in parallel by serving at any time that queue with the smallest workload. Considering a stationary system composed of two parallel queues and assuming Poisson arrivals and general service time distributions, we first establish the functional equations satisfied by the Laplace transforms of the workloads in each queue. We further specialize these equations to the so-called "symmetric case", with same arrival rates and identical exponential service time distributions at each queue; we then obtain a functional equation $$M(z) = q(z) \cdot M \circ h(z) + L(z)$$ for unknown function $M$, where given functions $q$, $L$ and $h$ are related to one branch of a cubic polynomial equation. We study the analyticity domain of function $M$ and express it by a series expansion involving all iterates of function $h$. This allows us to determine empty queue probabilities along with the tail of the workload distribution in each queue. This tail appears to be identical to that of the Head-of-Line preemptive priority system, which is the key feature desired for the SQF discipline