Routing policies for a partially observable two-server queueing system

We consider a queueing system controlled by decisions based on partial state information. The motivation for this work stems from road traffic, in which drivers may, or may not, be subscribed to a smartphone application for dynamic route planning. Our model consists of two queues with independent ex-ponential service times, serving two types of jobs. Arrivals occur according to a Poisson process; a fraction of the jobs (type X) is observable and controllable. At all times the number of X jobs in each queue and their individual po-sitions are known. Upon its arrival a router decides which queue the next X job should join. Y jobs are non-observable and non-controllable. They randomly join a queue according to some static routing probability. We address the following main research questions: 1) what penetration level is needed for effective control, 2) which policy should be implemented at the router, and 3) what is the added value of having more system information (e.g., average service times)? An extensive simulation study re-veals that for heavily loaded systems a low penetration level suucces and that the performance (in terms of the average sojourn time) of a simple policy that relies on little system information is close to w-JSQ (weighted join-the-shortest- queue policy) which is optimal in a fully controllable and observable system. The latter result is confirmed by the analysis of deterministic uid models that approximate the stochastic evolution under large loads

Stochastic methods for measurement-based network control

The main task of network administrators is to ensure that their network functions properly. Whether they manage a telecommunication or a road network, they generally base their decisions on the analysis of measurement data. Inspired by such network control applications, this dissertation investigates several stochastic modelling techniques for data analysis. The focus is on two areas within the field of stochastic processes: change point detection and queueing theory. Part I deals with statistical methods for the automatic detection of change points, being changes in the probability distribution underlying a data sequence. This part starts with a review of existing change point detection methods for data sequences consisting of independent observations. The main contribution of this part is the generalisation of the classic cusum method to account for dependence within data sequences. We analyse the false alarm probability of the resulting methods using a large deviations approach. The part also discusses numerical tests of the new methods and a cyber attack detection application, in which we investigate how to detect dns tunnels. The main contribution of Part II is the application of queueing models (probabilistic models for waiting lines) to situations in which the system to be controlled can only be observed partially. We consider two types of partial information. Firstly, we develop a procedure to get insight into the performance of queueing systems between consecutive system-state measurements and apply it in a numerical study, which was motivated by capacity management in cable access networks. Secondly, inspired by dynamic road control applications, we study routing policies in a queueing system for which just part of the jobs are observable and controllable

EUROPEAN CONFERENCE ON QUEUEING THEORY 2016

International audienceThis booklet contains the proceedings of the second European Conference in Queueing Theory (ECQT) that was held from the 18th to the 20th of July 2016 at the engineering school ENSEEIHT, Toulouse, France. ECQT is a biannual event where scientists and technicians in queueing theory and related areas get together to promote research, encourage interaction and exchange ideas. The spirit of the conference is to be a queueing event organized from within Europe, but open to participants from all over the world. The technical program of the 2016 edition consisted of 112 presentations organized in 29 sessions covering all trends in queueing theory, including the development of the theory, methodology advances, computational aspects and applications. Another exciting feature of ECQT2016 was the institution of the Takács Award for outstanding PhD thesis on "Queueing Theory and its Applications"

Optimal Network Control in Partially-Controllable Networks

The effectiveness of many optimal network control algorithms (e.g., BackPressure) relies on the premise that all of the nodes are fully controllable. However, these algorithms may yield poor performance in a partially-controllable network where a subset of nodes are uncontrollable and use some unknown policy. Such a partially-controllable model is of increasing importance in real-world networked systems such as overlay-underlay networks. In this paper, we design optimal network control algorithms that can stabilize a partially-controllable network. We first study the scenario where uncontrollable nodes use a queue-agnostic policy, and propose a low-complexity throughput-optimal algorithm, called Tracking-MaxWeight (TMW), which enhances the original MaxWeight algorithm with an explicit learning of the policy used by uncontrollable nodes. Next, we investigate the scenario where uncontrollable nodes use a queue-dependent policy and the problem is formulated as an MDP with unknown queueing dynamics. We propose a new reinforcement learning algorithm, called Truncated Upper Confidence Reinforcement Learning (TUCRL), and prove that TUCRL achieves tunable three-way tradeoffs between throughput, delay and convergence rate

Inventory management of remanufacturable products

Includes bibliographical references (p. 24-25).L. Beril Toktay, Lawrence M. Wein, Stefanos A. Zenios

Stability of the Greedy Algorithm on the Circle

We consider a single-server system with service stations in each point of the circle. Customers arrive after exponential times at uniformly-distributed locations. The server moves at finite speed and adopts a greedy routing mechanism. It was conjectured by Coffman and Gilbert in~1987 that the service rate exceeding the arrival rate is a sufficient condition for the system to be positive recurrent, for any value of the speed. In this paper we show that the conjecture holds true

Structural Properties of Optimal Fidelity Selection Policies for Human-in-the-loop Queues

We study optimal fidelity selection for a human operator servicing a queue of homogeneous tasks. The agent can service a task with a normal or high fidelity level, where fidelity refers to the degree of exactness and precision while servicing the task. Therefore, high-fidelity servicing results in higher-quality service but leads to larger service times and increased operator tiredness. We treat the cognitive state of the human operator as a lumped parameter that captures psychological factors such as workload and fatigue. The service time distribution of the human operator depends on her cognitive dynamics and the fidelity level selected for servicing the task. Her cognitive dynamics evolve as a Markov chain in which the cognitive state increases with high probability whenever she is busy and decreases while resting. The tasks arrive according to a Poisson process and the operator is penalized at a fixed rate for each task waiting in the queue. We address the trade-off between high-quality service of the task and consequent penalty due to subsequent increase in queue length using a discrete-time Semi-Markov Decision Process (SMDP) framework. We numerically determine an optimal policy and the corresponding optimal value function. Finally, we establish the structural properties of an optimal fidelity policy and provide conditions under which the optimal policy is a threshold-based policy