Marginal productivity index policies for dynamic priority allocation in restless bandit models

Esta tesis estudia tres complejos problemas dinámicos y estocásticos de asignación de recursos: (i) Enrutamiento y control de admisión con información retrasada, (ii) Promoción dinámica de productos y el Problema de la mochila para artículos perecederos, y (iii) Control de congestión en “routers” con información del recorrido futuro. Debido a que la solución óptima de estos problemas no es asequible computacionalmente a gran y mediana escala, nos concentramos en cambio en diseñar políticas heurísticas de prioridad que sean computacionalmente tratables y cuyo rendimiento sea cuasi-óptimo. Modelizamos los problemas arriba mencionados como problemas de “multi-armed restless bandit” en el marco de procesos de decisión Markovianos con estructura especial. Empleamos y enriquecemos resultados existentes en la literatura, que constituyen un principio unificador para el diseño de políticas de índices de prioridad basadas en la relajación Lagrangiana y la descomposición de dichos problemas. Esta descomposición permite considerar subproblemas de optimización paramétrica, y en ciertos casos “indexables”, resolverlos de manera óptima mediante el índice de productividad marginal (MP). El índice MP es usado como medida de prioridad dinámica para definir reglas heurísticas de prioridad para los problemas originales intratables. Para cada uno de los problemas bajo consideración realizamos tal descomposición, identificamos las condiciones de indexabilidad, y obtenemos fórmulas para los índices MP o algoritmos computacionalmente tratables para su cálculo. Los índices MP correspondientes a cada uno de estos tres problemas pueden ser interpretados en términos de prioridades como el nivel de: (i) la penalización de dirigir un trabajo a una cola particular, (ii) la necesidad de promocionar un cierto artículo perecedero, y (iii) la utilidad de una transmisión de flujo particular. Además de la contribución práctica de la obtención de reglas heurísticas de prioridad para los tres problemas analizados, las principales contribuciones teóricas son las siguientes: (i) un algoritmo lineal en el tiempo para el cómputo de los índices MP en el problema de control de admisión con información retrasada, igualando, por lo tanto, la complejidad del mejor algoritmo existente para el caso sin retrasos, (ii) un nuevo tipo de política de índice de prioridad basada en la resolución de un problema (determinista) de la mochila, y (iii) una nueva extensión del modelo existente de “multi-armed restless bandit” a través de la incorporación de las llegadas aleatorias de los “restless bandits”.This dissertation addresses three complex stochastic and dynamic resource allocation problems: (i) Admission Control and Routing with Delayed Information, (ii) Dynamic Product Promotion and Knapsack Problem for Perishable Items, and (iii) Congestion Control in Routers with Future-Path Information. Since these problems are intractable for finding an optimal solution at middle and large scale, we instead focus on designing tractable and well-performing heuristic priority rules. We model the above problems as the multi-armed restless bandit problems in the framework of Markov decision processes with special structure. We employ and enrich existing results in the literature, which identified a unifying principle to design dynamic priority index policies based on the Lagrangian relaxation and decomposition of such problems. This decomposition allows one to consider parametric-optimization subproblems and, in certain “indexable” cases, to solve them optimally via the marginal productivity (MP) index. The MP index is then used as a dynamic priority measure to define heuristic priority rules for the original intractable problems. For each of the problems considered we perform such a decomposition, identify indexability conditions, and obtain formulae for the MP indices or tractable algorithms for their computation. The MP indices admit the following priority interpretations in the three respective problems: (i) undesirability for routing a job to a particular queue, (ii) promotion necessity of a particular perishable product, and (iii) usefulness of a particular flow transmission. Apart from the practical contribution of deriving the heuristic priority rules for the three intractable problems considered, our main theoretical contributions are the following: (i) a linear-time algorithm for computing MP indices in the admission control problem with delayed information, matching thus the complexity of the best existing algorithm under no delays, (ii) a new type of priority index policy based on solving a (deterministic) knapsack problem, and (iii) a new extension of the existing multi-armed restless bandit model by incorporating random arrivals of restless bandits

Performance analysis of differentiated resource-sharing in a wireless ad-hoc network

In this paper we model and analyze a relay node in a wireless ad-hoc network; the capacity available at this node is used to both transmit traffic from the source nodes (towards the relay node), and to serve traffic at the relay node (so that it can be forwarded to successor nodes). Clearly, when a specific node is used more heavily than others, it is prone to becoming a performance bottleneck. In this paper we consider the situation that the relay node obtains a share of the capacity that is m times as large as the share that each source node receives. The main performance metrics considered are the workload at the relay node and the average overall flow transfer time, i.e., the average time required to transmit a flow from a source node via the relay node to the destination. Our aim is to find expressions for these performance metrics for a general resource-sharing ratio m, as well as a general flow-size distribution. The analysis consists of the following steps. First, for the special case of exponential flow sizes we analyze the source-node dynamics, as well as the workload at the relay node by a fluid-flow queueing model. Then we observe from extensive numerical experimentation over a broad set of parameter values that the distribution of the number of active source nodes is actually insensitive to the flow-size distribution. Using this remarkable (empirical) result as an approximation assumption, we obtain explicit expressions for both the mean workload at the relay node and the overall flow transfer time, both for general flow-size distributions

Analysis of generic discrete-time buffer models with irregular packet arrival patterns

De kwaliteit van de multimediadiensten die worden aangeboden over de huidige breedband-communicatienetwerken, wordt in hoge mate bepaald door de performantie van de buffers die zich in de diverse netwerkele-menten (zoals schakelknooppunten, routers, modems, toegangsmultiplexers, netwerkinter- faces, ...) bevinden. In dit proefschrift bestuderen we de performantie van een dergelijke buffer met behulp van een geschikt stochastisch discrete-tijd wachtlijnmodel, waarbij we het geval van meerdere uitgangskanalen en (niet noodzakelijk identieke) pakketbronnen beschouwen, en de pakkettransmissietijden in eerste instantie één slot bedragen. De grillige, of gecorreleerde, aard van een pakketstroom die door een bron wordt gegenereerd, wordt gekarakteriseerd aan de hand van een algemeen D-BMAP (discrete-batch Markovian arrival process), wat een generiek kader creëert voor het beschrijven van een superpositie van dergelijke informatiestromen. In een later stadium breiden we onze studie uit tot het geval van transmissietijden met een algemene verdeling, waarbij we ons beperken tot een buffer met één enkel uitgangskanaal. De analyse van deze wachtlijnmodellen gebeurt hoofdzakelijk aan de hand van een particuliere wiskundig-analytische aanpak waarbij uitvoerig gebruik gemaakt wordt van probabiliteitsgenererende functies, die er toe leidt dat de diverse performantiematen (min of meer expliciet) kunnen worden uitgedrukt als functie van de systeemparameters. Dit resul-teert op zijn beurt in efficiënte en accurate berekeningsalgoritmen voor deze grootheden, die op relatief eenvoudige wijze geïmplementeerd kunnen worden

Introduction to Queueing Theory and Stochastic Teletraffic Models

The aim of this textbook is to provide students with basic knowledge of stochastic models that may apply to telecommunications research areas, such as traffic modelling, resource provisioning and traffic management. These study areas are often collectively called teletraffic. This book assumes prior knowledge of a programming language, mathematics, probability and stochastic processes normally taught in an electrical engineering course. For students who have some but not sufficiently strong background in probability and stochastic processes, we provide, in the first few chapters, background on the relevant concepts in these areas.Comment: 298 page

Performance modelling for system-level design

xii+208hlm.;24c

Revenue Management of Reusable Resources with Advanced Reservations

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/137568/1/poms12672_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/137568/2/poms12672.pd

Entropy Maximisation and Queues With or Without Balking. An investigation into the impact of generalised maximum entropy solutions on the study of queues with or without arrival balking and their applications to congestion management in communication networks.

An investigation into the impact of generalised maximum entropy solutions on the study of queues with or without arrival balking and their applications to congestion management in communication networks Keywords: Queues, Balking, Maximum Entropy (ME) Principle, Global Balance (GB), Queue Length Distribution (QLD), Generalised Geometric (GGeo), Generalised Exponential (GE), Generalised Discrete Half Normal (GdHN), Congestion Management, Packet Dropping Policy (PDP) Generalisations to links between discrete least biased (i.e. maximum entropy (ME)) distribution inferences and Markov chains are conjectured towards the performance modelling, analysis and prediction of general, single server queues with or without arrival balking. New ME solutions, namely the generalised discrete Half Normal (GdHN) and truncated GdHN (GdHNT) distributions are characterised, subject to appropriate mean value constraints, for inferences of stationary discrete state probability distributions. Moreover, a closed form global balance (GB) solution is derived for the queue length distribution (QLD) of the M/GE/1/K queue subject to extended Morse balking, characterised by a Poisson prospective arrival process, i.i.d. generalised exponential (GE) service times and finite capacity, K. In this context, based on comprehensive numerical experimentation, the latter GB solution is conjectured to be a special case of the GdHNT ME distribution. ii Owing to the appropriate operational properties of the M/GE/1/K queue subject to extended Morse balking, this queueing system is applied as an ME performance model of Internet Protocol (IP)-based communication network nodes featuring static or dynamic packet dropping congestion management schemes. A performance evaluation study in terms of the model’s delay is carried out. Subsequently, the QLD’s of the GE/GE/1/K censored queue subject to extended Morse balking under three different composite batch balking and batch blocking policies are solved via the technique of GB. Following comprehensive numerical experimentation, the latter QLD’s are also conjectured to be special cases of the GdHNT. Limitations of this work and open problems which have arisen are included after the conclusion