On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queue

The derivation of structural properties for unbounded jump Markov processes cannot be done using standard mathematical tools, since the analysis is hindered due to the fact that the system is not uniformizable. We present a promising technique, a smoothed rate truncation method, to overcome the limitations of standard techniques and allow for the derivation of structural properties. We introduce this technique by application to a processor sharing queue with impatient customers that can retry if they renege. We are interested in structural properties of the value function of the system as a function of the arrival rate

EUROPEAN CONFERENCE ON QUEUEING THEORY 2016

International audienceThis booklet contains the proceedings of the second European Conference in Queueing Theory (ECQT) that was held from the 18th to the 20th of July 2016 at the engineering school ENSEEIHT, Toulouse, France. ECQT is a biannual event where scientists and technicians in queueing theory and related areas get together to promote research, encourage interaction and exchange ideas. The spirit of the conference is to be a queueing event organized from within Europe, but open to participants from all over the world. The technical program of the 2016 edition consisted of 112 presentations organized in 29 sessions covering all trends in queueing theory, including the development of the theory, methodology advances, computational aspects and applications. Another exciting feature of ECQT2016 was the institution of the Takács Award for outstanding PhD thesis on "Queueing Theory and its Applications"

Optimal Control of Parallel Queues for Managing Volunteer Convergence

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/163497/2/poms13224.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163497/1/poms13224_am.pd

Unreliable Retrial Queues in a Random Environment

This dissertation investigates stability conditions and approximate steady-state performance measures for unreliable, single-server retrial queues operating in a randomly evolving environment. In such systems, arriving customers that find the server busy or failed join a retrial queue from which they attempt to regain access to the server at random intervals. Such models are useful for the performance evaluation of communications and computer networks which are characterized by time-varying arrival, service and failure rates. To model this time-varying behavior, we study systems whose parameters are modulated by a finite Markov process. Two distinct cases are analyzed. The first considers systems with Markov-modulated arrival, service, retrial, failure and repair rates assuming all interevent and service times are exponentially distributed. The joint process of the orbit size, environment state, and server status is shown to be a tri-layered, level-dependent quasi-birth-and-death (LDQBD) process, and we provide a necessary and sufficient condition for the positive recurrence of LDQBDs using classical techniques. Moreover, we apply efficient numerical algorithms, designed to exploit the matrix-geometric structure of the model, to compute the approximate steady-state orbit size distribution and mean congestion and delay measures. The second case assumes that customers bring generally distributed service requirements while all other processes are identical to the first case. We show that the joint process of orbit size, environment state and server status is a level-dependent, M/G/1-type stochastic process. By employing regenerative theory, and exploiting the M/G/1-type structure, we derive a necessary and sufficient condition for stability of the system. Finally, for the exponential model, we illustrate how the main results may be used to simultaneously select mean time customers spend in orbit, subject to bound and stability constraints

Optimal control of admission in service in a queue with impatience and setup costs

International audienceWe consider a single server queue in continuous time, in which customers must be served before some limit sojourn time of exponential distribution. Customers who are not served before this limit leave the system: they are impatient. The fact of serving customers and the fact of losing them due to impatience induce costs. The fact of holding them in the queue also induces a constant cost per customer and per unit time. The purpose is to decide whether to serve customers or to keep the server idle, so as to minimize costs. We use a Markov Decision Process with infinite horizon and discounted cost. Since the standard uniformization approach is not applicable here, we introduce a family of approximated uniformizable models, for which we establish the structural properties of the stochastic dynamic programming operator, and we deduce that the optimal policy is of threshold type. The threshold is computed explicitly. We then pass to the limit to show that this threshold policy is also optimal in the original model and we characterize the optimal policy. A particular care is given to the completeness of the proof. We also illustrate the difficulties involved in the proof with numerical examples

Contrôle optimal de l’admission en service dans une file d’attente avec impatience et coûts de mise en route

We consider a single server queue in continuous time, in which customers must beserved before some limit sojourn time of exponential distribution. A customer who is not servedbefore this limit leaves the system: it is impatient. The fact of serving customers and the fact oflosing them due to impatience induce costs. The fact of holding them in the queue also induces aconstant cost per customer and per unit time. The purpose is to decide when to serve the customersso as to minimize costs. We use a Markov Decision Process with infinite horizon and discountedcost. Since the standard uniformization approach is not applicable here, we introduce a familyof approximated uniformizable models, for which we establish the structural properties of thestochastic dynamic programming operator, and we deduce that the optimal policy is of thresholdtype. The threshold is computed explicitly. We then pass to the limit to show that this thresholdpolicy is also optimal in the original model. A particular care is given to the completeness of theproof. We also illustrate the difficulties involved in the proof with numerical examples.Nous considérons un modèle d’une file d’attente à un serveur en temps continu, danslaquelle les clients doivent être servis avant une durée de séjour finie aléatoire, de distribution expo-nentielle. Un client qui n’est pas servi avant cette limite quitte le système: il est impatient. Le fait deservir les clients et le fait de perdre des clients par impatience induisent des coûts. Le fait de les garderdans la file induit également un coût constant par client et par unité de temps. Il s’agit de décider defaçon optimale quand servir les clients. Nous utilisons un processus de décision Markovien à horizoninfini et à coûts actualisés. La méthode standard d’uniformisation ne s’appliquant pas à cette situation,nous introduisons une famille de modèles approchés uniformisables pour lesquels nous établissons lespropriétés structurelles de l’opérateur de programmation dynamique stochastique, et nous déduisonsque la politique optimale est à seuil. Le seuil est calculé explicitement. Nous passons ensuite à lalimite pour montrer que cette politique à seuil est également optimale dans le modèle initial. Une at-tention particulière est apportée à la complétude de la preuve. Nous illustrons également les difficultésrencontrées à l’aide d’exemples numériques

Stability Problems for Stochastic Models: Theory and Applications II

Most papers published in this Special Issue of Mathematics are written by the participants of the XXXVI International Seminar on Stability Problems for Stochastic Models, 21­25 June, 2021, Petrozavodsk, Russia. The scope of the seminar embraces the following topics: Limit theorems and stability problems; Asymptotic theory of stochastic processes; Stable distributions and processes; Asymptotic statistics; Discrete probability models; Characterization of probability distributions; Insurance and financial mathematics; Applied statistics; Queueing theory; and other fields. This Special Issue contains 12 papers by specialists who represent 6 countries: Belarus, France, Hungary, India, Italy, and Russia

Bloody fast blood collection

This thesis consists of four parts: The first part contains an introduction, the second presents approaches for the evaluation of waiting times at blood collection sites, the third uses these to present approaches that improve waiting times at blood collection sites. The final part shows the application of two of the approaches to data from real blood collection sites, followed by the conclusions that can be drawn from this thesis. Part I: Introduction, contains two chapters. Chapter 1 introduces the context for this thesis: blood banks in general, the Dutch blood bank Sanquin and blood collection sites. The chapter sketches some of the challenges faced with respect to blood collection sites. As blood donors are voluntary and non-remunerated, delays and waiting times within blood collection sites should be kept at acceptable levels. However, waiting times are currently not incorporated in staff planning or in other decisions with respect to blood collection sites. These blood collection sites will be the primary focus of this thesis. This thesis provides methods that do take waiting times into account, aiming to decrease waiting times at blood collection sites and leveling work pressure for staff members, without the need for additional staff. Chapter 2 then presents a technical methods that will be used most of the chapters in this thesis: uniformization. Uniformization can be used to transform Continuous Time Markov Chains (CTMCs) — that are very hard to analyze — into Discrete Time Markov Chains (DTMCs) — that are much easier to analyze. The chapter shows how the method works, provides an extensive overview of the literature related to the method, the (technical) intuition behind the method as well as several extensions and applications. Although not all of the extensions and applications are necessary for this thesis, it does provide an overview of one of the most valuable methods for this thesis. Part II: Evaluation, contains two chapters that propose and adapt several methods to compute waiting times and queues at blood collection sites. A blood collection site is best modeled as a time-dependent queueing network, requiring non-standard approaches. Chapter 3 considers a stationary, i.e. not time-dependent model of blood collection sites as a first step. A blood collection site consists of three main stations: Registration, Interview and Donation. All three of the stations can have their own queue. This means that even the stationary model is non-trivial for some computations. However, for the stationary model, an analytic so-called product form expression is derived. Based on this product form, two more results are shown. The first result is that the standard waiting time distributions from M|M|s queues are applicable, as if the queue is in isolation. It is then concluded that no closed form expression exist for the total waiting or delay time distribution, as the distributions of the three stations in tandem are not independent. Therefore a numerical approach is presented to compute the total delay time distribution of a collection site. All of the results are supported by numerical examples based on a Dutch blood collection site. The approach for the computation of the total delay time distribution can also be combined with the approach from Chapter 4 for an extension to a time-dependent setting. Chapter 4 shows an approach to deal with these time-dependent aspects in queueing systems, as often experienced by blood collection sites and other service systems, typically due to time-dependent arrivals and capacities. Easy and quick to use queueing expressions generally do not apply to time-dependent situations. A large number of computational papers has been written about queue length distributions for time-dependent queues, but these are mostly theoretical and based on single queues. This chapter aims to combine computational methods with more realistic time-dependent queueing networks, with an approach based on uniformization. Although uniformization is generally perceived to be too computationally prohibitive, we show that our method is very effective for practical instances, as shown with an example of a Dutch blood collection site. The objective of the results is twofold: to show that a time-dependent queueing network approach can be beneficial and to evaluate possible improvements for Dutch blood collection sites that can only be properly assessed with a time-dependent queueing method. Part III: Optimization, contains four chapters that aim to improve service levels at Sanquin. The first three chapters focus on three different methods to decrease queues at blood collection sites. Chapters 5 and 6 focus on improving the service by optimizing staff allocation to shifts and stations. Chapter 7 focuses on improving the arrival process with the same goal. Chapter 8 is focused at improving inventory management of red blood cells. Donors do not arrive to blood collection sites uniformly throughout the day, but show clear preferences for certain times of the day. However, the arrival patterns that are shown by historical data, are not used for scheduling staff members at blood collection sites. As a first significant step to shorten waiting times we can align staff capacity and shifts with walk-in arrivals. Chapter 5 aims to optimize shift scheduling for blood collection sites. The chapter proposes a two-step procedure. First, the arrival patterns and methods from queueing theory are used to determine the required number of staff members for every half hour. Second, an integer linear program is used to compute optimal shift lengths and starting times, based on the required number of staff members. The chapter is concluded with numerical experiments that show, depending on the scenario, a reduction of waiting times, a reduction of staff members or a combination of both. At a blood collection site three stations (Registration, Interview and Donation) can roughly be distinguished. Staff members at Dutch blood collection sites are often trained to work at any of these stations, but are usually allocated to one of the stations for large fractions of a shift. If staff members change their allocation this is based on an ad hoc decision. Chapter 6 aims to take advantage of this mostly unused allocation flexibility to reduce queues at blood collection sites. As a collection site is a highly stochastic process, both in arrivals and services, an optimal allocation of staff members to the three stations is unknown, constantly changing and a challenge to determine. Chapter 6 provides and applies a so-called Markov Decision Process (MDP) to compute optimal staff assignments. Extensive numerical and simulation experiments show the potential reductions of queues when the reallocation algorithm would be implemented. Based on Dutch blood collection sites, reductions of 40 to 80% on the number of waiting donors seem attainable, depending on the scenario. Chapter 7 also aims to align the arrival of donors with scheduled staff, similarly to Chapter 5. Chapter 7 tries to achieve this by changing the arrivals of donors. By introducing appointments for an additional part of donors, arrivals can be redirected from the busiest times of the day to quiet times. An extended numerical queueing model with priorities is introduced for blood collection sites, as Sanquin wants to incentive donors to make appointments by prioritizing donors with appointments over donors without appointments. Appointment slots are added if the average queue drops below certain limits. The correct values for these limits, i.e. the values that plan the correct number of appointments, are then determined by binary search. Numerical results show that the method succeeds in decreasing excessive queues. However, the proposed priorities might result in unacceptably high waiting times for donors without appointments, and caution is therefore required before implementation. Although this thesis mainly focuses on blood collection sites, many more logistical challenges are present at a blood bank. One of these challenges arises from the expectation that Sanquin can supply hospitals with extensively typed red blood cell units directly from stock. Chapter 8 deals with this challenge. Currently, all units are issued according to the first-in-first-out principle, irrespective of their specific typing. These kind of issuing policies lead to shortages for rare blood units. Shortages for rare units could be avoided by keeping them in stock for longer, but this could also lead to unnecessary wastage. Therefore, to avoid both wastage and shortages, a trade-off between the age and rarity of a specific unit in stock should be made. For this purpose, we modeled the allocation of the inventory as a circulation flow problem, in which decisions about which units to issue are based on both the age and rarity of the units in stock. We evaluated the model for several settings of the input parameters. It turns out that, especially if only a few donors are typed for some combinations of antigens, shortages can be avoided by saving rare blood products. Moreover, the average issuing age remains unchanged. Part IV: Practice and Outlook concludes this thesis. The first of two chapters in this part shows the combined application of two approaches from this thesis to data from three collection sites in the Netherlands. The final chapter of this thesis presents the conclusions that can be drawn from this thesis and discusses an outlook for further research. Chapter 9 shows the combined application of the methods in Chapters 5 and 6 to three real collection sites in Dutch cities: Nijmegen, Leiden and Almelo. The collection sites in Nijmegen and Leiden are both large fixed collection sites. The collection site in Almelo is a mobile collection site. The application of each one of the methods individually reduce waiting times significantly, and the combined application of the methods reduces waiting times even further. Simultaneously, small reductions in the number of staff hours are attainable. The results from Chapter 9 summarize the main message of this thesis: waiting time for blood donors at blood collection sites can be reduced without the need for more staff members when the working times of staff members are used more effectively and efficiently, and controlling the arrival process of donors. The approaches presented in this thesis can be used for this purpose. This is not only beneficial for blood donors, but will also result in more balanced workload for staff members, as fluctuations in this workload are reduced significantly

The application of Approximate Dynamic Programming techniques

Koole, G.M. [Promotor]Bhulai, S. [Copromotor