Quasi-stationary distributions

This paper contains a survey of results related to quasi-stationary distributions, which arise in the setting of stochastic dynamical systems that eventually evanesce, and which may be useful in describing the long-term behaviour of such systems before evanescence. We are concerned mainly with continuous-time Markov chains over a finite or countably infinite state space, since these processes most often arise in applications, but will make reference to results for other processes where appropriate. Next to giving an historical account of the subject, we review the most important results on the existence and identification of quasi-stationary distributions for general Markov chains, and give special attention to birth-death processes and related models. Results on the question of whether a quasi-stationary distribution, given its existence, is indeed a good descriptor of the long-term behaviour of a system before evanescence, are reviewed as well. The paper is concluded with a summary of recent developments in numerical and approximation methods

The MDS Queue: Analysing the Latency Performance of Erasure Codes

In order to scale economically, data centers are increasingly evolving their data storage methods from the use of simple data replication to the use of more powerful erasure codes, which provide the same level of reliability as replication but at a significantly lower storage cost. In particular, it is well known that Maximum-Distance-Separable (MDS) codes, such as Reed-Solomon codes, provide the maximum storage efficiency. While the use of codes for providing improved reliability in archival storage systems, where the data is less frequently accessed (or so-called "cold data"), is well understood, the role of codes in the storage of more frequently accessed and active "hot data", where latency is the key metric, is less clear. In this paper, we study data storage systems based on MDS codes through the lens of queueing theory, and term this the "MDS queue." We analytically characterize the (average) latency performance of MDS queues, for which we present insightful scheduling policies that form upper and lower bounds to performance, and are observed to be quite tight. Extensive simulations are also provided and used to validate our theoretical analysis. We also employ the framework of the MDS queue to analyse different methods of performing so-called degraded reads (reading of partial data) in distributed data storage

Exact Solutions for M/M/c/Setup Queues

Recently multiserver queues with setup times have been extensively studied because they have applications in power-saving data centers. The most challenging model is the M/M/$c$/Setup queue where a server is turned off when it is idle and is turned on if there are some waiting jobs. Recently, Gandhi et al.~(SIGMETRICS 2013, QUESTA 2014) present the recursive renewal reward approach as a new mathematical tool to analyze the model. In this paper, we derive exact solutions for the same model using two alternative methodologies: generating function approach and matrix analytic method. The former yields several theoretical insights into the systems while the latter provides an exact recursive algorithm to calculate the joint stationary distribution and then some performance measures so as to give new application insights.Comment: Submitted for revie

The snowball effect of customer slowdown in critical many-server systems

Customer slowdown describes the phenomenon that a customer's service requirement increases with experienced delay. In healthcare settings, there is substantial empirical evidence for slowdown, particularly when a patient's delay exceeds a certain threshold. For such threshold slowdown situations, we design and analyze a many-server system that leads to a two-dimensional Markov process. Analysis of this system leads to insights into the potentially detrimental effects of slowdown, especially in heavy-traffic conditions. We quantify the consequences of underprovisioning due to neglecting slowdown, demonstrate the presence of a subtle bistable system behavior, and discuss in detail the snowball effect: A delayed customer has an increased service requirement, causing longer delays for other customers, who in turn due to slowdown might require longer service times.Comment: 23 pages, 8 figures -- version 3 fixes a typo in an equation. in Stochastic Models, 201

Simulating and analyzing order book data: The queue-reactive model

Through the analysis of a dataset of ultra high frequency order book updates, we introduce a model which accommodates the empirical properties of the full order book together with the stylized facts of lower frequency financial data. To do so, we split the time interval of interest into periods in which a well chosen reference price, typically the mid price, remains constant. Within these periods, we view the limit order book as a Markov queuing system. Indeed, we assume that the intensities of the order flows only depend on the current state of the order book. We establish the limiting behavior of this model and estimate its parameters from market data. Then, in order to design a relevant model for the whole period of interest, we use a stochastic mechanism that allows for switches from one period of constant reference price to another. Beyond enabling to reproduce accurately the behavior of market data, we show that our framework can be very useful for practitioners, notably as a market simulator or as a tool for the transaction cost analysis of complex trading algorithms

A Computational Framework for the Mixing Times in the QBD Processes with Infinitely-Many Levels

In this paper, we develop some matrix Poisson's equations satisfied by the mean and variance of the mixing time in an irreducible positive-recurrent discrete-time Markov chain with infinitely-many levels, and provide a computational framework for the solution to the matrix Poisson's equations by means of the UL-type of $RG$-factorization as well as the generalized inverses. In an important special case: the level-dependent QBD processes, we provide a detailed computation for the mean and variance of the mixing time. Based on this, we give new highlight on computation of the mixing time in the block-structured Markov chains with infinitely-many levels through the matrix-analytic method

Stability Region of a Slotted Aloha Network with K-Exponential Backoff

Stability region of random access wireless networks is known for only simple network scenarios. The main problem in this respect is due to interaction among queues. When transmission probabilities during successive transmissions change, e.g., when exponential backoff mechanism is exploited, the interactions in the network are stimulated. In this paper, we derive the stability region of a buffered slotted Aloha network with K-exponential backoff mechanism, approximately, when a finite number of nodes exist. To this end, we propose a new approach in modeling the interaction among wireless nodes. In this approach, we model the network with inter-related quasi-birth-death (QBD) processes such that at each QBD corresponding to each node, a finite number of phases consider the status of the other nodes. Then, by exploiting the available theorems on stability of QBDs, we find the stability region. We show that exponential backoff mechanism is able to increase the area of the stability region of a simple slotted Aloha network with two nodes, more than 40\%. We also show that a slotted Aloha network with exponential backoff may perform very near to ideal scheduling. The accuracy of our modeling approach is verified by simulation in different conditions.Comment: 30 pages, 6 figure

Numerical methods for queues with shared service

A queueing system is a mathematical abstraction of a situation where elements, called customers, arrive in a system and wait until they receive some kind of service. Queueing systems are omnipresent in real life. Prime examples include people waiting at a counter to be served, airplanes waiting to take off, traffic jams during rush hour etc. Queueing theory is the mathematical study of queueing phenomena. As often neither the arrival instants of the customers nor their service times are known in advance, queueing theory most often assumes that these processes are random variables. The queueing process itself is then a stochastic process and most often also a Markov process, provided a proper description of the state of the queueing process is introduced. This dissertation investigates numerical methods for a particular type of Markovian queueing systems, namely queueing systems with shared service. These queueing systems differ from traditional queueing systems in that there is simultaneous service of the head-of-line customers of all queues and in that there is no service if there are no customers in one of the queues. The absence of service whenever one of the queues is empty yields particular dynamics which are not found in traditional queueing systems. These queueing systems with shared service are not only beautiful mathematical objects in their own right, but are also motivated by an extensive range of applications. The original motivation for studying queueing systems with shared service came from a particular process in inventory management called kitting. A kitting process collects the necessary parts for an end product in a box prior to sending it to the assembly area. The parts and their inventories being the customers and queues, we get ``shared service'' as kitting cannot proceed if some parts are absent. Still in the area of inventory management, the decoupling inventory of a hybrid make-to-stock/make-to-order system exhibits shared service. The production process prior to the decoupling inventory is make-to-stock and driven by demand forecasts. In contrast, the production process after the decoupling inventory is make-to-order and driven by actual demand as items from the decoupling inventory are customised according to customer specifications. At the decoupling point, the decoupling inventory is complemented with a queue of outstanding orders. As customisation only starts when the decoupling inventory is nonempty and there is at least one order, there is again shared service. Moving to applications in telecommunications, shared service applies to energy harvesting sensor nodes. Such a sensor node scavenges energy from its environment to meet its energy expenditure or to prolong its lifetime. A rechargeable battery operates very much like a queue, customers being discretised as chunks of energy. As a sensor node requires both sensed data and energy for transmission, shared service can again be identified. In the Markovian framework, "solving" a queueing system corresponds to finding the steady-state solution of the Markov process that describes the queueing system at hand. Indeed, most performance measures of interest of the queueing system can be expressed in terms of the steady-state solution of the underlying Markov process. For a finite ergodic Markov process, the steady-state solution is the unique solution of $N-1$ balance equations complemented with the normalisation condition, $N$ being the size of the state space. For the queueing systems with shared service, the size of the state space of the Markov processes grows exponentially with the number of queues involved. Hence, even if only a moderate number of queues are considered, the size of the state space is huge. This is the state-space explosion problem. As direct solution methods for such Markov processes are computationally infeasible, this dissertation aims at exploiting structural properties of the Markov processes, as to speed up computation of the steady-state solution. The first property that can be exploited is sparsity of the generator matrix of the Markov process. Indeed, the number of events that can occur in any state --- or equivalently, the number of transitions to other states --- is far smaller than the size of the state space. This means that the generator matrix of the Markov process is mainly filled with zeroes. Iterative methods for sparse linear systems --- in particular the Krylov subspace solver GMRES --- were found to be computationally efficient for studying kitting processes only if the number of queues is limited. For more queues (or a larger state space), the methods cannot calculate the steady-state performance measures sufficiently fast. The applications related to the decoupling inventory and the energy harvesting sensor node involve only two queues. In this case, the generator matrix exhibits a homogene block-tridiagonal structure. Such Markov processes can be solved efficiently by means of matrix-geometric methods, both in the case that the process has finite size and --- even more efficiently --- in the case that it has an infinite size and a finite block size. Neither of the former exact solution methods allows for investigating systems with many queues. Therefore we developed an approximate numerical solution method, based on Maclaurin series expansions. Rather than focussing on structural properties of the Markov process for any parameter setting, the series expansion technique exploits structural properties of the Markov process when some parameter is sent to zero. For the queues with shared exponential service and the service rate sent to zero, the resulting process has a single absorbing state and the states can be ordered such that the generator matrix is upper-diagonal. In this case, the solution at zero is trivial and the calculation of the higher order terms in the series expansion around zero has a computational complexity proportional to the size of the state space. This is a case of regular perturbation of the parameter and contrasts to singular perturbation which is applied when the service times of the kitting process are phase-type distributed. For singular perturbation, the Markov process has no unique steady-state solution when the parameter is sent to zero. However, similar techniques still apply, albeit at a higher computational cost. Finally we note that the numerical series expansion technique is not limited to evaluating queues with shared service. Resembling shared queueing systems in that a Markov process with multidimensional state space is considered, it is shown that the regular series expansion technique can be applied on an epidemic model for opinion propagation in a social network. Interestingly, we find that the series expansion technique complements the usual fluid approach of the epidemic literature