METFAC-2.1 User's Guide

Technical ReportPostprint (author’s final draft

Transient Reward Approximation for Continuous-Time Markov Chains

We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power grids, of computer virus vulnerability, and in the study of crowd dynamics. We use abstraction techniques together with novel algorithms for the computation of bounds on the expected final and accumulated rewards in continuous-time Markov decision processes (CTMDPs). These ingredients are combined in a partly symbolic and partly explicit (symblicit) analysis approach. In particular, we circumvent the use of multi-terminal decision diagrams, because the latter do not work well if facing a large number of different rates. We demonstrate the practical applicability and efficiency of the approach on two case studies.Comment: Accepted for publication in IEEE Transactions on Reliabilit

Stochastic graph models with phase type distributed edge weights

Most stochastic shortest path problems include an assumption of independent weights at edges. For many practical problems, however, this assumption is often violated indicating an increased number of applications with stochastic graphs where edge weights follow a distribution that has a possible correlation with weights at adjacent edges. Real-world information in conjunction with existing dependencies in networks can significantly contribute to the computation of the optimal solution to stochastic shortest path problems. For example, the knowledge of a congestion arising on the current road results in a better traveler's choice of an alternative route. Markov dependability models describing the correlation in the length of availability and unavailability intervals of technical components could support optimal decisions to avoid high maintenance costs. In this thesis, an innovative model class for stochastic graphs with correlated weights at the edges is introduced. In the developed model edge weights are modeled by PH distributions and correlations between them can be encoded using transfer matrices for PH distributions of adjacent edge weights. Stochastic graph models including correlations are important to describe many practical situations where the knowledge about system parameters like traveling times and costs is incomplete or changes over time. Based on PH-Graphs efficient solution methods for Stochastic Shortest Path Problems with correlations can be developed. Competing paths from origin to destination in a PH-Graph can be interpreted as CTMDPs. Optimal solutions to different shortest path problems can be obtained from finding an optimal policy in a CTMDP. For example, the problem of finding the reliable shortest path to maximize the probability of arriving on time under realistic assumptions can be efficiently treated. Formulations of shortest path problems with correlations as well as solution methods from the CTMDP field are presented. We address the parameterization of PH-Graphs based on measured data from simulated systems. Fitting methods for parameterization of transfer matrices are adopted from MAP fitting approaches. Also similarity transformations for order 2 acyclic PHDs in composition are considered. Our experiments and examples show that correlation has a significant impact on shortest paths in stochastic weighted networks and that our solution methods are effective and usable in online decision senarios

List of requirements on formalisms and selection of appropriate tools

This deliverable reports on the activities for the set-up of the modelling environments for the evaluation activities of WP5. To this objective, it reports on the identified modelling peculiarities of the electric power infrastructure and the information infrastructures and of their interdependencies, recalls the tools that have been considered and concentrates on the tools that are, and will be, used in the project: DrawNET, DEEM and EPSys which have been developed before and during the project by the partners, and M\uf6bius and PRISM, developed respectively at the University of Illinois at Urbana Champaign and at the University of Birmingham (and recently at the University of Oxford)

Numerical iterative methods for Markovian dependability and performability models: new results and a comparison

In this paper we deal with iterative numerical methods to solve linear systems arising in continuous-time Markov chain (CTMC) models. We develop an algorithm to dynamically tune the relaxation parameter of the successive over-relaxation method. We give a sufficient condition for the Gauss-Seidel method to converge when computing the steady-state probability vector of a finite irreducible CTMC, an a suffient condition for the Generalized Minimal Residual projection method not to converge to the trivial solution 0 when computing that vector. Finally, we compare several splitting-based iterative methods an a variant of the Generalized Minimal Residual projection method.Postprint (published version

Transient Reward Approximation for Continuous-Time Markov Chains

Abstract We are interested in the analysis of very large continuoustime Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e. g., of computer network performability analysis, of power grids, of computer virus vulnerability, and in the study of crowd dynamics. We use abstraction techniques together with novel algorithms for the computation of bounds on the expected final and accumulated rewards in continuous-time Markov decision processes (CTMDPs). These ingredients are combined in a partly symbolic and partly explicit (symblicit) analysis approach. In particular, we circumvent the use of multi-terminal decision diagrams, because the latter do not work well if facing a large number of different rates. We demonstrate the practical applicability and efficiency of the approach on two case studies

Multivariate phase type distributions - Applications and parameter estimation

Den bedst kendte univariate sandsynlighedsfordeling er normalfordelingen. Den er grundigt beskrevet i litteraturen inden for et bredt felt af anvendelsesområder. I de tilfælde, hvor det ikke er meningsfuldt at anvende normalfordelingen, findes alternative sandsynlighedsfordelinger som alle er godt beskrevet; mange af disse tilhører klassen af fasetypefordelinger. Fasetypefordelinger har adskillige fordele. De er alsidige forstået på den måde, at de kan benyttes til at tilnærme en vilkårlig sandsynlighedsfordeling defineret på den positive reelle akse. Der eksisterer generelle probabilistiske resultater for hele klassen af fasetypefordelinger, hvilket bidrager til anvendelsen af forskellige estimeringsmetoder på enten klassen af fasetypefordelinger eller dens delklasser. Disse egenskaber gør klassen af fasetypefordelinger til et interessant alternativ til normalfordelingen.Når det kommer til multivariate problemer, så er den multivariate normalfordeling den eneste generelle fordeling, der tillader parameterestimering og statistisk inferens. Desværre er kendskabet til egenskaberne af den multivariate fasetypefordeling stærk begrænset. Resultaterne for parameterestimering og inferensteori for den univariate fasetypefordeling indikerer et potentiale for lignende gode resultater for klassen af multivariate fasetypefordelinger. Mit ph.d.-studium var en del afWork Package 3 i UNITE-projektet. UNITEprojektet arbejder mod det overordnede mål at forbedre kvaliteten af beslutningsgrundlaget for projekter. Dette gøres ved at reducere systematisk model bias og ved at beskrive og reducere model usikkerheder generelt. Forskning har vist, at afvigelsen fra omkostningsestimater for infrastrukturprojekter tydeligvis ikke er normaltfordelt men i stedet hælder mod budgetoverskridelser. Denne skævhed kan beskrives med fasetypefordelinger. Cost-benefit-analyser bruges til at evaluere potentielle fremtidige projekter og til at udvikle pålidelige omkostningsvurderinger. Successiv Princippet er en gruppebaseret analysemetode, der primært bruges til at prædiktere omkostninger og varighed af mellem til store projekter. Vi mener, at den matematiske modellering, der ligger til grund for Successiv Princippet, kan forbedres. Vi foreslår derfor en ny tilgang til modellering af den samlede varighed af et projekt ved hjælp af univariate fasetypefordelinger. Den matematiske model er dernæst udvidet til også at beskrive korrelationen mellem projektvarighed og omkostninger nu baseret på bivariate fasetypefordelinger. Vores model kan anvendes til at forbedre estimater for varighed og omkostninger, og derved hjælpe projekters beslutningstagere til at træffe en optimal beslutning.Det arbejde, jeg har udført som en del af mit ph.d.-studium, sigtede efter at belyse klassen af multivariate fasetypefordelinger. Denne afhandling indeholder analytiske og numeriske resultater for parameterestimering og inferensteori for en gruppe af multivariate fasetypefordelinger. Resultaterne kan betragtes som et første skridt i retning af en mere tilbundsgående forståelse af multivariate fasetypefordelinger. Vi er imidlertid langt fra at have afdækket det fulde potentiale af generelle fasetypefordelinger. En dybere forståelse af multivariate fasetypefordelinger vil åbne op for et bredt felt af anvendelsesområder.Afhandlingen består af en opsummerende rapport og to videnskabelige artikler. Det bagvedliggende arbejde var udført i perioden 2010 til 2014.The best known univariate probability distribution is the normal distribution. It is used throughout the literature in a broad field of applications. In cases where it is not sensible to use the normal distribution alternative distributions are at hand and well understood, many of these belonging to the class of phase type distributions. Phase type distributions have several advantages. They are versatile in the sense that they can be used to approximate any given probability distribution on the positive reals. There exist general probabilistic results for the entire class of phase type distributions, allowing for different estimation methods for the whole class or subclasses of phase type distributions. These attributes make this class of distributions an interesting alternative to the normal distribution. When facing multivariate problems, the only general distribution that allows for estimation and statistical inference, is the multivariate normal distribution. Unfortunately only little is known about the general class of multivariate phase type distribution. Considering the results concerning parameter estimation and inference theory of univariate phase type distributions, the class of multivariate phase type distributions shows potential for similar great results.My PhD studies were part of the the work package 3 of the UNITE project. The overall goal of the UNITE project is to improve the decision support prior to deciding on a project by reducing systematic model bias and by quantifying and reducing model uncertainties.Research has shown that the errors on cost estimates for infrastructure projects clearly do not follow a normal distribution but is skewed towards cost overruns. This skewness can be described using phase type distributions. Cost benefit analysis assesses potential future projects and depend on reliable cost estimates. The Successive Principle is a group analysis method primarily used for analyzing medium to large projects in relation to cost or duration. We believe that the mathematical modeling used in the Successive Principle can be improved. We suggested a novel approach for modeling the total duration of a project using a univariate phase type distribution. The model is then extended to catch the correlation between duration and cost estimates using a bivariate phase type distribution. The use of our model can improve estimates for duration and costs and therefore help project management to make the optimal decisions. The work conducted during my PhD studies aimed at shedding light on the class of multivariate phase type distributions. This thesis contains analytical and numerical results for parameter estimations and inference theory for a family of multivariate phase type distributions. The results can be used as a stepping stone towards understanding multivariate phase type distributions better. However, we are far from uncovering the full potential of general multivariate phase type distributions. Deeper understanding of multivariate phase type distributions will open up a broad field of research areas they can be applied to.This thesis consists of a summary report and two research papers. The work was carried out in the period 2010 - 2014

Bloody fast blood collection

This thesis consists of four parts: The first part contains an introduction, the second presents approaches for the evaluation of waiting times at blood collection sites, the third uses these to present approaches that improve waiting times at blood collection sites. The final part shows the application of two of the approaches to data from real blood collection sites, followed by the conclusions that can be drawn from this thesis. Part I: Introduction, contains two chapters. Chapter 1 introduces the context for this thesis: blood banks in general, the Dutch blood bank Sanquin and blood collection sites. The chapter sketches some of the challenges faced with respect to blood collection sites. As blood donors are voluntary and non-remunerated, delays and waiting times within blood collection sites should be kept at acceptable levels. However, waiting times are currently not incorporated in staff planning or in other decisions with respect to blood collection sites. These blood collection sites will be the primary focus of this thesis. This thesis provides methods that do take waiting times into account, aiming to decrease waiting times at blood collection sites and leveling work pressure for staff members, without the need for additional staff. Chapter 2 then presents a technical methods that will be used most of the chapters in this thesis: uniformization. Uniformization can be used to transform Continuous Time Markov Chains (CTMCs) — that are very hard to analyze — into Discrete Time Markov Chains (DTMCs) — that are much easier to analyze. The chapter shows how the method works, provides an extensive overview of the literature related to the method, the (technical) intuition behind the method as well as several extensions and applications. Although not all of the extensions and applications are necessary for this thesis, it does provide an overview of one of the most valuable methods for this thesis. Part II: Evaluation, contains two chapters that propose and adapt several methods to compute waiting times and queues at blood collection sites. A blood collection site is best modeled as a time-dependent queueing network, requiring non-standard approaches. Chapter 3 considers a stationary, i.e. not time-dependent model of blood collection sites as a first step. A blood collection site consists of three main stations: Registration, Interview and Donation. All three of the stations can have their own queue. This means that even the stationary model is non-trivial for some computations. However, for the stationary model, an analytic so-called product form expression is derived. Based on this product form, two more results are shown. The first result is that the standard waiting time distributions from M|M|s queues are applicable, as if the queue is in isolation. It is then concluded that no closed form expression exist for the total waiting or delay time distribution, as the distributions of the three stations in tandem are not independent. Therefore a numerical approach is presented to compute the total delay time distribution of a collection site. All of the results are supported by numerical examples based on a Dutch blood collection site. The approach for the computation of the total delay time distribution can also be combined with the approach from Chapter 4 for an extension to a time-dependent setting. Chapter 4 shows an approach to deal with these time-dependent aspects in queueing systems, as often experienced by blood collection sites and other service systems, typically due to time-dependent arrivals and capacities. Easy and quick to use queueing expressions generally do not apply to time-dependent situations. A large number of computational papers has been written about queue length distributions for time-dependent queues, but these are mostly theoretical and based on single queues. This chapter aims to combine computational methods with more realistic time-dependent queueing networks, with an approach based on uniformization. Although uniformization is generally perceived to be too computationally prohibitive, we show that our method is very effective for practical instances, as shown with an example of a Dutch blood collection site. The objective of the results is twofold: to show that a time-dependent queueing network approach can be beneficial and to evaluate possible improvements for Dutch blood collection sites that can only be properly assessed with a time-dependent queueing method. Part III: Optimization, contains four chapters that aim to improve service levels at Sanquin. The first three chapters focus on three different methods to decrease queues at blood collection sites. Chapters 5 and 6 focus on improving the service by optimizing staff allocation to shifts and stations. Chapter 7 focuses on improving the arrival process with the same goal. Chapter 8 is focused at improving inventory management of red blood cells. Donors do not arrive to blood collection sites uniformly throughout the day, but show clear preferences for certain times of the day. However, the arrival patterns that are shown by historical data, are not used for scheduling staff members at blood collection sites. As a first significant step to shorten waiting times we can align staff capacity and shifts with walk-in arrivals. Chapter 5 aims to optimize shift scheduling for blood collection sites. The chapter proposes a two-step procedure. First, the arrival patterns and methods from queueing theory are used to determine the required number of staff members for every half hour. Second, an integer linear program is used to compute optimal shift lengths and starting times, based on the required number of staff members. The chapter is concluded with numerical experiments that show, depending on the scenario, a reduction of waiting times, a reduction of staff members or a combination of both. At a blood collection site three stations (Registration, Interview and Donation) can roughly be distinguished. Staff members at Dutch blood collection sites are often trained to work at any of these stations, but are usually allocated to one of the stations for large fractions of a shift. If staff members change their allocation this is based on an ad hoc decision. Chapter 6 aims to take advantage of this mostly unused allocation flexibility to reduce queues at blood collection sites. As a collection site is a highly stochastic process, both in arrivals and services, an optimal allocation of staff members to the three stations is unknown, constantly changing and a challenge to determine. Chapter 6 provides and applies a so-called Markov Decision Process (MDP) to compute optimal staff assignments. Extensive numerical and simulation experiments show the potential reductions of queues when the reallocation algorithm would be implemented. Based on Dutch blood collection sites, reductions of 40 to 80% on the number of waiting donors seem attainable, depending on the scenario. Chapter 7 also aims to align the arrival of donors with scheduled staff, similarly to Chapter 5. Chapter 7 tries to achieve this by changing the arrivals of donors. By introducing appointments for an additional part of donors, arrivals can be redirected from the busiest times of the day to quiet times. An extended numerical queueing model with priorities is introduced for blood collection sites, as Sanquin wants to incentive donors to make appointments by prioritizing donors with appointments over donors without appointments. Appointment slots are added if the average queue drops below certain limits. The correct values for these limits, i.e. the values that plan the correct number of appointments, are then determined by binary search. Numerical results show that the method succeeds in decreasing excessive queues. However, the proposed priorities might result in unacceptably high waiting times for donors without appointments, and caution is therefore required before implementation. Although this thesis mainly focuses on blood collection sites, many more logistical challenges are present at a blood bank. One of these challenges arises from the expectation that Sanquin can supply hospitals with extensively typed red blood cell units directly from stock. Chapter 8 deals with this challenge. Currently, all units are issued according to the first-in-first-out principle, irrespective of their specific typing. These kind of issuing policies lead to shortages for rare blood units. Shortages for rare units could be avoided by keeping them in stock for longer, but this could also lead to unnecessary wastage. Therefore, to avoid both wastage and shortages, a trade-off between the age and rarity of a specific unit in stock should be made. For this purpose, we modeled the allocation of the inventory as a circulation flow problem, in which decisions about which units to issue are based on both the age and rarity of the units in stock. We evaluated the model for several settings of the input parameters. It turns out that, especially if only a few donors are typed for some combinations of antigens, shortages can be avoided by saving rare blood products. Moreover, the average issuing age remains unchanged. Part IV: Practice and Outlook concludes this thesis. The first of two chapters in this part shows the combined application of two approaches from this thesis to data from three collection sites in the Netherlands. The final chapter of this thesis presents the conclusions that can be drawn from this thesis and discusses an outlook for further research. Chapter 9 shows the combined application of the methods in Chapters 5 and 6 to three real collection sites in Dutch cities: Nijmegen, Leiden and Almelo. The collection sites in Nijmegen and Leiden are both large fixed collection sites. The collection site in Almelo is a mobile collection site. The application of each one of the methods individually reduce waiting times significantly, and the combined application of the methods reduces waiting times even further. Simultaneously, small reductions in the number of staff hours are attainable. The results from Chapter 9 summarize the main message of this thesis: waiting time for blood donors at blood collection sites can be reduced without the need for more staff members when the working times of staff members are used more effectively and efficiently, and controlling the arrival process of donors. The approaches presented in this thesis can be used for this purpose. This is not only beneficial for blood donors, but will also result in more balanced workload for staff members, as fluctuations in this workload are reduced significantly