Solution methods for controlled queueing networks

In this dissertation we look at a controlled queueing network where a controller routes the incoming arrivals to parallel queues using state-dependent rules. Besides this general arrival there are dedicated arrivals to each queue. The dedicated arrivals can only be served by their designated server, hence there is no routing decision involved. The goal of the controller is to find a stationary policy that will minimize the average number of customers in the system;The problem is modeled as a semi-Markov decision process and solved using techniques from the theory of Markov decision processes. We develop an efficient policy iteration based methodology which performs better than the value iteration method which is widely thought of as the best method to use for large-scale problems. The novelty in our approach is to use iterative methods in solving the system of linear equations, and also take advantage of the sparsity of matrices. The methodology could be used for other problems that are similar in nature. Using this methodology we solve much larger problems than reported in the literature. We also look at how several heuristic methods perform on our problem. No heuristic method is suitable to use for all instances. In general, however, these heuristic methods offer quick and reasonable solutions to very large problems

Scheduling in service systems with impatient customers and insights on mass-casualty triage

In this dissertation, we study a resource allocation problem for a service system with customers who may differ in their tolerance for wait. In this system, if a customer waits longer than her tolerance (which we call the lifetime), then she leaves the system without receiving any service. On the other hand, if a customer enters service, a random reward is earned. The objective is to obtain dynamic scheduling policies that maximize the total (or average) expected reward. Our motivation for this study is a resource allocation problem commonly observed in the aftermath of mass-casualty events, where the medical resources are overwhelmed with the nearly simultaneous arrivals of large numbers of patients. In such situations, the common practice is to triage the casualties, i.e., categorize them into priority groups, based on only the type of the injuries. In this dissertation, we study the benefits of taking into account the number of patients, the available resources, and the changes that occur with time while giving prioritization decisions during a mass-casualty event. We formulate the problem as a priority assignment problem for a queueing system with multiple types of impatient customers (patients). In our base model, there is a fixed number of customers to be cleared and there are no future arrivals. For this clearing problem, we consider the multi-server case under the assumption that service times are identically distributed, and when we relax this assumption, we restrict our attention to a single server. In our analysis, we use sample path methods and stochastic dynamic programming to characterize structures of good scheduling policies. For example, we provide analytical results that give sufficient conditions for the optimality of state independent optimal policies and that show when and how the optimal policy might depend on the state of the system. Based on these partial characterizations of the optimal policy, we develop state-dependent and state-independent heuristic policies, and test their performance by a numerical study. Finally, we extend our base model by considering arrivals after time zero and allowing customers to change their types while waiting in the queue