Routing in multi-class queueing networks

PhD ThesisWe consider the problem of routing (incorporating local scheduling) in a distributed network. Dedicated jobs arrive directly at their specified station for processing. The choice of station for generic jobs is open. Each job class has an associated holding cost rate. We aim to develop routing policies to minimise the long-run average holding cost rate. We first consider the class of static policies. Dacre, Glazebrook and Nifio-Mora (1999) developed an approach to the formulation of static routing policies, in which the work at each station is scheduled optimally, using the achievable region approach. The achievable region approach attempts to solve stochastic optimisation problems by characterising the space of all possible performances and optimising the performance objective over this space. Optimal local scheduling takes the form of a priority policy. Such static routing policies distribute the generic traffic to the stations via a simple Bernoulli routing mechanism. We provide an overview of the achievements made in following this approach to static routing. In the course of this discussion we expand upon the study of Becker et al. (2000) in which they considered routing to a collection of stations specialised in processing certain job classes and we consider how the composition of the available stations affects the system performance for this particular problem. We conclude our examination of static routing policies with an investigation into a network design problem in which the number of stations available for processing remains to be determined. The second class of policies of interest is the class of dynamic policies. General DP theory asserts the existence of a deterministic, stationary and Markov optimal dynamic policy. However, a full DP solution may be unobtainable and theoretical difficulties posed by simple routing problems suggest that a closed form optimal policy may not be available. This motivates a requirement for good heuristic policies. We consider two approaches to the development of dynamic routing heuristics. We develop an idea proposed, in the context of simple single class systems, by Krishnan (1987) by applying a single policy improvement step to some given static policy. The resulting dynamic policy is shown to be of simple structure and easily computable. We include an investigation into the comparative performance of the dynamic policy with a number of competitor policies and of the performance of the heuristic as the number of stations in the network changes. In our second approach the generic traffic may only access processing when the station has been cleared of all (higher priority) jobs and can be considered as background work. We deploy a prescription of Whittle (1988) developed for RBPs to develop a suitable approach to station indexation. Taking an approximative approach to Whittle's proposal results in a very simple form of index policy for routing the generic traffic. We investigate the closeness to optimality of the index policy and compare the performance of both of the dynamic routing policies developed here

Adaptive Matching for Expert Systems with Uncertain Task Types

A matching in a two-sided market often incurs an externality: a matched resource may become unavailable to the other side of the market, at least for a while. This is especially an issue in online platforms involving human experts as the expert resources are often scarce. The efficient utilization of experts in these platforms is made challenging by the fact that the information available about the parties involved is usually limited. To address this challenge, we develop a model of a task-expert matching system where a task is matched to an expert using not only the prior information about the task but also the feedback obtained from the past matches. In our model the tasks arrive online while the experts are fixed and constrained by a finite service capacity. For this model, we characterize the maximum task resolution throughput a platform can achieve. We show that the natural greedy approaches where each expert is assigned a task most suitable to her skill is suboptimal, as it does not internalize the above externality. We develop a throughput optimal backpressure algorithm which does so by accounting for the `congestion' among different task types. Finally, we validate our model and confirm our theoretical findings with data-driven simulations via logs of Math.StackExchange, a StackOverflow forum dedicated to mathematics.Comment: A part of it presented at Allerton Conference 2017, 18 page

Teaching Multiple Concepts to Forgetful Learners

How can we help a forgetful learner learn multiple concepts within a limited time frame? While there have been extensive studies in designing optimal schedules for teaching a single concept given a learner's memory model, existing approaches for teaching multiple concepts are typically based on heuristic scheduling techniques without theoretical guarantees. In this paper, we look at the problem from the perspective of discrete optimization and introduce a novel algorithmic framework for teaching multiple concepts with strong performance guarantees. Our framework is both generic, allowing the design of teaching schedules for different memory models, and also interactive, allowing the teacher to adapt the schedule to the underlying forgetting mechanisms of the learner. Furthermore, for a well-known memory model, we are able to identify a regime of model parameters where our framework is guaranteed to achieve high performance. We perform extensive evaluations using simulations along with real user studies in two concrete applications: (i) an educational app for online vocabulary teaching; and (ii) an app for teaching novices how to recognize animal species from images. Our results demonstrate the effectiveness of our algorithm compared to popular heuristic approaches

Marginal Productivity Indices and Linear Programming Relaxations for Dynamic Resource Allocation in Queueing Systems

Many problems concerning resource management in modern communication systems can be simplified to queueing models under Markovian assumptions. The computation of the optimal policy is however often hindered by the curse of dimensionality especially for models that support multiple traffic or job classes. The research focus naturally turns to computationally efficient bounds and high performance heuristics. In this thesis, we apply the indexability theory to the study of admission control of a single server queue and to the buffer sharing problem for a multi-class queueing system. Our main contributions are the following: we derive the Marginal Productivity Index (MPI) and give a sufficient indexability condition for the admission control model by viewing the buffer as the resource; we construct hierarchical Linear Programming (LP) relaxations for the buffer sharing problem and propose an MPI based heuristic with its performance evaluated by discrete event simulation. In our study, the admission control model is used as the building block for the MPI heuristic deployed for the buffer sharing problem. Our condition for indexability only requires that the reward function is concavelike. We also give the explicit non-recursive expression for the MPI calculation. We compare with the previous result of the indexability condition and the MPI for the admission control model that penalizes the rejection action. The study of hierarchical LP relaxations for the buffer sharing problem is based on the exact but intractable LP formulation of the continuous-time Markov Decision Process (MDP). The number of hierarchy levels is equal to the number of job classes. The last one in the hierarchy is exact and corresponds to the exponentially sized LP formulation of the MDP. The first order relaxation is obtained by relaxing the constraint that no buffer overflow may occur in any sample path to the constraint that the average buffer utilization does not exceed the available capacity. Based on the Lagrangian decomposition of the first order relaxation, we propose a heuristic policy based on the concept of MPI. Each one of the decomposed subproblems corresponds to the admission control model we described above. The link to the decomposed sub-problems is the Lagrangian multiplier for the relaxed buffer size constraint in the first order relaxation. Our simulation study indicates the near optimal performance of the heuristic in the (randomly generated) instances investigated

Estimation and Control of Dynamical Systems with Applications to Multi-Processor Systems

System and control theory is playing an increasingly important role in the design and analysis of computing systems. This thesis investigates a set of estimation and control problems that are driven by new challenges presented by next-generation Multi-Processor Systems on Chips (MPSoCs). Specifically, we consider problems related to state norm estimation, state estimation for positive systems, sensor selection, and nonlinear output tracking. Although these problems are motivated by applications to multi-processor systems, the corresponding theory and algorithms are developed for general dynamical systems. We first study state norm estimation for linear systems with unknown inputs. Specifically, we consider a formulation where the unknown inputs and initial condition of the system are bounded in magnitude, and the objective is to construct an unknown input norm-observer which estimates an upper bound for the norm of the states. This class of problems is motivated by the need to estimate the maximum temperature across a multi-core processor, based on a given model of the thermal dynamics. In order to characterize the existence of the norm observer, we propose a notion of bounded-input-bounded-output-bounded-state (BIBOBS) stability; this concept supplements various system properties, including bounded-input-bounded-output (BIBO) stability, bounded-input-bounded-state (BIBS) stability, and input-output-to-state stability (IOSS).We provide necessary and sufficient conditions on the system matrices under which a linear system is BIBOBS stable, and show that the set of modes of the system with magnitude 1 plays a key role. A construction for the unknown input norm-observer follows as a byproduct. Then we investigate the state estimation problem for positive linear systems with unknown inputs. This problem is also motivated by the need to monitor the temperature of a multi-processor system and the property of positivity arises due to the physical nature of the thermal model. We extend the concept of strong observability to positive systems and as a negative result, we show that the additional information on positivity does not help in state estimation. Since the states of the system are always positive, negative state estimates are meaningless and the positivity of the observers themselves may be desirable in certain applications. Moreover, positive systems possess certain desired robustness properties. Thus, for positive systems where state estimation with unknown inputs is possible, we provide a linear programming based design procedure for delayed positive observers. Next we consider the problem of selecting an optimal set of sensors to estimate the states of linear dynamical systems; in the context of multi-core processors, this problem arises due to the need to place thermal sensors in order to perform state estimation. The goal is to choose (at design-time) a subset of sensors (satisfying certain budget constraints) from a given set in order to minimize the trace of the steady state a priori or a posteriori error covariance produced by a Kalman filter. We show that the a priori and a posteriori error covariance-based sensor selection problems are both NP-hard, even under the additional assumption that the system is stable. We then provide bounds on the worst-case performance of sensor selection algorithms based on the system dynamics, and show that certain greedy algorithms are optimal for two classes of systems. However, as a negative result, we show that certain typical objective functions are not submodular or supermodular in general. While this makes it difficult to evaluate the performance of greedy algorithms for sensor selection (outside of certain special cases), we show via simulations that these greedy algorithms perform well in practice. Finally, we study the output tracking problem for nonlinear systems with constraints. This class of problems arises due to the need to optimize the energy consumption of the CPU-GPU subsystem in multi-processor systems while satisfying certain Quality of Service (QoS) requirements. In order for the system output to track a class of bounded reference signals with limited online computational resources, we propose a sampling-based explicit nonlinear model predictive control (ENMPC) approach, where only a bound on the admissible references is known to the designer a priori. The basic idea of sampling-based ENMPC is to sample the state and reference signal space using deterministic sampling and construct the ENMPC by using regression methods. The proposed approach guarantees feasibility and stability for all admissible references and ensures asymptotic convergence to the set-point. Furthermore, robustness through the use of an ancillary controller is added to the nominal ENMPC for a class of nonlinear systems with additive disturbances, where the robust controller keeps the system output close to the desired nominal trajectory

Maintenance scheduling for modular systems-models and algorithms

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 185-188).Maintenance scheduling is an integral part of many complex systems. For instance, without effective maintenance scheduling, the combined effects of preventative and corrective maintenance can have severe impacts on the availability of those systems. Based on current Air Force trends including maintenance manpower, dispersed aircraft basing, and increased complexity, there has been a renewed focus on preventative maintenance. To address these concerns, this thesis develops two models for preventative maintenance scheduling for complex systems, the first of interest in the system concept development and design phase, and the second of interest during operations. Both models are highly complex and intractable to solve in their original forms. For the first model, we develop approximation algorithms that yield high quality and easily implementable solutions. To address the second model, we propose a decomposition strategy that produces submodels that can be solved via existing algorithms or via specialized algorithms we develop. While much of the literature has examined stochastically failing systems, preventative maintenance of usage limited systems has received less attention. Of particular interest is the design of modular systems whose components must be repaired/replaced to prevent a failure. By making cost tradeoffs early in development, program managers, designers, engineers, and test conductors can better balance the up front costs associated with system design and testing with the long term cost of maintenance. To facilitate such a tradeoff, the Modular Maintenance Scheduling Problem provides a framework for design teams to evaluate different design and operations concepts and then evaluate the long term costs. While the general Modular Maintenance Scheduling Problem does not require maintenance schedules with specific structure, operational considerations push us to consider cyclic schedules in which components are maintained at a fixed frequency. In order to efficiently find cyclic schedules, we propose the Cycle Rounding algorithm, which has an approximation guarantee of 2, and a family of Shifted Power-of-Two algorithms, which have an approximation guarantee of 1/ ln(2) ~ 1.4427. Computational results indicate that both algorithms perform much better than their associated performance guarantees providing solutions within 15%-25% of a lower bound. Once a modular system has moved into operations, manpower and transportation scheduling become important considerations when developing maintenance schedules. To address the operations phase, we develop the Modular Maintenance and System Assembly Model to balance the tradeoffs between inventory, maintenance capacity, and transportation resources. This model explicitly captures the risk-pooling effects of a central repair facility while also modeling the interaction between repair actions at such a facility. The full model is intractable for all but the smallest instances. Accordingly, we decompose the problem into two parts, the system assembly portion and module repair portion. Finally, we tie together the Modular Maintenance and System Assembly Model with key concepts from the Modular Maintenance Scheduling Problem to propose an integrated methodology for design and operation.by Eric Jack Zarybnisky.Ph.D

Appointment Scheduling in Health Care

We propose two complementary approaches to the scheduling of outpatient appointments. The first approach is to dynamically assign appointment times depending on the continuously updated patient schedule. The second approach is to statically design the system by either limiting the appointment backlog or regulating the demand rate through controlling the panel size, i.e. the population receiving the medical service. For the first approach - dynamic scheduling, we start with the assumption that patients come from a single class with homogeneous no-show and cancellation behaviors. We develop a Markov decision process model and propose easily implementable heuristic dynamic policies. In a simulation study that considers a model clinic, which is created using data from practice, we find that the proposed heuristics outperform all the other benchmark policies, particularly when the patient load is high compared with the regular capacity. We then extend our model to consider the scheduling of patients from multiple classes. In this model, different classes of patients are assumed to have different probability distributions for their no-show and cancellation behaviors. As in the single-class case, we develop heuristic dynamic policies. Simulation results suggest that our proposed heuristics perform well when the regular capacity is small. For the second approach - static design, we model the appointment backlog as a single-server queue where new appointments join the backlog from the back of the queue. Motivated by empirical findings, we assume that patients do not show up for their appointments with probabilities that increase with their waiting times before receiving service. We first study the model under the assumption of exponential service times. We characterize the optimal appointment backlog size and the optimal demand rate that maximize the system throughput and investigate how they change with other system parameters. Then we consider a special case where patients' no-show probabilities follow a specific parametric form. Under this special case, we obtain a simple closed-form expression for the optimal demand rate if we do not put a limit on the appointment backlog. Finally we conduct extensive numerical studies to investigate the situation where the service times are deterministic. The numerical results suggest that the insights generated in our analytical study by assuming exponential service times also hold for the situation with deterministic service times