Approximate Dynamic Programming: Health Care Applications

This dissertation considers different approximate solutions to Markov decision problems formulated within the dynamic programming framework in two health care applications. Dynamic formulations are appropriate for problems which require optimization over time and a variety of settings for different scenarios and policies. This is similar to the situation in a lot of health care applications for which because of the curses of dimensionality, exact solutions do not always exist. Thus, approximate analysis to find near optimal solutions are motivated. To check the quality of approximation, additional evidence such as boundaries, consistency analysis, or asymptotic behavior evaluation are required. Emergency vehicle management and dose-finding clinical trials are the two heath care applications considered here in order to investigate dynamic formulations, approximate solutions, and solution quality assessments. The dynamic programming formulation for real-time ambulance dispatching and relocation policies, response-adaptive dose-finding clinical trial, and optimal stopping of adaptive clinical trials is presented. Approximate solutions are derived by multiple methods such as basis function regression, one-step look-ahead policy, simulation-based gridding algorithm, and diffusion approximation. Finally, some boundaries to assess the optimality gap and a proof of consistency for approximate solutions are presented to ensure the quality of approximation

Strategic Location and Dispatch Management of Assets in a Military Medical Evacuation Enterprise

This dissertation considers the importance of optimizing deployed military medical evacuation (MEDEVAC) systems and utilizes operations research techniques to develop models that allow military medical planners to analyze different strategies regarding the management of MEDEVAC assets in a deployed environment. For optimization models relating to selected subproblems of the MEDEVAC enterprise, the work herein leverages integer programming, multi-objective optimization, Markov decision processes, approximate dynamic programming, and machine learning, as appropriate, to identify relevant insights for aerial MEDEVAC operations

A Chance Constrained Programming Model for Reliable Emergency Vehicles Relocation Problem

AbstractEmergency vehicles relocation is one mechanism of increasing preparedness for potential emergencies. This paper addresses the problem of designing reliable emergency vehicles relocation system. Under this respect, we extend the DYNACO model with chance-constrained programming framework for the optimal redeployment of emergency vehicles. The model deals with the availability of emergency vehicles by approximate hypercube. In addition, other random elements including travel time and emergency demand are taken into account in the model. Solution procedure based on genetic algorithm and Monte-Carlo simulation is developed to solve the stochastic model. Computational experiences are reported to illustrate the performance and the effectiveness of the proposed solution

Coordination of Mobile Mules via Facility Location Strategies

In this paper, we study the problem of wireless sensor network (WSN) maintenance using mobile entities called mules. The mules are deployed in the area of the WSN in such a way that would minimize the time it takes them to reach a failed sensor and fix it. The mules must constantly optimize their collective deployment to account for occupied mules. The objective is to define the optimal deployment and task allocation strategy for the mules, so that the sensors' downtime and the mules' traveling distance are minimized. Our solutions are inspired by research in the field of computational geometry and the design of our algorithms is based on state of the art approximation algorithms for the classical problem of facility location. Our empirical results demonstrate how cooperation enhances the team's performance, and indicate that a combination of k-Median based deployment with closest-available task allocation provides the best results in terms of minimizing the sensors' downtime but is inefficient in terms of the mules' travel distance. A k-Centroid based deployment produces good results in both criteria.Comment: 12 pages, 6 figures, conferenc

The Impact of Threat Levels at the Casualty Collection Point on Military Medical Evacuation System Performance

One of the primary duties of the Military Health System is to provide effective and efficient medical evacuation (MEDEVAC) to injured battlefield personnel. To accomplish this, military medical planners seek to develop high-quality dispatching policies that dictate how deployed MEDEVAC assets are utilized throughout combat operations. This thesis seeks to determine dispatching policies that improve the performance of the MEDEVAC system. A discounted, infinite-horizon continuous-time Markov decision process (MDP) model is developed to examine the MEDEVAC dispatching problem. The model incorporates problem features that are not considered under the current dispatching policy (e.g., myopic policy), which tasks the closest-available MEDEVAC unit to service an incoming request. More specifically, the MDP model explicitly accounts for admission control, precedence level of calls, different asset types (e.g., Army versus Air Force helicopters), and threat level at casualty collection points. An approximate dynamic programming (ADP) algorithm is developed within an approximate policy iteration algorithmic framework that leverages kernel regression to approximate the state value function. The ADP algorithm is used to develop high-quality solutions for large scale problems that cannot be solved to optimality due to the curse of dimensionality. We develop a notional scenario based on combat operations in southern Afghanistan to investigate model performance, which is measured in terms of casualty survivability. The results indicate that significant improvement in MEDEVAC system performance can be obtained by utilizing either the MDP or ADP generated policies. These results inform the development and implementation of tactics, techniques and procedures for the military medical planning community

Comparison of Emergency Medical Services Delivery Performance using Maximal Covering Location and Gradual Cover Location Problems

Ambulance location is one of the critical factors that determine the efficiency of emergency medical services delivery. Maximal Covering Location Problem is one of the widely used ambulance location models. However, its coverage function is considered unrealistic because of its ability to abruptly change from fully covered to uncovered. On the contrary, Gradual Cover Location Problem coverage is considered more realistic compared to Maximal Cover Location Problem because the coverage decreases over distance. This paper examines the delivery of Emergency Medical Services under the models of Maximal Covering Location Problem and Gradual Cover Location Problem. The results show that the latter model is superior, especially when the Maximal Covering Location Problem has been deemed fully covered

Efficiency and fairness in ambulance planning

Mei, R.D. van der [Promotor]Bhulai, S. [Promotor