Approximate Dynamic Programming: Health Care Applications

This dissertation considers different approximate solutions to Markov decision problems formulated within the dynamic programming framework in two health care applications. Dynamic formulations are appropriate for problems which require optimization over time and a variety of settings for different scenarios and policies. This is similar to the situation in a lot of health care applications for which because of the curses of dimensionality, exact solutions do not always exist. Thus, approximate analysis to find near optimal solutions are motivated. To check the quality of approximation, additional evidence such as boundaries, consistency analysis, or asymptotic behavior evaluation are required. Emergency vehicle management and dose-finding clinical trials are the two heath care applications considered here in order to investigate dynamic formulations, approximate solutions, and solution quality assessments. The dynamic programming formulation for real-time ambulance dispatching and relocation policies, response-adaptive dose-finding clinical trial, and optimal stopping of adaptive clinical trials is presented. Approximate solutions are derived by multiple methods such as basis function regression, one-step look-ahead policy, simulation-based gridding algorithm, and diffusion approximation. Finally, some boundaries to assess the optimality gap and a proof of consistency for approximate solutions are presented to ensure the quality of approximation

Dynamic coordination in fleet management systems: Toward smart cyber fleets

Fleet management systems are commonly used to coordinate mobility and delivery services in a broad variety of domains. However, their traditional top-down control architecture becomes a bottleneck in open and dynamic environments, where scalability, proactiveness, and autonomy are becoming key factors for their success. Here, the authors present an abstract event-based architecture for fleet management systems that supports tailoring dynamic control regimes for coordinating fleet vehicles, and illustrate it for the case of medical emergency management. Then, they go one step ahead in the transition toward automatic or driverless fleets, by conceiving fleet management systems in terms of cyber-physical systems, and putting forward the notion of cyber fleets. © 2014 IEEE.This work has been partially supported by the Spanish Ministry of Economy and Competitiveness through the projects “Agreement Technologies” (grant CSD2007-0022; CONSOLIDER-INGENIO 2010), “intelligent Human-Agent Societies” (grant TIN2012-36586-C03-02), and “Smart Delivery” (grant RTC-2014-1850-4).Peer Reviewe

Coordination of Mobile Mules via Facility Location Strategies

In this paper, we study the problem of wireless sensor network (WSN) maintenance using mobile entities called mules. The mules are deployed in the area of the WSN in such a way that would minimize the time it takes them to reach a failed sensor and fix it. The mules must constantly optimize their collective deployment to account for occupied mules. The objective is to define the optimal deployment and task allocation strategy for the mules, so that the sensors' downtime and the mules' traveling distance are minimized. Our solutions are inspired by research in the field of computational geometry and the design of our algorithms is based on state of the art approximation algorithms for the classical problem of facility location. Our empirical results demonstrate how cooperation enhances the team's performance, and indicate that a combination of k-Median based deployment with closest-available task allocation provides the best results in terms of minimizing the sensors' downtime but is inefficient in terms of the mules' travel distance. A k-Centroid based deployment produces good results in both criteria.Comment: 12 pages, 6 figures, conferenc

Towards smart open dynamic fleets

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-33509-4_32Nowadays, vehicles of modern fleets are endowed with advanced devices that allow the operators of a control center to have global knowledge about fleet status, including existing incidents. Fleet management systems support real-time decision making at the control center so as to maximize fleet perform‐ ance. In this paper, setting out from our experience in dynamic coordination of fleet management systems, we focus on fleets that are open, dynamic and highly autonomous. Furthermore, we propose how to cope with the scalability problem as the number of vehicles grows. We present our proposed architecture for open fleet management systems and use the case of taxi services as example of our proposal.Work partially supported by Spanish Government through the projects iHAS (grant TIN2012-36586-C03) and SURF (grant TIN2015-65515-C4-X-R), the Autonomous Region of Madrid through grant S2013/ICE-3019 (“MOSI-AGIL-CM”, cofunded by EU Structural Funds FSE and FEDER) and URJC-Santander (30VCPIGI15).Billhardt, H.; Fernández, A.; Lujak, M.; Ossowski, S.; Julian Inglada, VJ.; Paz, JFD.; Hernández, JZ. (2016). Towards smart open dynamic fleets. En Multi-Agent Systems and Agreement Technologies. Springer. 410-424. https://doi.org/10.1007/978-3-319-33509-4_32S41042

Comparison of Emergency Medical Services Delivery Performance using Maximal Covering Location and Gradual Cover Location Problems

Ambulance location is one of the critical factors that determine the efficiency of emergency medical services delivery. Maximal Covering Location Problem is one of the widely used ambulance location models. However, its coverage function is considered unrealistic because of its ability to abruptly change from fully covered to uncovered. On the contrary, Gradual Cover Location Problem coverage is considered more realistic compared to Maximal Cover Location Problem because the coverage decreases over distance. This paper examines the delivery of Emergency Medical Services under the models of Maximal Covering Location Problem and Gradual Cover Location Problem. The results show that the latter model is superior, especially when the Maximal Covering Location Problem has been deemed fully covered

A taxonomy for emergency service station location problem

The emergency service station (ESS) location problem has been widely studied in the literature since 1970s. There has been a growing interest in the subject especially after 1990s. Various models with different objective functions and constraints have been proposed in the academic literature and efficient solution techniques have been developed to provide good solutions in reasonable times. However, there is not any study that systematically classifies different problem types and methodologies to address them. This paper presents a taxonomic framework for the ESS location problem using an operations research perspective. In this framework, we basically consider the type of the emergency, the objective function, constraints, model assumptions, modeling, and solution techniques. We also analyze a variety of papers related to the literature in order to demonstrate the effectiveness of the taxonomy and to get insights for possible research directions

A generic method to develop simulation models for ambulance systems

In this paper, we address the question of generic simulation models and their role in improving emergency care around the world. After reviewing the development of ambulance models and the contexts in which they have been applied, we report the construction of a reusable model for ambulance systems. Further, we describe the associated parameters, data sources, and performance measures, and report on the collection of information, as well as the use of optimisation to configure the service to best effect. Having developed the model, we have validated it using real data from the emergency medical system in a Brazilian city, Belo Horizonte. To illustrate the benefits of standardisation and reusability we apply the model to a UK context by exploring how different rules of engagement would change the performance of the system. Finally, we consider the impact that one might observe if such rules were adopted by the Brazilian system