An Approximate Dynamic Programming Mode for Optimal MEDEVAC Dispatching

We develop a Markov decision process (MDP) model to examine military medical evacuation (MEDEVAC) dispatch policies. To solve our MDP, we apply an approximate dynamic programming (ADP) technique. The problem of deciding which aeromedical asset to dispatch to which service request is complicated by the service locations and the priority class of each casualty event. We assume requests for MEDEVAC arrive sequentially, with the location and the priority of each casualty known upon initiation of the request. The proposed model finds a high quality dispatching policy which outperforms the traditional myopic policy of sending the nearest available unit. Utility is gained by servicing casualties based on both their priority and the actual time until a casualty arrives at a medical treatment facility (MTF). The model is solved using approximate policy iteration (API) and least squares temporal difference (LSTD). Computational examples are used to investigate dispatch policies for a scenario set in northern Syria. Results indicate that a myopic policy is not always the best policy to use for quickly dispatching MEDEVAC units, and insight is gained into the value of specific MEDEVAC locations

Learning excursion sets of vector-valued Gaussian random fields for autonomous ocean sampling

Improving and optimizing oceanographic sampling is a crucial task for marine science and maritime resource management. Faced with limited resources in understanding processes in the water-column, the combination of statistics and autonomous systems provide new opportunities for experimental design. In this work we develop efficient spatial sampling methods for characterizing regions defined by simultaneous exceedances above prescribed thresholds of several responses, with an application focus on mapping coastal ocean phenomena based on temperature and salinity measurements. Specifically, we define a design criterion based on uncertainty in the excursions of vector-valued Gaussian random fields, and derive tractable expressions for the expected integrated Bernoulli variance reduction in such a framework. We demonstrate how this criterion can be used to prioritize sampling efforts at locations that are ambiguous, making exploration more effective. We use simulations to study and compare properties of the considered approaches, followed by results from field deployments with an autonomous underwater vehicle as part of a study mapping the boundary of a river plume. The results demonstrate the potential of combining statistical methods and robotic platforms to effectively inform and execute data-driven environmental sampling

United States Air Force Officer Manpower Planning Problem via Approximate Dynamic Programming

The United States Air Force (USAF) is concerned with managing its officer corps to ensure sufficient personnel for mission readiness. Manpower planning for the USAF is a complex process which requires making decisions about accessions. Uncertainty about officer retention complicates such decisions. We formulate a Markov decision process model of the Air Force officer manpower planning problem (AFO-MPP) and utilize a least squares approximate policy iteration algorithm as an approximate dynamic programming (ADP) technique to attain solutions. Computational experiments are conducted on two AFO-MPP instances to compare the performance of the policy determined with the ADP algorithm to a benchmark policy. We find that the ADP algorithm performs well for the basis functions selected, providing policies which reduce soft costs associated with shortages and surpluses in the force

Approximate Dynamic Programming Algorithms for United States Air Force Officer Sustainment

The United States Air Force (USAF) officer sustainment system involves making accession and promotion decisions for nearly 64 thousand officers annually. We formulate a discrete time stochastic Markov decision process model to examine this military workforce planning problem. The large size of the motivating problem suggests that conventional exact dynamic programming algorithms are inappropriate. As such, we propose two approximate dynamic programming (ADP) algorithms to solve the problem. We employ a least-squares approximate policy iteration (API) algorithm with instrumental variables Bellman error minimization to determine approximate policies. In this API algorithm, we use a modified version of the Bellman equation based on the post-decision state variable. Approximating the value function using a post-decision state variable allows us to find the best policy for a given approximation using a decomposable mixed integer nonlinear programming formulation. We also propose an approximate value iteration algorithm using concave adaptive value estimation (CAVE). The CAVE algorithm identities an improved policy for a test problem based on the current USAF officer sustainment system. The CAVE algorithm obtains a statistically significant 2.8% improvement over the currently employed USAF policy, which serves as the benchmark

Using Approximate Dynamic Programming to Solve the Stochastic Demand Military Inventory Routing Problem with Direct Delivery

A brigade combat team must resupply forward operating bases (FOBs) within its area of operations from a central location, mainly via ground convoy operations, in a way that closely resembles vendor managed inventory practices. Military logisticians routinely decide when and how much inventory to distribute to each FOB. Technology currently exists that makes utilizing cargo unmanned aerial vehicles (CUAVs) for resupply an attractive alternative due to the dangers of utilizing convoy operations. However, enemy actions, austere conditions, and inclement weather pose a significant risk to a CUAV\u27s ability to safely deliver supplies to a FOB. We develop a Markov decision process model that allows for multiple supply classes to examine the military inventory routing problem, explicitly accounting for the possible loss of CUAVs during resupply operations. The large size of the motivating problem instance renders exact dynamic programming techniques computationally intractable. To overcome this challenge, we employ approximate dynamic programming (ADP) techniques to obtain high-quality resupply policies. We employ an approximate policy iteration algorithmic strategy that utilizes least squares temporal differencing for policy evaluation. We construct a representative problem instance based on an austere combat environment in order to demonstrate the efficacy of our model formulation and solution methodology. Because our ADP algorithm has many tunable features, we perform a robust, designed computational experiment to determine the ADP policy with the best quality of solutions. Results indicate utilizing least squares temporal differences with a first-order basis function is insufficient to approximate the value function when stochastic demand and penalty functions are implemented