United States Air Force Officer Manpower Planning Problem via Approximate Dynamic Programming

The United States Air Force (USAF) is concerned with managing its officer corps to ensure sufficient personnel for mission readiness. Manpower planning for the USAF is a complex process which requires making decisions about accessions. Uncertainty about officer retention complicates such decisions. We formulate a Markov decision process model of the Air Force officer manpower planning problem (AFO-MPP) and utilize a least squares approximate policy iteration algorithm as an approximate dynamic programming (ADP) technique to attain solutions. Computational experiments are conducted on two AFO-MPP instances to compare the performance of the policy determined with the ADP algorithm to a benchmark policy. We find that the ADP algorithm performs well for the basis functions selected, providing policies which reduce soft costs associated with shortages and surpluses in the force

Autonomous dynamic decision making in fuel cycle simulators using a game theoretic approach

A novel methodology for optimizing nuclear fuel cycle transitions that captures interactions between a policy maker and electric utility company is presented. The methodology is demonstrated using a two-person general-sum sequential game with uncertainty that is implemented using a nuclear fuel cycle simulator capable of calculating a material- and technology-constrained material balance, coupled to a multi-objective optimization solver. The solver explicitly treats uncertainties using a stochastic programming approach with chance nodes depicted as a Nature player who moves randomly. The methodology is demonstrated through a Transition Game that features tradeoffs between investments in competing reprocessing and waste disposal technologies, dynamic reactor deployment responses to resolutions in reactor capital cost uncertainty, and the influence of capital subsidies on the future nuclear technology mix. Each player in the game uses a unique set of decision criteria to identify optimal near-term hedging strategies that consider all of Nature’s possible moves as well as the other player’s available decisions. These hedging strategies balance the exchange between the risk of immediate action and delay and maintain flexibility to allow for intelligent recourse decisions once uncertainties are resolved. Results from the Transition Game indicate that early transition to high-temperature gas-cooled reactors is preferred, with the option to abandon the transition following a learning period if capital costs are unfavorable. Under these conditions, transition to used fuel recycling in sodium-cooled fast reactors may be spurred by policy incentives under some certain decision criteria weightings. Otherwise, operating with a baseline set of decision criteria weightings, transition to a closed fuel is never observed when players hedge optimally against Nature’s moves. It is only when players have perfect information regarding Nature’s future moves will transition to a closed fuel be observed.Mechanical Engineerin

Scheduling Allocation and Inventory Replenishment Problems Under Uncertainty: Applications in Managing Electric Vehicle and Drone Battery Swap Stations

In this dissertation, motivated by electric vehicle (EV) and drone application growth, we propose novel optimization problems and solution techniques for managing the operations at EV and drone battery swap stations. In Chapter 2, we introduce a novel class of stochastic scheduling allocation and inventory replenishment problems (SAIRP), which determines the recharging, discharging, and replacement decisions at a swap station over time to maximize the expected total profit. We use Markov Decision Process (MDP) to model SAIRPs facing uncertain demands, varying costs, and battery degradation. Considering battery degradation is crucial as it relaxes the assumption that charging/discharging batteries do not deteriorate their quality (capacity). Besides, it ensures customers receive high-quality batteries as we prevent recharging/discharging and swapping when the average capacity of batteries is lower than a predefined threshold. Our MDP has high complexity and dimensions regarding the state space, action space, and transition probabilities; therefore, we can not provide the optimal decision rules (exact solutions) for SAIRPs of increasing size. Thus, we propose high-quality approximate solutions, heuristic and reinforcement learning (RL) methods, for stochastic SAIRPs that provide near-optimal policies for the stations. In Chapter 3, we explore the structure and theoretical findings related to the optimal solution of SAIRP. Notably, we prove the monotonicity properties to develop fast and intelligent algorithms to provide approximate solutions and overcome the curses of dimensionality. We show the existence of monotone optimal decision rules when there is an upper bound on the number of batteries replaced in each period. We demonstrate the monotone structure for the MDP value function when considering the first, second, and both dimensions of the state. We utilize data analytics and regression techniques to provide an intelligent initialization for our monotone approximate dynamic programming (ADP) algorithm. Finally, we provide insights from solving realistic-sized SAIRPs. In Chapter 4, we consider the problem of optimizing the distribution operations of a hub using drones to deliver medical supplies to different geographic regions. Drones are an innovative method with many benefits including low-contact delivery thereby reducing the spread of pandemic and vaccine-preventable diseases. While we focus on medical supply delivery for this work, it is applicable to drone delivery for many other applications, including food, postal items, and e-commerce delivery. In this chapter, our goal is to address drone delivery challenges by optimizing the distribution operations at a drone hub that dispatch drones to different geographic locations generating stochastic demands for medical supplies. By considering different geographic locations, we consider different classes of demand that require different flight ranges, which is directly related to the amount of charge held in a drone battery. We classify the stochastic demands based on their distance from the drone hub, use a Markov decision process to model the problem, and perform computational tests using realistic data representing a prominent drone delivery company. We solve the problem using a reinforcement learning method and show its high performance compared with the exact solution found using dynamic programming. Finally, we analyze the results and provide insights for managing the drone hub operations

National freight transport planning: towards a Strategic Planning Extranet Decision Support System (SPEDSS)

This thesis provides a `proof-of-concept' prototype and a design architecture for a Object Oriented (00) database towards the development of a Decision Support System (DSS) for the national freight transport planning problem. Both governments and industry require a Strategic Planning Extranet Decision Support System (SPEDSS) for their effective management of the national Freight Transport Networks (FTN). This thesis addresses the three key problems for the development of a SPEDSS to facilitate national strategic freight planning: 1) scope and scale of data available and required; 2) scope and scale of existing models; and 3) construction of the software. The research approach taken embodies systems thinking and includes the use of: Object Oriented Analysis and Design (OOA/D) for problem encapsulation and database design; artificial neural network (and proposed rule extraction) for knowledge acquisition of the United States FTN data set; and an iterative Object Oriented (00) software design for the development of a `proof-of-concept' prototype. The research findings demonstrate that an 00 approach along with the use of 00 methodologies and technologies coupled with artificial neural networks (ANNs) offers a robust and flexible methodology for the analysis of the FTN problem domain and the design architecture of an Extranet based SPEDSS. The objectives of this research were to: 1) identify and analyse current problems and proposed solutions facing industry and governments in strategic transportation planning; 2) determine the functional requirements of an FTN SPEDSS; 3) perform a feasibility analysis for building a FTN SPEDSS `proof-of-concept' prototype and (00) database design; 4) develop a methodology for a national `internet-enabled' SPEDSS model and database; 5) construct a `proof-of-concept' prototype for a SPEDSS encapsulating identified user requirements; 6) develop a methodology to resolve the issue of the scale of data and data knowledge acquisition which would act as the `intelligence' within a SPDSS; 7) implement the data methodology using Artificial Neural Networks (ANNs) towards the validation of it; and 8) make recommendations for national freight transportation strategic planning and further research required to fulfil the needs of governments and industry. This thesis includes: an 00 database design for encapsulation of the FTN; an `internet-enabled' Dynamic Modelling Methodology (DMM) for the virtual modelling of the FTNs; a Unified Modelling Language (UML) `proof-of-concept' prototype; and conclusions and recommendations for further collaborative research are identified

Energy Biased Technical Change: A CGE Analysis

This paper studies energy bias in technical change. For this purpose, we develop a computable general equilibrium model that builds on endogenous growth models. The model explicitly captures links between energy, the rate and direction of technical change, and the economy. We derive the equilibrium determinants of biased technical change and show the importance of feedback in technical change, substitution possibilities between final goods, and general-equilibrium effects for the equilibrium bias. If the feedback effect is strong, or the substitution elasticity large, or both, our model tends to a corner solution in which only technologies are developed that are appropriate for production of non-energy intensive goods.Computable general-equilibrium models, Endogenous technical change, Energy, Environment