Advances in ranking and selection: variance estimation and constraints

In this thesis, we first show that the performance of ranking and selection (R&S) procedures in steady-state simulations depends highly on the quality of the variance estimates that are used. We study the performance of R&S procedures using three variance estimators --- overlapping area, overlapping Cramer--von Mises, and overlapping modified jackknifed Durbin--Watson estimators --- that show better long-run performance than other estimators previously used in conjunction with R&S procedures for steady-state simulations. We devote additional study to the development of the new overlapping modified jackknifed Durbin--Watson estimator and demonstrate some of its useful properties. Next, we consider the problem of finding the best simulated system under a primary performance measure, while also satisfying stochastic constraints on secondary performance measures, known as constrained ranking and selection. We first present a new framework that allows certain systems to become dormant, halting sampling for those systems as the procedure continues. We also develop general procedures for constrained R&S that guarantee a nominal probability of correct selection, under any number of constraints and correlation across systems. In addition, we address new topics critical to efficiency of the these procedures, namely the allocation of error between feasibility check and selection, the use of common random numbers, and the cost of switching between simulated systems.Ph.D.Committee Co-chairs: Sigrun Andradottir, Dave Goldsman and Seong-Hee Kim; Committee Members:Shabbir Ahmed and Brani Vidakovi

Performance of sequential batching-based methods of output data analysis in distributed steady-state stochastic simulation

Wir haben die Anpassung von Sequentiellen Analysemethoden von Stochastik Simulationen an einem Szenario von mehreren Unabhängigen Replikationen in Parallel (MRIP) untersucht. Die Hauptidee ist, die statistische Kontrole bzw. die Beschleunigung eines Simulationexperiment zu automatisieren. Die vorgeschlagenen Methoden der Literatur sind auf einzelne Prozessorszenarien orientiert. Wenig ist bekannt hinsichtlich der Anwendungen von Verfahen, die auf Methoden unter MRIP basieren. Auf den ersten Blick sind beide Ziele entgegengesetzt, denn man braucht eine grosse Menge von Beobachtungen, um eine hohe Qualität der Resultate zu erreichen. Dafür benötig man viel Zeit. Man kann jedoch durch einen ausfürlichen Entwurf zusammen mit einem robusten Werkzeug, das auf unabhängige Replikationen basiert ist, ein effizientes Mittel bezüglich Analyse der Resultate produzieren. Diese Recherche wurde mit einer sequentiellen Version des klassischen Verfahren von Nonoverlaping Batch Means (NOBM) angefangen. Obwohl NOBM sehr intuitiv und populär ist, bietet es keine gute Lösung für das Problem starker Autokorrelation zwischen den Beobachtungen an, die normalerweise bei hohen Auslastungen entstehen. Es lohnt sich nicht, grösserer Rechnerleistung zu benutzen, um diese negative Merkmale zu vermindern. Das haben wir mittles einer vollständigen Untersuchung einer Gruppe von Warteschlangsystemen bestätig. Deswegen haben wir den Entwurf von sequentiellen Versionen von ein paar Varianten von Batch Means vorgeschlagen und sie genauso untersucht. Unter den implementierten Verfahren gibt es ein sehr attraktives: Overlapping Batch Means (OBM). OBM ermöglicht eine bessere Nutzung der Daten, da jede Beobachtungen ein neues Batch anfängt, d.h., die Anzahl von Batches ist viel grösser, und das ergibt eine kleinere Varianz. In diesem Fall ist die Anwendung von MRIP empfehlenswert, da diese Kombination weniger Beobachtungen benötigt und somit eine höhere Beschleunigung. Im Laufe der Recherche haben wir eine Klasse von Methoden (Standardized Time Series - STS) untersucht, die teoretisch bessere asymptotische Resultate als NOBM produziert. Die negative Auswirkung von STS ist, dass sie mehr Beobachtungen als die Batch-Means-Verfahren benoetigt. Aber das ist kein Hindernis, wenn wir STS zusammen mit MRIP anwenden. Die experimentelle Untersuchungen bestätigte, dass die Hypothese richtig ist. Die nächste Phase war es, OBM und STS einzustellen, um beide Verfahren unter den grösstmöglichen Anzahl von Prozessoren arbeiten lassen zu können. Fallstudien zeigten uns, dass sich beide sequentiellen Verfahren für die parallele Simulation sowie MRIP einigen.We investigated the feasibility of sequential methods of analysis of stochastic simulation under an environment of Multiple Replications in Parallel (MRIP). The main idea is twofold, the automation of the statistical control and speedup of simulation experiments. The methods of analysis found suggested in the literature were conceived for a single processor environment. Very few is known concerning the application of procedures based in such methods under MRIP. At first glance, sind both goals in opposition, since one needs a large amount of observations in order to achieve good quality of the results, i.e., the simulation takes frequently long time. However, by means of a careful design, together with a robust simulation tool based on independent replications, one can produce an efficient instrument of analysis of the simulation results. This research began with a sequential version of the classical method of Nonoverlapping Batch Means (NOBM). Although intuitiv and popular, under hight traffic intensity NOBM offers no good solution to the problem of strong correlation among the observations. It is not worthwhile to apply more computing power aiming to diminish this negative effect. We have confirmed this claim by means of a detailed and exhaustive analysis of four queuing systems. Therefore, we proposed the design of sequential versions of some Batch Means variants, and we investigated their statistical properties under MRIP. Among the implemented procedures there is one very attractive : Overlapping Batch Means (OBM). OBM makes a better use of collected data, since each observation initiates a new (overlapped) batch, that is, die number of batches is much larger, and this yields smaller variance. In this case, MRIP is highly recommended, since this combination requires less observations and, therefore, speedup. During the research, we investigated also a class of methods based on Standardized Time Series -- STS, that produces theoretically better asymptotical results than NOBM. The undesired negative effect of STS is the large number of observations it requires, when compared to NOBM. But that is no obstacle when we apply STS together with MRIP. The experimental investigation confirmed this hypothesis. The next phase was to tun OBM and STS, in order to put them working with the possible largest number of processors. A case study showed us that both procedures are suitable to the environment of MRIP

Une nouvelle approche pour l’identification des états dynamiques de la parcellisation fonctionnelle cérébrale individuelle

Les parcellations cérébrales sont appliquées en neuroimagerie pour aider les chercheurs à ré- duire la haute dimensionnalité des données d’IRM fonctionnelle. L’objectif principal est une meilleure compréhension de l’organisation fonctionnelle du cerveau tant chez les sujets sains que chez les sujets souffrant de troubles neurologiques, dont la maladie d’Alzheimer. Malgré la vague d’approches de parcellations précédentes, les mesures de performance doivent en- core être améliorées pour générer des parcellations fiables, même avec de longues acquisitions. Autrement dit, une reproductibilité plus élevée qui permet aux chercheurs de reproduire des parcellations et de comparer leurs études. Il est également important de minimiser la perte d’informations entre les données compressées et les données brutes pour représenter avec précision l’organisation d’un cerveau individuel. Dans cette thèse, j’ai développé une nou- velle approche pour parcellaire le cerveau en reconfigurations spatiales distinctes appelées «états dynamiques de parcellations». J’ai utilisé une méthode d’agrégation de cluster simple DYPAC1.0 de parcelles basées sur des semences sur plusieurs fenêtres de temps. J’ai émis l’hypothèse que cette nouvelle façon de formaliser le problème de parcellisation améliorera les mesures de performance par rapport aux parcellations statiques. Le premier chapitre de ce document est une introduction générale au contexte des réseaux à grande échelle du cerveau humain. Je montre également l’importance des parcellations pour une meilleure compréhension du cerveau humain à l’aide de connectomes fonctionnels afin de prédire les schémas de progression de la maladie. Ensuite, j’explique pourquoi le problème de parcelli- sation cérébrale est difficile et les différentes questions de recherche ouvertes associées à ce domaine. Mes contributions à la recherche sont subdivisées en deux articles. Les deuxième et troisième chapitres sont consacrés au premier article principal et à son supplément publié dans Network Neuroscience Journal. Le quatrième chapitre représente le deuxième document en préparation. Le cinquième chapitre conclut mes contributions et ses implications dans le domaine de la neuroimagerie, ainsi que des orientations de recherche ouvertes. En un mot, la principale conclusion de ce travail est l’existence de reconfigurations spatiales distinctes dans tout le cerveau avec des scores de reproductibilité presque parfaits sur les données de test-retest (jusqu’à 0,9 coefficient de corrélation de Pearson). Un algorithme d’agrégation de cluster simple et évolutif appelé DYPAC 1.0 est expliqué pour identifier ces reconfigu- rations ou «états dynamiques de parcellations» pour des sous-réseaux de départ spécifiques (deuxième chapitre). L’analyse de ces états a montré l’existence d’un répertoire plus riche «d’états dynamiques» dans le cas des cortex hétéromodaux (ex: cortex cingulaire posté- rieur et cortex cingulaire antérieur dorsal) par rapport aux cortex unimodaux (ex: cortex visuel). En outre, les résultats de l’analyse de reproductibilité ont montré que DYPAC 1.0 a de meilleurs résultats de reproductibilité (en termes de corrélation de Pearson) par rapport aux parcelles statiques (deuxième chapitre). Plusieurs analyses démontrent que DYPAC 1.0 est robuste au choix de ses paramètres (troisième chapitre). Ces résultats et l’évolutivité de DYPAC 1.0 ont motivé une analyse complète du niveau cérébral. Je présente DYPAC 2.0 comme une approche au niveau cérébral complet pour fragmenter le cerveau en «états dynamiques de parcellations». Des reconfigurations spatiales distinctes et se chevauchant ou «états dynamiques» sont identifiées pour différentes régions du cerveau (quatrième chapitre). Ces états ont des scores de compression prometteurs qui montrent une faible perte d’infor- mations entre les cartes de stabilité d’état réduit et les données d’origine dans les cortex cérébraux, c’est-à-dire jusqu’à seulement 20% de perte de la variance expliquée. Cette thèse présente ainsi de nouvelles contributions dans le domaine de la parcellisation fonctionnelle qui pourraient avoir un impact sur la manière dont les chercheurs modélisent les interactions riches et dynamiques entre les réseaux cérébraux dans la santé et la maladie.Brain parcellations are applied in neuroimaging to help researchers reduce the high dimen- sionality of the functional MRI data. The main objective is a better understanding of the brain functional organization in both healthy subjects and subjects having neurological dis- orders, including Alzheimer disease. Despite the flurry of previous parcellation approaches, the performance measures still need improvement to generate reliable parcellations even with long acquisitions. That is, a higher reproducibility that allows researchers to replicate par- cellations and compare their studies. It is also important to minimize the information loss between the compressed data and the raw data to accurately represent the organization of an individual brain. In this thesis, I developed a new approach to parcellate the brain into distinct spatial reconfigurations called “dynamic states of parcellations”. I used a simple cluster aggregation method DYPAC1.0 of seed based parcels over multiple time windows. I hypothesized this new way to formalize the parcellation problem will improve performance measures over static parcellations. The first chapter of this document is a general context introduction to the human brain large scale networks. I also show the importance of par- cellations for a better understanding of the human brain using functional connectomes in order to predict patterns of disease progression. Then, I explain why the brain parcellation problem is hard and the different open research questions associated with this field. My research contributions are subdivided into two papers. The second and the third chapters are dedicated to the first main paper and its supplementary published in Network Neuro- science Journal. The fourth chapter represents the second paper under preparation. The fifth chapter concludes my contributions and its implications in the neuroimaging field, along with open research directions. In a nutshell, the main finding of this work is the existence of distinct spatial reconfigurations throughout the brain with near perfect reproducibility scores across test-retest data (up to .9 Pearson correlation coefficient). A simple and scalable clus- ter aggregation algorithm called DYPAC 1.0 is explained to identify these reconfigurations or “dynamic states of parcellations” for specific seed subnetworks (second chapter). The analysis of these states showed the existence of a richer repertoire of “dynamic states” in the case of heteromodal cortices (e.g., posterior cingulate cortex and the dorsal anterior cingulate cortex) compared to unimodal cortices (e.g., visual cortex). Also, the reproducibility analysis results showed that DYPAC 1.0 has better reproducibility results (in terms of Pearson corre- lation) compared to static parcels (second chapter). Several analyses demonstrate DYPAC 1.0 is robust to the choice of its parameters (third chapter). These findings and the scalabil- ity of DYPAC 1.0 motivated a full brain level analysis. I present DYPAC 2.0 as the full brain level approach to parcellate the brain into “dynamic states of parcellations”. Distinct and overlapping spatial reconfigurations or “dynamic states” are identified for different regions throughout the brain (fourth chapter). These states have promising compression scores that show low information loss between the reduced state stability maps and the original data throughout the cerebral cortices, i.e. up to only 20% loss in explained variance. This thesis thus presents new contributions in the functional parcellation field that may impact how researchers model the rich and dynamic interactions between brain networks in health and disease

On automated sequential steady-state simulation.

The credibility of the final results from stochastic simulation has had limited discussion in the simulation literature so far. However, it is important that the final results from any simulations be credible. To achieve this, validation, which determines whether the conceptual simulation model is an accurate representation of the system under study, has to be done carefully. Additionally, a proper statistical analysis of simulation output data, including a confidence interval or other assessment of statistical errors, has to be conducted before any valid inferences or conclusions about the performance of simulated dynamic systems, such as for example telecommunication networks, are made. There are many other issues, such as choice of a good pseudo-random number generator, elimination of initialisation bias in steady-state simulations, and consideration of auto correlations in collected observations, which have to be appropriately addressed for the final results to be credible. However, many of these issues are not trivial, particularly for simulation users who may not be experts in these areas. As a consequence, a fully-automated simulation package, which can control all important aspects of stochastic simulation, is needed. This dissertation focuses on the following contributions to such a package for steady-state simulation: properties of confidence intervals (CIs) used in coverage analysis, heuristic rules for improving the coverage of the final CIs in practical applications, automated sequential analysis of mean values by the method of regenerative cycles, automatic detection of the initial transient period for steady-state quantile estimation, and sequential steady-state quantile estimation with the automated detection of the length of initial transient period. One difficulty in obtaining precise estimates of a system using stochastic simulation can be the cost of the computing time needed to collect the large amount of output data required. Indeed there are situations, such as estimation of rare events, where, even assuming an appropriate statistical analysis procedure is available, the cost of collecting the number of observations needed by the analysis procedure can be prohibitively large. Fortunately, inexpensive computer network resources enable computationally intensive simulations by allowing us to run parallel and distributed simulations. Therefore, where possible, we extend the contributions to the distributed stochastic simulation scenario known as the Multiple Replications In Parallel (MRIP), in which multiple processors run their own independent replications of the simulated system but cooperate with central analysers that collect data to estimate the final results

Entwicklung und Einführung von Produktionssteuerungsverbesserungen für die kundenorientierte Halbleiterfertigung

Production control in a semiconductor production facility is a very complex and timeconsuming task. Different demands regarding facility performance parameters are defined by customer and facility management. These requirements are usually opponents, and an efficient strategy is not simple to define. In semiconductor manufacturing, the available production control systems often use priorities to define the importance of each production lot. The production lots are ranked according to the defined priorities. This process is called dispatching. The priority allocation is carried out by special algorithms. In literature, a huge variety of different strategies and rules is available. For the semiconductor foundry business, there is a need for a very flexible and adaptable policy taking the facility state and the defined requirements into account. At our case the production processes are characterized by a low-volume high-mix product portfolio. This portfolio causes additional stability problems and performance lags. The unstable characteristic increases the influence of reasonable production control logic. This thesis offers a very flexible and adaptable production control policy. This policy is based on a detailed facility model with real-life production data. The data is extracted from a real high-mix low-volume semiconductor facility. The dispatching strategy combines several dispatching rules. Different requirements like line balance, throughput optimization and on-time delivery targets can be taken into account. An automated detailed facility model calculates a semi-optimal combination of the different dispatching rules under a defined objective function. The objective function includes different demands from the management and the customer. The optimization is realized by a genetic heuristic for a fast and efficient finding of a close-to-optimal solution. The strategy is evaluated with real-life production data. The analysis with the detailed facility model of this fab shows an average improvement of 5% to 8% for several facility performance parameters like cycle time per mask layer. Finally the approach is realized and applied at a typical high-mix low-volume semiconductor facility. The system realization bases on a JAVA implementation. This implementation includes common state-of-the-art technologies such as web services. The system replaces the older production control solution. Besides the dispatching algorithm, the production policy includes the possibility to skip several metrology operations under defined boundary conditions. In a real-life production process, not all metrology operations are necessary for each lot. The thesis evaluates the influence of the sampling mechanism to the production process. The solution is included into the system implementation as a framework to assign different sampling rules to different metrology operations. Evaluations show greater improvements at bottleneck situations. After the productive introduction and usage of both systems, the practical results are evaluated. The staff survey offers good acceptance and response to the system. Furthermore positive effects on the performance measures are visible. The implemented system became part of the daily tools of a real semiconductor facility.Produktionssteuerung im Bereich der kundenorientierten Halbleiterfertigung ist heutzutage eine sehr komplexe und zeitintensive Aufgabe. Verschiedene Anforderungen bezüglich der Fabrikperformance werden seitens der Kunden als auch des Fabrikmanagements definiert. Diese Anforderungen stehen oftmals in Konkurrenz. Dadurch ist eine effiziente Strategie zur Kompromissfindung nicht einfach zu definieren. Heutige Halbleiterfabriken mit ihren verfügbaren Produktionssteuerungssystemen nutzen oft prioritätsbasierte Lösungen zur Definition der Wichtigkeit eines jeden Produktionsloses. Anhand dieser Prioritäten werden die Produktionslose sortiert und bearbeitet. In der Literatur existiert eine große Bandbreite verschiedener Algorithmen. Im Bereich der kundenorientierten Halbleiterfertigung wird eine sehr flexible und anpassbare Strategie benötigt, die auch den aktuellen Fabrikzustand als auch die wechselnden Kundenanforderungen berücksichtigt. Dies gilt insbesondere für den hochvariablen geringvolumigen Produktionsfall. Diese Arbeit behandelt eine flexible Strategie für den hochvariablen Produktionsfall einer solchen Produktionsstätte. Der Algorithmus basiert auf einem detaillierten Fabriksimulationsmodell mit Rückgriff auf Realdaten. Neben synthetischen Testdaten wurde der Algorithmus auch anhand einer realen Fertigungsumgebung geprüft. Verschiedene Steuerungsregeln werden hierbei sinnvoll kombiniert und gewichtet. Wechselnde Anforderungen wie Linienbalance, Durchsatz oder Liefertermintreue können adressiert und optimiert werden. Mittels einer definierten Zielfunktion erlaubt die automatische Modellgenerierung eine Optimierung anhand des aktuellen Fabrikzustandes. Die Optimierung basiert auf einen genetischen Algorithmus für eine flexible und effiziente Lösungssuche. Die Strategie wurde mit Realdaten aus der Fertigung einer typischen hochvariablen geringvolumigen Halbleiterfertigung geprüft und analysiert. Die Analyse zeigt ein Verbesserungspotential von 5% bis 8% für die bekannten Performancekriterien wie Cycletime im Vergleich zu gewöhnlichen statischen Steuerungspolitiken. Eine prototypische Implementierung realisiert diesen Ansatz zur Nutzung in der realen Fabrikumgebung. Die Implementierung basiert auf der JAVA-Programmiersprache. Aktuelle Implementierungsmethoden erlauben den flexiblen Einsatz in der Produktionsumgebung. Neben der Fabriksteuerung wurde die Möglichkeit der Reduktion von Messoperationszeit (auch bekannt unter Sampling) unter gegebenen Randbedingungen einer hochvariablen geringvolumigen Fertigung untersucht und geprüft. Oftmals ist aufgrund stabiler Prozesse in der Fertigung die Messung aller Lose an einem bestimmten Produktionsschritt nicht notwendig. Diese Arbeit untersucht den Einfluss dieses gängigen Verfahrens aus der Massenfertigung für die spezielle geringvolumige Produktionsumgebung. Die Analysen zeigen insbesondere in Ausnahmesituationen wie Anlagenausfällen und Kapazitätsengpässe einen positiven Effekt, während der Einfluss unter normalen Produktionsbedingungen aufgrund der hohen Produktvariabilität als gering angesehen werden kann. Nach produktiver Einführung in einem typischen Vertreter dieser Halbleiterfabriken zeigten sich schnell positive Effekte auf die Fabrikperformance als auch eine breite Nutzerakzeptanz. Das implementierte System wurde Bestandteil der täglichen genutzten Werkzeuglandschaft an diesem Standort

Study of Control Actions on a Manufacturing System Subject to a Dynamic Environment Created by Customers, Suppliers and Competitors

The existence of multiple, overlapping manufacturing philosophies creates confusion among manufacturing system managers who seek to better control the performance of the manufacturing system when responding to changes in the external elements such as suppliers, customers and competitors. Given the scenario where a set of random and difficult to predict external influences may change and impact the performance metrics of the system, a manager is often unable to pick most cost effective control actions to mitigate or exploit the effects of the influences. The control action is an investment to recover (mitigate/exploit) the effect of external influence(s) and to improve system performance. This research study addresses the managerial actions needed to select the control actions in the face of individual and grouped external influences acting on a system's performance metrics. Based on preliminary findings, the transient response characteristics of selected performance metrics were able to provide better understanding of the value of the control actions. Using the performance metrics response and a methodology proposed in this study to select control actions, a manager could estimate the performance of the control action(s) at the different intensity levels of the external influences. This study connects the cost impact of the changes in external influences to the control action(s) investment cost using net present value.Industrial Engineering & Managemen

Estimating credibility of science claims : analysis of forecasting data from metascience projects : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand

The veracity of scientific claims is not always certain. In fact, sufficient claims have been proven incorrect that many scientists believe that science itself is facing a “replication crisis”. Large scale replication projects provided empirical evidence that only around 50% of published social and behavioral science findings are replicable. Multiple forecasting studies showed that the outcomes of replication projects could be predicted by crowdsourced human evaluators. The research presented in this thesis builds on previous forecasting studies, deriving new findings and exploring new scope and scale. The research is centered around the DARPA SCORE (Systematizing Confidence in Open Research and Evidence) programme, a project aimed at developing measures of credibility for social and behavioral science claims. As part of my contribution to SCORE, myself, along with a international collaboration, elicited forecasts from human experts via surveys and prediction markets to predict the replicability of 3000 claims. I also present research on other forecasting studies. In chapter 2, I pool data from previous studies to analyse the performance of prediction markets and surveys with higher statistical power. I confirm that prediction markets are better at forecasting replication outcomes than surveys. This study also demonstrates the relationship between p-values of original findings and replication outcomes. These findings are used to inform the experimental and statistical design to forecast the replicability of 3000 claims as part of the SCORE programme. A full description of the design including planned statistical analyses is included in chapter 3. Due to COVID-19 restrictions, our generated forecasts could not be validated through direct replication, experiments conducted by other teams within the SCORE collaboration, thereby preventing results being presented in this thesis. The completion of these replications is now scheduled for 2022, and the pre-analysis plan presented in Chapter 3 will provide the basis for the analysis of the resulting data. In chapter 4, an analysis of ‘meta’ forecasts, or forecasts regarding field wide replication rates and year specific replication rates, is presented. We presented and published community expectations that replication rates will differ by field and will increase over time. These forecasts serve as valuable insights into the academic community’s views of the replication crisis, including those research fields for which no large-scale replication studies have been undertaken yet. Once the full results from SCORE are available, there will be additional insights from validations of the community expectations. I also analyse forecaster’s ability to predict replications and effect sizes in Chapters 5 (Creative Destruction in Science) and 6 (A creative destruction approach to replication: Implicit work and sex morality across cultures). In these projects a ‘creative destruction’ approach to replication was used, where a claim is compared not only to the null hypothesis but to alternative contradictory claims. I conclude forecasters can predict the size and direction of effects. Chapter 7 examines the use of forecasting for scientific outcomes beyond replication. In the COVID-19 preprint forecasting project I find that forecasters can predict if a preprint will be published within one year, including the quality of the publishing journal. Forecasters can also predict the number of citations preprints will receive. This thesis demonstrates that information about scientific claims with respect to replicability is dispersed within scientific community. I have helped to develop methodologies and tools to efficiently elicit and aggregate forecasts. Forecasts about scientific outcomes can be used as guides to credibility, to gauge community expectations and to efficiently allocate sparse replication resources

Retention Prediction and Policy Optimization for United States Air Force Personnel Management

Effective personnel management policies in the United States Air Force (USAF) require methods to predict the number of personnel who will remain in the USAF as well as to replenish personnel with different skillsets over time as they depart. To improve retention predictions, we develop and test traditional random forest models and feedforward neural networks as well as partially autoregressive forms of both, outperforming the benchmark on a test dataset by 62.8% and 34.8% for the neural network and the partially autoregressive neural network, respectively. We formulate the workforce replenishment problem as a Markov decision process for active duty enlisted personnel, then extend this formulation to include the Air Force Reserve and Air National Guard. We develop and test an adaptation of the Concave Adaptive Value Estimation (CAVE) algorithm and a parameterized Deep Q-Network on the active duty problem instance with 7050 dimensions, finding that CAVE reduces costs from the benchmark policy by 29.76% and 17.38% for the two cost functions tested. We test CAVE across a range of hyperparameters for the larger intercomponent problem instance with 21,240 dimensions, reducing costs by 23.06% from the benchmark, then develop the Stochastic Use of Perturbations to Enhance Robustness of CAVE (SUPERCAVE) algorithm, reducing costs by another 0.67%. Resulting algorithms and methods are directly applicable to contemporary USAF personnel business practices and enable more accurate, less time-intensive, cogent, and data-informed policy targets for current processes