Approximate Dynamic Programming for Military Resource Allocation

This research considers the optimal allocation of weapons to a collection of targets with the objective of maximizing the value of destroyed targets. The weapon-target assignment (WTA) problem is a classic non-linear combinatorial optimization problem with an extensive history in operations research literature. The dynamic weapon target assignment (DWTA) problem aims to assign weapons optimally over time using the information gained to improve the outcome of their engagements. This research investigates various formulations of the DWTA problem and develops algorithms for their solution. Finally, an embedded optimization problem is introduced in which optimization of the multi-stage DWTA is used to determine optimal weaponeering of aircraft. Approximate dynamic programming is applied to the various formulations of the WTA problem. Like many in the field of combinatorial optimization, the DWTA problem suffers from the curses of dimensionality and exact solutions are often computationally intractability. As such, approximations are developed which exploit the special structure of the problem and allow for efficient convergence to high-quality local optima. Finally, a genetic algorithm solution framework is developed to test the embedded optimization problem for aircraft weaponeering

Operational Decision Making under Uncertainty: Inferential, Sequential, and Adversarial Approaches

Modern security threats are characterized by a stochastic, dynamic, partially observable, and ambiguous operational environment. This dissertation addresses such complex security threats using operations research techniques for decision making under uncertainty in operations planning, analysis, and assessment. First, this research develops a new method for robust queue inference with partially observable, stochastic arrival and departure times, motivated by cybersecurity and terrorism applications. In the dynamic setting, this work develops a new variant of Markov decision processes and an algorithm for robust information collection in dynamic, partially observable and ambiguous environments, with an application to a cybersecurity detection problem. In the adversarial setting, this work presents a new application of counterfactual regret minimization and robust optimization to a multi-domain cyber and air defense problem in a partially observable environment

Techniques for the allocation of resources under uncertainty

L’allocation de ressources est un problème omniprésent qui survient dès que des ressources limitées doivent être distribuées parmi de multiples agents autonomes (e.g., personnes, compagnies, robots, etc). Les approches standard pour déterminer l’allocation optimale souffrent généralement d’une très grande complexité de calcul. Le but de cette thèse est de proposer des algorithmes rapides et efficaces pour allouer des ressources consommables et non consommables à des agents autonomes dont les préférences sur ces ressources sont induites par un processus stochastique. Afin d’y parvenir, nous avons développé de nouveaux modèles pour des problèmes de planifications, basés sur le cadre des Processus Décisionnels de Markov (MDPs), où l’espace d’actions possibles est explicitement paramétrisés par les ressources disponibles. Muni de ce cadre, nous avons développé des algorithmes basés sur la programmation dynamique et la recherche heuristique en temps-réel afin de générer des allocations de ressources pour des agents qui agissent dans un environnement stochastique. En particulier, nous avons utilisé la propriété acyclique des créations de tâches pour décomposer le problème d’allocation de ressources. Nous avons aussi proposé une stratégie de décomposition approximative, où les agents considèrent des interactions positives et négatives ainsi que les actions simultanées entre les agents gérants les ressources. Cependant, la majeure contribution de cette thèse est l’adoption de la recherche heuristique en temps-réel pour l’allocation de ressources. À cet effet, nous avons développé une approche basée sur la Q-décomposition munie de bornes strictes afin de diminuer drastiquement le temps de planification pour formuler une politique optimale. Ces bornes strictes nous ont permis d’élaguer l’espace d’actions pour les agents. Nous montrons analytiquement et empiriquement que les approches proposées mènent à des diminutions de la complexité de calcul par rapport à des approches de planification standard. Finalement, nous avons testé la recherche heuristique en temps-réel dans le simulateur SADM, un simulateur d’allocation de ressource pour une frégate.Resource allocation is an ubiquitous problem that arises whenever limited resources have to be distributed among multiple autonomous entities (e.g., people, companies, robots, etc). The standard approaches to determine the optimal resource allocation are computationally prohibitive. The goal of this thesis is to propose computationally efficient algorithms for allocating consumable and non-consumable resources among autonomous agents whose preferences for these resources are induced by a stochastic process. Towards this end, we have developed new models of planning problems, based on the framework of Markov Decision Processes (MDPs), where the action sets are explicitly parameterized by the available resources. Given these models, we have designed algorithms based on dynamic programming and real-time heuristic search to formulating thus allocations of resources for agents evolving in stochastic environments. In particular, we have used the acyclic property of task creation to decompose the problem of resource allocation. We have also proposed an approximative decomposition strategy, where the agents consider positive and negative interactions as well as simultaneous actions among the agents managing the resources. However, the main contribution of this thesis is the adoption of stochastic real-time heuristic search for a resource allocation. To this end, we have developed an approach based on distributed Q-values with tight bounds to diminish drastically the planning time to formulate the optimal policy. These tight bounds enable to prune the action space for the agents. We show analytically and empirically that our proposed approaches lead to drastic (in many cases, exponential) improvements in computational efficiency over standard planning methods. Finally, we have tested real-time heuristic search in the SADM simulator, a simulator for the resource allocation of a platform

AFIT UAV Swarm Mission Planning and Simulation System

The purpose of this research is to design and implement a comprehensive mission planning system for swarms of autonomous aerial vehicles. The system integrates several problem domains including path planning, vehicle routing, and swarm behavior. The developed system consists of a parallel, multi-objective evolutionary algorithm-based path planner, a genetic algorithm-based vehicle router, and a parallel UAV swarm simulator. Each of the system\u27s three primary components are developed on AFIT\u27s Beowulf parallel computer clusters. Novel aspects of this research include: integrating terrain following technology into a swarm model as a means of detection avoidance, combining practical problems of path planning and routing into a comprehensive mission planning strategy, and the development of a swarm behavior model with path following capabilities

Proceedings of the 5th MIT/ONR Workshop on C[3] Systems, held at Naval Postgraduate School, Monterey, California, August 23 to 27, 1982

"December 1982."Includes bibliographies and index.Office of Naval Research Contract no. ONR/N00014-77-C-0532 NR041-519edited by Michael Athans ... [et al.]

Essays on Market Design and Auction Theory

This dissertation consists of three essays on market design and auction theory. In the first chapter, we develop a model of decentralized college admissions in which students' preferences for colleges are uncertain, and colleges must incur costs when their enrollments exceed their capacities. Colleges' admission decisions then become a tool for strategic yield management, because the enrollment at a college depends on not only students' uncertain preferences but also other colleges' admission decisions. We find that colleges' equilibrium admission decisions exhibit "strategic targeting''---colleges may forgo admitting (even good) students likely sought after by the others and may admit (not as good) students likely overlooked by the others. Randomization in admissions may also emerge. The resulting assignment fails to be efficient (among students, among colleges and among all parties including colleges and students) and leads to justified envy among students. When the colleges consider multiple dimensions of students merits, their evaluations are unlikely to be perfectly correlated. In such a case, colleges may avoid head-on competition by distorting their evaluation to place excessive weight on less correlated dimensions, such as extra curricular activities and non-academic aspects of students' application portfolios. Restricting the number of applications or allowing for wait-listing might alleviate colleges' yield management problem, but the resulting assignments are still inefficient and admit justified envy. Centralized matching via Gale and Shapley's Deferred Acceptance algorithm eliminates colleges' yield management problem and justified envy among students and attains efficiency. It also attains the outcome that is jointly optimal among colleges, but some colleges may be worse off relative to decentralized matching. The second chapter studies a keyword auction model where bidders have constrained budgets. In the absence of budget constraints, Edelman, Ostrovsky, and Schwarz (2007) and Varian (2007) analyze "locally envy-free equilibrium'' or "symmetric Nash equilibrium'' bidding strategies in generalized second-price (GSP) auctions. However, bidders often have to set their daily budgets when they participate in an auction; once a bidder's payment reaches his budget, he drops out of the auction. This raises an important strategic issue that has been overlooked in the previous literature: Bidders may change their bids to inflict higher prices on their competitors because under GSP, the per-click price paid by a bidder is the next highest bid. We provide budget thresholds under which equilibria analyzed in Edelman, Ostrovsky, and Schwarz (2007) and Varian (2007) are sustained as "equilibria with budget constraints'' in our setting. We then consider a simple environment with one position and two bidders and show that a search engine's revenue with budget constraints may be larger than its revenue without budget constraints. In the third chapter, we study the procurement of an innovation in which firms exert effort and create innovations, where the quality of innovation is stochastic. Both the effort level and the quality of innovation are unverifiable, and the procurer cannot extract up-front payment from the firms. Given the uncertainty of quality realization, there is a trade-off regarding the number of participating firms in the procurement process: If many firms participate in the process, they may be discouraged from expending their initial investment because each of them has a small chance of winning (we call this incentive effect). At the same time, as the number of participants increases, the procurer has a growing chance of getting a higher quality because of the randomness of the quality realization (sampling effect). Therefore, the procurer faces a nontrivial problem of how many firms to invite in the procurement process. We consider two prominent contest mechanisms, a first-price auction and a fixed-prize tournament. We show that if the randomness is large enough, it is optimal for the buyer to invite as many firms as possible in both mechanisms, and the fixed-prize tournament outperforms the first-price auction. In the limit at which the randomness vanishes, inviting only two firms is optimal in both mechanisms, and the first-price auction outperforms the fixed-prize tournament. Under the first-price auction, we show that any equilibrium converges to an equilibrium as the randomness diminishes and provide a characterization of the limit equilibrium. We also provide a constructive example of a mixed-strategy equilibrium with two firms when the randomness is moderate

NASA Thesaurus. Volume 2: Access vocabulary

The NASA Thesaurus -- Volume 2, Access Vocabulary -- contains an alphabetical listing of all Thesaurus terms (postable and nonpostable) and permutations of all multiword and pseudo-multiword terms. Also included are Other Words (non-Thesaurus terms) consisting of abbreviations, chemical symbols, etc. The permutations and Other Words provide 'access' to the appropriate postable entries in the Thesaurus