Resource Allocation Among Agents with MDP-Induced Preferences

Allocating scarce resources among agents to maximize global utility is, in general, computationally challenging. We focus on problems where resources enable agents to execute actions in stochastic environments, modeled as Markov decision processes (MDPs), such that the value of a resource bundle is defined as the expected value of the optimal MDP policy realizable given these resources. We present an algorithm that simultaneously solves the resource-allocation and the policy-optimization problems. This allows us to avoid explicitly representing utilities over exponentially many resource bundles, leading to drastic (often exponential) reductions in computational complexity. We then use this algorithm in the context of self-interested agents to design a combinatorial auction for allocating resources. We empirically demonstrate the effectiveness of our approach by showing that it can, in minutes, optimally solve problems for which a straightforward combinatorial resource-allocation technique would require the agents to enumerate up to 2^100 resource bundles and the auctioneer to solve an NP-complete problem with an input of that size

Voluntary Matching Grants can Forestall Social Dumping

The European economic integration leads to increasing mobility of factors, thereby threatening the stability of social transfer programs. This paper investigates the possibility to achieve by means of voluntary matching grants both the optimal allocation of factors and the optimal level of redistribution in the presence of factor mobility. We use a fiscal competition model a la Wildasin (1991) in which states differ in their technologies and preferences for redistribution. We first investigate a simple process in which the regulatory authority progressively raises the matching grants to the district choosing the lowest transfer and all districts respond optimally to the resulting change in transfers all around. This process is shown to increase total production and the level of reditribution. However it does not guarantee that all districts gain, nor that an efficient level of redistribution is attained. Assuming complete information among districts, we first derive the willingness of each district to match the contribution of other districts and we show that the aggregate willingness to pay for matching rates converges to zero when both the efficient level of redistribution and the efficient allocation of factors are achieved. We then describe the ajustment process for the matching rates that will lead districts to the efficient outcome and guarantee that everyone will gain.

Techniques for the allocation of resources under uncertainty

L’allocation de ressources est un problème omniprésent qui survient dès que des ressources limitées doivent être distribuées parmi de multiples agents autonomes (e.g., personnes, compagnies, robots, etc). Les approches standard pour déterminer l’allocation optimale souffrent généralement d’une très grande complexité de calcul. Le but de cette thèse est de proposer des algorithmes rapides et efficaces pour allouer des ressources consommables et non consommables à des agents autonomes dont les préférences sur ces ressources sont induites par un processus stochastique. Afin d’y parvenir, nous avons développé de nouveaux modèles pour des problèmes de planifications, basés sur le cadre des Processus Décisionnels de Markov (MDPs), où l’espace d’actions possibles est explicitement paramétrisés par les ressources disponibles. Muni de ce cadre, nous avons développé des algorithmes basés sur la programmation dynamique et la recherche heuristique en temps-réel afin de générer des allocations de ressources pour des agents qui agissent dans un environnement stochastique. En particulier, nous avons utilisé la propriété acyclique des créations de tâches pour décomposer le problème d’allocation de ressources. Nous avons aussi proposé une stratégie de décomposition approximative, où les agents considèrent des interactions positives et négatives ainsi que les actions simultanées entre les agents gérants les ressources. Cependant, la majeure contribution de cette thèse est l’adoption de la recherche heuristique en temps-réel pour l’allocation de ressources. À cet effet, nous avons développé une approche basée sur la Q-décomposition munie de bornes strictes afin de diminuer drastiquement le temps de planification pour formuler une politique optimale. Ces bornes strictes nous ont permis d’élaguer l’espace d’actions pour les agents. Nous montrons analytiquement et empiriquement que les approches proposées mènent à des diminutions de la complexité de calcul par rapport à des approches de planification standard. Finalement, nous avons testé la recherche heuristique en temps-réel dans le simulateur SADM, un simulateur d’allocation de ressource pour une frégate.Resource allocation is an ubiquitous problem that arises whenever limited resources have to be distributed among multiple autonomous entities (e.g., people, companies, robots, etc). The standard approaches to determine the optimal resource allocation are computationally prohibitive. The goal of this thesis is to propose computationally efficient algorithms for allocating consumable and non-consumable resources among autonomous agents whose preferences for these resources are induced by a stochastic process. Towards this end, we have developed new models of planning problems, based on the framework of Markov Decision Processes (MDPs), where the action sets are explicitly parameterized by the available resources. Given these models, we have designed algorithms based on dynamic programming and real-time heuristic search to formulating thus allocations of resources for agents evolving in stochastic environments. In particular, we have used the acyclic property of task creation to decompose the problem of resource allocation. We have also proposed an approximative decomposition strategy, where the agents consider positive and negative interactions as well as simultaneous actions among the agents managing the resources. However, the main contribution of this thesis is the adoption of stochastic real-time heuristic search for a resource allocation. To this end, we have developed an approach based on distributed Q-values with tight bounds to diminish drastically the planning time to formulate the optimal policy. These tight bounds enable to prune the action space for the agents. We show analytically and empirically that our proposed approaches lead to drastic (in many cases, exponential) improvements in computational efficiency over standard planning methods. Finally, we have tested real-time heuristic search in the SADM simulator, a simulator for the resource allocation of a platform

Preference Based Fair Allocation of Limitted Resources

The fair division of scarce resources among agents is a challenging issue across a range of applications, especially when there is competition among agents. One application of resource division is in Air Traffic Management (ATM). This dissertation is motivated by the fairness issues that arise in the resource allocation procedures that have been introduced under Collaborative Decision Making (CDM). Fair rationing and allocation of available en-route time slots are two major challenges that we address in this research. The first challenge, fair rationing, is about how to compute a fair share of available resources among agents, when the available resources fall below the total demand. Since the demand, (flights), are time dependent, we introduce a new rationing method that includes the time dependency of demand. The new procedure gives every flight that is disrupted by an AFP a share of available resources. This is in contrast to Ration-By-Schedule (RBS), the allocation method currently in use, where later scheduled flights do not receive any slots. We will discuss and prove the fairness properties of our novel rationing procedure. The second challenge, allocation of en-route resources, is about how to allocate resources among competitive agents, (flight operators), when each agent has different preferences over resources, (time slots). We design four randomized procedures for allocating scarce resources when the airlines' preferences are included. These procedures use an exogenous fair share, which can be computed using the method described above, as a fairness standard for the allocation of slots among airlines. The first two procedures, Preference Based Proportional Random Allocation (PBPRA) and Modified-PBPRA, implicity assume equal weight for each time slot. Compared to RBS, PBPRA and M-PBPRA reduce the total internal cost of airlines and also assign each airline a number of slots close (in expectation) to their fair share. The fairness, efficiency and incentive properties of PBPRA and M-PBPRA are evaluated. The value (or cost of delay) an airline associates with a particular flight may vary substantially from flight to flight. Airlines who wish to receive priority for certain flights usually are willing to pay more for specific time slots. To address the need to express varying priorities, we propose two procedures, Dual Price Proportional Random Allocation (DP-PRA) and Modified-DP-PRA (MDP-PRA) , that assign dual prices to resources, i.e. time slots, in order to capture the airlines' preferences over delays, rerouting and cancelations. We explore the fairness, efficiency and incentive properties of DP-PRA and MDP-PRA