Distributed Multiagent Resource Allocation using Reservations to Improve Handling of Dynamic Task Arrivals

In the artificial intelligence subfield of multi-agent systems, there are many applications for algorithms which optimally allocate a set of resources among many available tasks which demand those resources. In this thesis we present a distributed algorithm to solve this problem which adapts well to dynamic task arrivals, where new work arises at short notice. This algorithm builds on prior work which focused on finding the optimal allocation in a closed environment with a fixed number of tasks. Our algorithm is designed to leverage preemption if it is available, revoking resource allocations to tasks in progress if new opportunities arise which those resources are better suited to handle. However, interrupting tasks in progress is rarely without cost, and our algorithm both respects these costs and may reserve resources to avoid unnecessary costs from hasty allocation. Our multi-agent model assigns a task agent to each task which must be completed and a proxy agent to each resource which is available. These proxy agents are responsible for allocating the resource they manage, while task agents are responsible for learning about their environment and planning out which resources to request for their task. The distributed nature of our model makes it easy to dynamically introduce new tasks with associated task agents. Preemption occurs when a task agent approaches a proxy agent with a sufficiently compelling need that the proxy agent determines the newcomer derives more benefit from the proxy agent's resource than the task agent currently using that resource. We compare to other multi-agent resource allocation frameworks which permit preemption under more conservative assumptions, and show through simulation that our planning and learning techniques allow for improved allocations through more permissive preemption. Our simulations present a medical application which models fallible human resources, though the techniques used are applicable to other domains such as computer scheduling. We then revisit the model with a focus on opportunity cost, introducing resource reservation as an alternative method to preemption for addressing expected future changes in the task allocation environment. Simulations help identify the scenarios where opportunity cost is a significant concern. The model is then further expanded to account for switching costs, where interrupting tasks in progress is worse than simply delaying tasks, and the logical extreme where resource allocation is irrevocable thus encouraging careful decisions about where to commit resources. This thesis makes three primary contributions to multi-agent resource allocation. The first is an improved distributed resource allocation framework which uses Transfer-of-Control strategies and learning to rapidly find good allocations in a dynamic environment. The second is a discussion of the importance of opportunity cost in resource allocation, accompanied by a simple "dummy agent" implementation which validates the use of resource reservation to address scenarios vulnerable to opportunity cost. Finally, the effectiveness of this resource allocation framework with reservation is extended to environments where preemption is costly or impossible

Applying revenue management to agent-based transportation planning

We consider a multi-company, less-than-truckload, dynamic VRP based on the concept of multi-agent systems. We focus on the intelligence of one vehicle agent and especially on its bidding strategy. We address the problem how to price loads that are offered in real-time such that available capacity is used in the most profitable way taking into account possible future revenues. We develop methods to price loads dynamically based on revenue management concepts.\ud We consider a one leg problem, i.e., a vehicle travels from i to j and can wait at most τ time units in which it can get additional loads from i to j. We develop a DP to price loads given a certain amount of remaining capacity and an expected number of auctions in the time-to-go. Because a DP might be impractical if parameters change frequently and bids has to be determined in real-time, we derived two approximations to speed up calculations. The performance of these approximations are compared with the performance of the DP. Besides we introduce a new measure to calculate the average vehicle utilisation in consolidated shipments. This measure can be calculated based on a limited amount of data and gives an indication of the efficiency of schedules and the performance of vehicles

Learning Scheduling Algorithms for Data Processing Clusters

Efficiently scheduling data processing jobs on distributed compute clusters requires complex algorithms. Current systems, however, use simple generalized heuristics and ignore workload characteristics, since developing and tuning a scheduling policy for each workload is infeasible. In this paper, we show that modern machine learning techniques can generate highly-efficient policies automatically. Decima uses reinforcement learning (RL) and neural networks to learn workload-specific scheduling algorithms without any human instruction beyond a high-level objective such as minimizing average job completion time. Off-the-shelf RL techniques, however, cannot handle the complexity and scale of the scheduling problem. To build Decima, we had to develop new representations for jobs' dependency graphs, design scalable RL models, and invent RL training methods for dealing with continuous stochastic job arrivals. Our prototype integration with Spark on a 25-node cluster shows that Decima improves the average job completion time over hand-tuned scheduling heuristics by at least 21%, achieving up to 2x improvement during periods of high cluster load

A Framework for Resource Allocation in Time Critical Dynamic Environments Based on Social Welfare and Local Search and its Application to Healthcare

This thesis provides an artificial intelligence approach for the problem of resource allocation in time-critical dynamic environments. Motivated by healthcare scenarios such as mass casualty incidents, we are concerned with making effective decisions about allocating to patients the limited resources of ambulances, doctors and other medical staff members, in real-time, under changing circumstances. We cover two distinct stages: the Ambulance stage (at the location of the incident) and the Hospital stage (where the patient requires treatment). Our work addresses both determining the best allocation and supporting decision making (for medical staff to explore possible options). Our approach uses local search with social welfare functions in order to find the best allocations, making use of a centralized tracking of patients and resources. We also clarify how sensing can assist in updating the central system with new information. A key concept in our solution is that of a policy that attempts to minimize cost and maximize utility. To confirm the value of our approach, we present a series of detailed simulations of ambulance and hospital scenarios, and compare algorithms with competing principles of allocation (e.g. sickest first) and societal preferences (e.g. egalitarian allotment). In all, we offer a novel direction for resource allocation that is principled and that offers quantifiable feedback for professionals who are engaged in making resource allocation decisions

Interaction between intelligent agent strategies for real-time transportation planning

In this paper we study the real-time scheduling of time-sensitive full truckload pickup-and-delivery jobs. The problem involves the allocation of jobs to a fixed set of vehicles which might belong to dfferent collaborating transportation agencies. A recently proposed solution methodology for this problem is the use of a multi-agent system where shipper agents other jobs through sequential auctions and vehicle agents bid on these jobs. In this paper we consider such a multi-agent system where both the vehicle agents and the shipper agents are using profit maximizing look-ahead strategies. Our main contribution is that we study the interrelation of these strategies and their impact on the system-wide logistical costs. From our simulation results, we conclude that the system-wide logistical costs (i) are always reduced by using the look-ahead policies instead of a myopic policy (10-20%) and (ii) the joint effect of two look-ahead policies is larger than the effect of an individual policy. To provide an indication of the savings that might be realized with a central solution methodology, we benchmark our results against an integer programming approach

Look-ahead strategies for dynamic pickup and delivery problems

In this paper we consider a dynamic full truckload pickup and delivery problem with time-windows. Jobs arrive over time and are offered in a second-price auction. Individual vehicles bid on these jobs and maintain a schedule of the jobs they have won. We propose a pricing and scheduling strategy based on dynamic programming where not only the direct costs of a job insertion are taken into account, but also the impact on future opportunities. Simulation is used to evaluate the benefits of pricing opportunities compared to simple pricing strategies in various market settings. Numerical results show that the proposed approach provides high quality solutions, in terms of profits, capacity utilization, and delivery reliability

Frontiers In Operations Research For Overcoming Barriers To Vehicle Electrification

Electric vehicles (EVs) hold many promises including diversification of the transportation energy feedstock and reduction of greenhouse gas and other emissions. However, achieving large-scale adoption of EVs presents a number of challenges resulting from a current lack of supporting infrastructure and difficulties in overcoming technological barriers. This dissertation addresses some of these challenges by contributing to the advancement of theories in the areas of network optimization and mechanism design. To increase the electric driving range of plug-in hybrid electric vehicles (PHEVs), we propose a powertrain energy management control system that exploits energy efficiency dif- ferences of the electric machine and the internal combustion engine during route planning. We introduce the Energy-Efficient Routing problem (EERP) for PHEVs, and formulate this problem as a new class of the shortest path problem. We prove that the EERP is NP-complete. We then propose two exact algorithms that find optimal solutions by exploiting the transitive structure inherent in the network. To tackle the intractability of the problem, we proposed a Fully Polynomial Time Approximation Scheme (FPTAS). From a theoretic perspective, the proposed two-phase approaches improve the state-of-the-art to optimally solving shortest path problems on general constrained multi-graph networks. These novel approaches are scalable and offer broad potential in many network optimization problems. In the context of vehicle routing, this is the first study to take into account energy efficiency difference of different operating modes of PHEVs during route planning, which is a high level powertrain energy management procedure. Another challenge for EV adoption is the inefficiency of current charging systems. In addition, high electricity consumption rates of EVs during charging make the load manage- ment of micro grids a challenge. We proposed an offline optimal mechanism for scheduling and pricing of electric vehicle charging considering incentives of both EV owners and utility companies. In the offline setting, information about future supply and demand is known to the scheduler. By considering uncertainty about future demand, we then designed a family of online mechanisms for real-time scheduling of EV charging. A fundamental problem with significant economic implications is how to price the charging units at different times under dynamic demand. We propose novel bidding based mechanisms for online scheduling and pricing of electric vehicle charging. The proposed preemption-aware charging mechanisms consider incentives of both EV drivers and grid operators. We also prove incentive-compatibility of the mechanisms, that is, truthful reporting is a dominant strategy for self-interested EV drivers. The proposed mechanisms demonstrate the benefits of electric grid load management, revenue maximization, and quick response, key attributes when providing online charging services