Comparison of agent-based scheduling to look-ahead heuristics for real-time transportation problems

We consider the real-time scheduling of full truckload transportation orders with time windows that arrive during schedule execution. Because a fast scheduling method is required, look-ahead heuristics are traditionally used to solve these kinds of problems. As an alternative, we introduce an agent-based approach where intelligent vehicle agents schedule their own routes. They interact with job agents, who strive for minimum transportation costs, using a Vickrey auction for each incoming order. This approach offers several advantages: it is fast, requires relatively little information and facilitates easy schedule adjustments in reaction to information updates. We compare the agent-based approach to more traditional hierarchical heuristics in an extensive simulation experiment. We find that a properly designed multiagent approach performs as good as or even better than traditional methods. Particularly, the multi-agent approach yields less empty miles and a more stable service level

Agent-based transportation planning compared with scheduling heuristics

Here we consider the problem of dynamically assigning vehicles to transportation orders that have di¤erent time windows and should be handled in real time. We introduce a new agent-based system for the planning and scheduling of these transportation networks. Intelligent vehicle agents schedule their own routes. They interact with job agents, who strive for minimum transportation costs, using a Vickrey auction for each incoming order. We use simulation to compare the on-time delivery percentage and the vehicle utilization of an agent-based planning system to a traditional system based on OR heuristics (look-ahead rules, serial scheduling). Numerical experiments show that a properly designed multi-agent system may perform as good as or even better than traditional methods

A decision model for natural oil buying policy under uncertainty

A manufacturer, in a fast moving consumer goods industry, buys Natural oils from a number of oil suppliers world-wide. The prices of these oils are the major raw material cost in producing the consumer goods, which are also sold world-wide. The volatility in the international prices of the Natural oils has signi¯cant impact on the planning and budgets decisions. Since the oils are bought and the ¯nished products are sold in markets throughout the world, the manufacturer is exposed to a variety of market uncertainties and the resulting risks. These uncertainties are the raw material prices, the demand and the therefore the selling prices for the finished goods- all of which influence the profitability of the manufacturing firm. The risks can be minimised by entering into futures contract of appropriate duration, that is, by following a schedule of "forward"' purchase of oil (with specific series of future delivery dates) with the oil suppliers. We formulate this problem as a two-stage Stochastic Program (SP) using the futures and the spot prices for the Natural oil. This SP model gives robust decisions that hedge against the uncertainties in the Natural oil prices and the demand for the finished products. The uncertainty in the oil prices and the demand are modelled through a scenario generator. We have constructed a decision support system (DSS) that integrates the SP model, the scenario generator and the solution algorithm. This DSS also provides the decision maker a profile of the risk and return exposures for different policies

Interaction between intelligent agent strategies for real-time transportation planning

In this paper we study the real-time scheduling of time-sensitive full truckload pickup-and-delivery jobs. The problem involves the allocation of jobs to a fixed set of vehicles which might belong to dfferent collaborating transportation agencies. A recently proposed solution methodology for this problem is the use of a multi-agent system where shipper agents other jobs through sequential auctions and vehicle agents bid on these jobs. In this paper we consider such a multi-agent system where both the vehicle agents and the shipper agents are using profit maximizing look-ahead strategies. Our main contribution is that we study the interrelation of these strategies and their impact on the system-wide logistical costs. From our simulation results, we conclude that the system-wide logistical costs (i) are always reduced by using the look-ahead policies instead of a myopic policy (10-20%) and (ii) the joint effect of two look-ahead policies is larger than the effect of an individual policy. To provide an indication of the savings that might be realized with a central solution methodology, we benchmark our results against an integer programming approach

Rescheduling frequency in an FMS with uncertain processing times and unreliable machines

Cataloged from PDF version of article.This paper studies the scheduling/rescheduling problem in a multi-resource FMS environment. Several reactive scheduling policies are proposed to address the effects of machine breakdowns and processing time variations. Both off-line and on-line scheduling methods are tested under a variety of experimental conditions. The performance of the system is measured for mean tardiness and makespan criteria. The relationships between scheduling frequency and other scheduling factors are investigated. The results indicated that a periodic response with an appropriate period length would be sufficient to cope with interruptions. It was also observed that machine breakdowns have more significant impact on the system performance than processing time variations. In addition, dispatching rules were found to be more robust to interruptions than the optimum-seeking off-line scheduling algorithm. A comprehensive bibliography is also included in the paper

Comparing policy gradient and value function based reinforcement learning methods in simulated electrical power trade

In electrical power engineering, reinforcement learning algorithms can be used to model the strategies of electricity market participants. However, traditional value function based reinforcement learning algorithms suffer from convergence issues when used with value function approximators. Function approximation is required in this domain to capture the characteristics of the complex and continuous multivariate problem space. The contribution of this paper is the comparison of policy gradient reinforcement learning methods, using artificial neural networks for policy function approximation, with traditional value function based methods in simulations of electricity trade. The methods are compared using an AC optimal power flow based power exchange auction market model and a reference electric power system model

Dynamic threshold policy for delaying and breaking commitments in transportation auctions

In this paper we consider a transportation procurement auction consisting of shippers and carriers. Shippers offer time sensitive pickup and delivery jobs and carriers bid on these jobs. We focus on revenue maximizing strategies for shippers in sequential auctions. For this purpose we propose two strategies, namely delaying and breaking commitments. The idea of delaying commitments is that a shipper will not agree with the best bid whenever it is above a certain reserve price. The idea of breaking commitments is that the shipper allows the carriers to break commitments against certain penalties. The benefits of both strategies are evaluated with simulation. In addition we provide insight in the distribution of the lowest bid, which is estimated by the shippers

Vehicle Dispatching and Routing of On-Demand Intercity Ride-Pooling Services: A Multi-Agent Hierarchical Reinforcement Learning Approach

The integrated development of city clusters has given rise to an increasing demand for intercity travel. Intercity ride-pooling service exhibits considerable potential in upgrading traditional intercity bus services by implementing demand-responsive enhancements. Nevertheless, its online operations suffer the inherent complexities due to the coupling of vehicle resource allocation among cities and pooled-ride vehicle routing. To tackle these challenges, this study proposes a two-level framework designed to facilitate online fleet management. Specifically, a novel multi-agent feudal reinforcement learning model is proposed at the upper level of the framework to cooperatively assign idle vehicles to different intercity lines, while the lower level updates the routes of vehicles using an adaptive large neighborhood search heuristic. Numerical studies based on the realistic dataset of Xiamen and its surrounding cities in China show that the proposed framework effectively mitigates the supply and demand imbalances, and achieves significant improvement in both the average daily system profit and order fulfillment ratio